- Richard Feynman -
Assignment:Exposing data over http
In this three-part assignment you will create an open web API for publishing data from the Systembolaget product line. The web API will allow for searches via HTTP/REST. It is extremely important that you read this page and make sure you understand the goal of the three-part assignment. Also read up on the technology listed below.
Background and goals of the assignment
The Swedish alcohol monopoly Systembolaget publishes an "API" with all their products in their product line. The "API" consists of a large (huge, really) XML file. The XML file is updated every day. While this is helpful for developers, the problem is that in order to create an app accessing the product catalog, the developer needs to download the complete XML file. The XML file from Systembolaget can be found here.
The idea behind this three-part assignment, is to create a web service which allows us to query for products according to some criteria. We haven't found such a web API with Systembolaget, so let's create one ourselves! When we have such a web API, we can build apps and services around it, and even allow others to access the data in this new way. If you plan to take an Android course later, you may very well write an app fetching data from your own web API.
We have provided a database with the products from the XML file for you. Optimally, this database should be updated every day, but we'll disregard that in this assignment. So your task boils down to writing the web backend for the data and writing some GUI which will connect to the backend and fetch data from it using various queries.
Preparing yourself for the assignments
The three sub assignments will cover a whole lot of acronyms and techie buzzwords. Therefore, we strongly encourage you to read up on and orient yourselves on the technologies involved. Below is a list of resources explaining some of the terms involved:
- API (wikipedia) (note the different kinds of APIs)
- REST: (wikipedia)
- HTTP: (tutorialspoint)
- Java Swing (wikipedia)
- JDBC (Java Database Connectivity) (wikipedia)
Each of the sub assignments will explain the application of various technologies further, as will the lectures and workshops accompanying these assignments.
Strategy for the web API
A REST web API works as a web server, listening for request and responding with data formatted in some way. It is common to use some standard format for the data, such as JSON or XML. In this assignment, we'll settle for JSON, because it sounds modern and nice.
So what about the actual data, then? Well, this assignment is given as part of the university course "Programmeringsteknik och databaser" (Programming and databases), so why not program against a database? We have fetched the XML file with the products, and inserted (most of) the data into a database for you. It is possible to use this assignment in other courses too, but it was produced with this particular course in mind, so a lot of references will be tied to that course in the documentation and descriptions found here.
So, given a database with all the products, you will write the web API using Java Servlet technology. But as you are learning about SQL and databases in the course, we will start with the backend without a real connection to the database. It is not uncommon to start working on a part of a system before all parts are ready. We'll instruct you to create a fake database access (which simply pretends to fetch data from somewhere and returns a list of product objects as if they really were fetched from the database). Then you can focus on making sure the web API works (and don't care where the actual data comes from).
There are various ways to test the web API which we will show you. All this will be done in the first sub assignment, Assignment:Exposing_data_over_http_lab1_Web_API (in the source code referred to as simply "Lab1").
In the second sub assignment, Assignment:Exposing_data_over_http_lab2_GUI_Client (in the source code referred to as simply "Lab2"), you will create a small GUI client using Java Swing (in later editions of this assignment we will move on to JavaFX but this year we'll settle with Swing - apologies for that). The idea of this GUI client is to create an app with a graphical user interface which lets the user search for (or, rather, give filter criteria) products and display the products satisfying the filter criteria in a table-like view. Of course, this GUI client will later get the data from the web API, but in the second sub assignment, we'll again create fake data. This time, we'll fake data so that we don't have to focus on networking and parsing JSON (JSON is the format of the data we'll get from the web API). We want to show you that you can write a GUI which gets products from somewhere and displays them in some kind of table component. When the GUI is functioning, layout-wise, we will also show you that you can create the correct query strings to later send to the web API. This means that you can test your GUI and verify that it creates the correct query strings based on the user's input in the criteria input text fields.
Finally, we'll tie everything together in the third sub assignment, Assignment:Exposing_data_over_http_lab3_integrate_the_subsystems. In this third, and last, sub assignment, you'll switch out the fake data component in the web API and replace it with data from the database, using JDBC. At this point, if you have followed the course's database lectures, and exercises, you know enough SQL to create a query for fetching all product data from the database. If you have followed our instructions for lab1, then the switch will be really easy and no code will have to be changed in the actual Servlet. You will get to focus on writing the database component (but don't worry, we'll provide a lot of code to get you started).
On the client-side, we'll switch from using fake data to creating a component which actually uses the query string produced by the GUI and connects to the web API and requests products according to the query criteria (e.g.
min_price=100&max_price=200&min_alcohol=50). Since you have finished and tested the GUI in lab2, we can here focus on how to connect to a web API from a Java application, read the data and convert the JSON string to Java objects (which will be presented in the GUI).
If we have the time, we'll also demonstrate an Android app, which will do the same thing; present the user with options for specifying some criteria, fetch JSON from the web API and present it in some tabular widget in the app.
Why did we design the assignment this way?
We wanted to give you a three-part assignment which involves four parts:
- Java programming which is a little more complicated than the introduction course
- Database technology - A database which you can fetch data from using SQL
- Java programming for a GUI application
- Database connectivity - Connecting a Java application to a database
Other than that, we wanted the assignment to cover these features:
- Data in the database should be real data, and not some made up boring stuff
- Data should be exposed from the database to applications in some standard way
- APIs at different levels should be used
- Application frameworks (like JDBC, Swing, An API which you write in part yourselves, e.g. for handling Product objects)
- A web API which lets clients query for data matching some criteria and responds with JSON over HTTP
- A description of a larger system overview
- some design patterns
- system integration (e.g. using JSON as the data format between systems, JDBC between an application and a database)
- Something you can keep after the course and use in future courses or projects
- Some buzzword compliance so that you can say that you have worked with REST, HTTP, Servlets, JSON etc
- A system with a few subsystems distributed and networked with a larger scale than the typical school exercise in programming
- Something that resembles a real-world application with client-server or frontend/backend
- Data transfer using some standard data format like JSON
What to do now
OK, so how do we get started? We suggest that you start with lab1, Assignment:Exposing_data_over_http_lab1_Web_API and get the web API functional. Then you can move on to lab2, Assignment:Exposing_data_over_http_lab2_GUI_Client, and after that, finally wrap it up in lab3 Assignment:Exposing_data_over_http_lab3_integrate_the_subsystems.
Good luck and happy reading!
// Rikard and Henrik