View source for Assignment:Exposing data over http lab3 integrate the subsystems

=Introduction=
{{#ev:vimeo|249091769}}

The source code with examples etc for this lab 3 can be found [https://github.com/progund/exposing-data-lab-final/archive/master.zip here] (one big zip)

=Prerequisites - introductory workshop etc=
Before starting with this assignment, you should attend the workshop for Lab3, the manuscript for which can be found [[Workshop Lab3 - Tying it together| here]].

During the course, we'll have more workshops (work in progress) which will provide pieces of the puzzle for you!

List of workshops (see course schedule for when to attend):
* [[Workshop Lab3 - Tying it together]]
* [[Workshop Lab3 - Fetching JSON and Parsing JSON]]
* [[Workshop_Lab3_-_Servlet_with_JDBC]]

=Lab 3 - Integrate the subsystems=
In this third and last sub-assignment, we'll connect the dots between labs 1 and 2. This sub-assignment has two goals:
* Your web API (the servlet etc) should use a database with all the products (i.e. "real data")
** All products will be read into the servlet once by a <code>ProductLine</code> object, so all code in the servlet will be unchanged
** You will change the configuration of the servlet (the <code>web.xml</code> configuration file) so that the ProductLineFactory knows that it should use a database version of the ProductStore
** You will write code that talks to the database (SQL) for reading all products into memory using JDBC
* Your GUI (the Swing application) should get its data from the web API and get new data when the user enters filter criteria (e.g. min price = 20, max price = 30 etc)
** You will write code in an ApiAccess implementation which talks to the servlet over the network
** You will investigate and work with the code which translates the JSON data from the servlet to Java Product objects
Technologies which are the focus of this sub assignment include:
* JDBC - connecting to a database from Java
* SQL - writing the SQL SELECT statement for reading all products and product group information
* Servlet configuration (e.g. changing web.xml so that the system uses a different data source - database instead of fake products)
* HTTP - How to connect to e.g. a Servlet from a Java program (your Swing client)
* JSON - How to parse (translate JSON to Java objects)
==Videos, lectures etc required before starting==
You will need to learn about two things before starting: JDBC and parsing JSON in Java. You need to know about JDBC in order to write the server-side database connectivity stuff (the servlet will now read all products from the database instead of using the FakeProductLine!). Here are the chapters (including) video lectures for the JDBC stuff:

Read, do exercises and see all video lectures linked from the following chapters (these lectures are also in the schedule, and given in the classroom!):
* [[JavaDB:Introduction to JDBC|Introduction to JDBC]]
* [[JavaDB:Exercise - Introduction to JDBC|Exercise - Introduction to JDBC]]
* [[JavaDB:JDBC-INSERT-UPDATE-SQLInjection|JDBC-INSERT-UPDATE-SQLInjection]]
* [[JavaDB:Exercise - JDBC-INSERT-UPDATE-SQLInjection|Exercise - JDBC-INSERT-UPDATE-SQLInjection]]
* [[JavaDB:PreparedStatement|PreparedStatement]]
* [[JavaDB:Exercise - PreparedStatement|Exercise - PreparedStatement]]
* [[JavaDB:Commit - Rollback|Commit - Rollback]]
* [[JavaDB:Exercise - Commit - Rollback|Exercise - Commit - Rollback]]
* [[JavaDB:Assignment - SQL injection|Assignment - SQL injection]]
* [[JavaDB:Introduction to layered architectures|Introduction to layered architectures]]
* [[JavaDB:Exercises - Introduction to layered architectures|Exercises - Introduction to layered architectures]]

You need to know how to convert the JSON String you get from the servlet in the GUI to a <code>List<Product></code>. Here are the videos for parsing JSON you should see before starting with the client-side (GUI) (these are self-studies! Not given in the classroom):
Read (exercises, videos etc):
* Wikipage: [[Java-Web:Introduction_to_JSON|Introduction_to_JSON]]
** Wiki exercises: [[Java-Web:Exercises_-_Introduction_to_JSON|Exercises_-_Introduction_to_JSON]] '''''Note:''' Focus on org.json'' 
Watch:
* [https://vimeo.com/249091769 Introduction to Lab3 (embedded above on this page)]
* [https://vimeo.com/channels/1350886 JSON - Introduction] - Lecture slides: [[:Media:Introduction to JSON.pdf|Introduction to JSON.pdf]] (PDF)
** [https://vimeo.com/256960568 JSON - Introduction - 1]
** [https://vimeo.com/256960602 JSON - Introduction - 2]
** [https://vimeo.com/256960587 JSON - Introduction - 3]
* Videokanal: [https://vimeo.com/channels/1188518 Introduktion till JSON och introduktion till parsing JSON i Java] (Swe)  - Presentation: [[:Media:Introduction to JSON.pdf|Introduction to JSON.pdf]] (Eng)
** [https://vimeo.com/199675038 Introduktion till JSON del 1 av 2]
** [https://vimeo.com/199675024 Introduktion till JSON del 2 av 2]
** [https://vimeo.com/199675009 Parse:a JSON från Java del 1/3]
** [https://vimeo.com/199674992 Parse:a JSON från Java del 2/3]
** [https://vimeo.com/199674979 Parse:a JSON från Java del 3/3]
* Video channel: [https://vimeo.com/channels/1333865 Java - Parsing with org.json]
** [[:Media:Org.json_-_parsing.pdf|Org.json_-_parsing.pdf]]
** Source code for the lecture: [https://github.com/progund/java-web/tree/master/java-json/org.json/parsing github.com/progund/java-web/tree/master/java-json/org.json/parsing]

== Software introduction ==
=== Server-side===
The part on the server-side that you will work on in this last lab, is the <code>SQLBasedProductLine</code> class. It is an implementation which reads all products from a database, rather than creating fake products as the one you worked on in Lab 1. The <code>ProductLineFactory</code> and the <code>SQLBasedProductLine</code> are part of the <code>systemet-api</code>, so you will need to re-deploy the systemet.jar after doing the changes.

We have provided a helper class, <code>[https://raw.githubusercontent.com/progund/exposing-data-lab-final/master/server-side/systemet-api/se/itu/systemet/storage/DBHelper.java DBHelper]</code> to file! which talks to the database, but it is your task to identify what SQL String with a <code>SELECT</code> for all products and product groups it uses to get a <code>ResultSet</code> back, which you will use to create <code>Product</code> objects and put in a <code>List&lt;Product&gt;</code>. You must describe this SELECT statement in your report and diary.

A take-away from this part of the lab, is that you will discover that the actual Servlet doesn't need any code changes, since we have separated the ProductLine from the Servlet code, using a factory and interfaces.

So, how does the Servlet get products from your new SQLBasedProductLine, then? You will change a configuration file for the Servlet, and the ProductLineFactory will discover that it should use your SQLBasedProductLine, and not the FakeProductLine from the Lab 1. The way we have written the ProductLineFactory, makes it look at a Java system property called "ProductLine", and let that property decide which type of ProductLine to create.

* Servlet configuration sets the type of product line to use (DB or Fake)
* The Servlet reads its configuration and sets the property ProductLine
* The Servlet uses a factory from the systemet-api to get a ProductLine
* The factory reads the property and decides which class to use when creating and returning the ProductLine

===Client-side===
The part of the client-side that you will work on (the Swing client), is the way the GUI gets product using an ApiAccess object. You will change the factory for the ApiAccess object creation, so that it uses an ApiAccess object of type <code>SystemetRestApiAccess</code> (instead of using the <code>FakeApiAccess</code> as was done in lab 2) which will actually talk to the Servlet over HTTP, and get JSON back. You will also work on the part of that class, which parses the JSON into a list of Product objects (parsing could here be thought of as "translating the JSON string from the Servlet to a Java List&lt;Product&gt;"). You have to write the class <code>SystemetRestApiAccess</code>.

==Directory overview==
===Server-side===
You will continue with the code you wrote in Lab 1, but you need some extra files from the teachers. See below.
===Client-side===
You will continue with the code you wrote in Lab 2.
==Server-side task 1 - Change the Servlet configuration file==
The file to change for the configuration of the Servlet is the <code>web.xml</code> which you can find in:

<code>webroot/WEB-INF/web.xml</code>

Change the following:
<source lang="XML" highlight="2">
<param-name>ProductLine</param-name>
<param-value>Fake</param-value>
</source>
to:
<source lang="XML" highlight="2">
<param-name>ProductLine</param-name>
<param-value>DB</param-value>
</source>

==Server-side task 2 make the SQLBasedProductLine do something so that you can compile and test==
The next step, is to make sure that the changes you made to the servlet configuration took effect, and works.

In order to test this, you must make the <code>SQLBasedProductLine</code> class return something in its <code>getAllProducts()</code> method, and see that it shows up in the JSON when requesting all products from the web API.

This is our suggestion:
# getAllProducts() in SQLBasedProductLine checks if products is null
## If so (it is, since the class hasn't initialized products yet), it will call readProductsFromDatabase()
# readProductsFromDatabase() is empty, so we need to add code here!
# initialize the products List to a <code>new ArrayList&lt;&gt;</code>
# add one single product to the list, with a name of "Testing SQLBasedProductLine"
# run the deployment script (which will re-compile the systemet-api, create a jar file and put that jar file so that the servlet will use it)
# request all methods via e.g. a browser (you may use http://localhost:8080/search/products/all as the url)

If the product showed up, you have verified that your configuration change worked, and the Servlet web API is using your SQLBasedProductLine and you may move on to Part 3.

==Server-side task 3 - Figure out the SQL SELECT for getting all products==
You must now figure out what SQL SELECT String our DBHelper class is using, so that you can describe how it gets the ResultSet to send back. In order to pass this assignment, you need to provide your own working SQL SELECT statement with a [[Database:Combining_rows_of_data_from_related_tables_-_SQL_JOIN|JOIN]], to prove that you have understood this part of the system. Look at the DBHelper code in order to figure out the SQL, but even better, read on and investigate the database!

The database can be found in <code>webroot/WEB-INF/db/bolaget.db</code> .

We suggest that you open a session in SQLite3 for that database and investigate the tables and schemas:

<source lang="bash">
$ sqlite3 webroot/WEB-INF/db/bolaget.db
SQLite version 3.11.0 2016-02-15 17:29:24
Enter ".help" for usage hints.
sqlite>
</source>

As usual, you do all work from the root directory of the project (the directory above <code>webroot</code> ).

Investigate the tables and schemas:
<source lang="SQL">
.schema
CREATE TABLE productGroup(id INT PRIMARY KEY NOT NULL, name text);
CREATE TABLE product(nr INT primary key not null, name text, price REAL, alcohol REAL, volume INT, productGroupId INT, type text, foreign key(productGroupId) references productGroup(id));
</source>

From the above, we can see that luckily, there are only two tables. One with most product information, and one with the product groups. The column <code>product.productGroupId</code> is a foreign key to the <code>productGroup</code> table on its primary key <code>productGroup.id</code>.

You should JOIN these tables on those columns, then. We suggest that you try formulating a JOIN which works, but add <code>LIMIT 10</code> to the end of the query, so that you won't get the full 20 000 product database as a result.

Start with something simple to see that only the JOIN works, like selecting <code>product.name</code>, <code>product.price</code>, <code>product group</code> for the first 10 products.

* Note that there is a column <code>name</code> in both tables - you need to use aliases or table.column syntax for the database to understand how to differentiate them
* Note that there is no column named "product group" so you need to create an alias for <code>productGroup.name</code> if you want that column header in your result

When you have a working SELECT with a correct JOIN, you can move on to the next task.

Here is a sample result from the database (where we have ordered the result on the <code>nr</code> column):
<pre>
name                  price       product group       
--------------------  ----------  --------------------
Renat                 204.0       Okryddad sprit      
Renat                 117.0       Okryddad sprit      
Explorer Vodka        108.0       Okryddad sprit      
Kronvodka             226.0       Okryddad sprit      
Kronvodka             126.0       Okryddad sprit      
Hallands Fläder       249.0       Kryddad sprit       
Hallands Fläder       145.0       Kryddad sprit       
Los Tres Toños Repo   419.0       Tequila och Mezcal  
Vanlig Vodka          199.0       Okryddad sprit      
Motörhead Vödka       299.0       Okryddad sprit  
</pre>

Now, look carefully at the JSON for one product (which you probably remember from lab 1):
<source lang="JavaScript">
  {
    "name": "Johanneshof Reinisch Pinot Noir",
    "price": 143.00,
    "volume": 750,
    "alcohol": 13.00,
    "nr": 7440201,
    "product_group": "Rött vin"
  }
</source>

From this JSON object, reverse engineer (figure out by thinking backwards) the full SQL SELECT with the JOIN, to use to get all those fields. Include it in your report.

Explain also how the code in Java must handle the fact that we have two columns named <code>"name"</code>. 

'''''Hint:''' the methods in ResultSet allow you to get values for a column based on the order in which they occur. This will be good to know in the next task! ''

'''''Hint:''' The following doesn't work, because you cannot have two columns with the same name in a JOIN: <code>SELECT name, name FROM product JOIN productgroup ON product.productgroupid = productgroup.id;</code> - how would SQLite3 know that you mean "name" from product and "name" from productgroup? Using aliases would not help, since SQLite3 still will complain about the fact that both tables have a column called "name". So we need to qualify the column name somehow. Look at the code of the SELECT to see how the columns are qualified.''

'''''Extra:''' Many students found this task confusing, so we created this extra lecture: [[:Media:Column names and JDBC.pdf|Column names and JDBC]] - video version is on our TODO-list''

==Server-side task 4 - Using DBHelper, get a ResultSet back and populate the products list==
Now that you know what SQL to use, you can make the <code>readProductsFromDatabase()</code> method get a ResultSet from the DBHelper class. The DBHelper class will do the actual JDBC stuff, so that you don't have to focus on that in this lab, but you need to understand the contents and structure of that ResultSet. Review the [[JavaDB:Introduction_to_JDBC|Introduction_to_JDBC]] chapter for a recap of how to use ResultSet.

Here's an overview:

* The nested static class DBHelper.ColumnId contains constants with the indices for the various columns in the ResultSet
* You may use these constants when you get the values from the seven (7) columns in the ResultSet
** Getting the column number for e.g. the price, you should use <code>DBHelper.ColumnId.PRICE</code> (which happens to be the forth column and thus get you an int with the value 4)

Investigate the <code>se.itu.systemet.storage.DBHelper</code> class. You will find that it defines only one public method:

<source lang="Java">
public static ResultSet productsResultSet()
</source> 

This is the one to call from your <code>readProductsFromDatabase()</code> method.

Strategy for looping through the ResultSet and creating a Product to add to the products list:
* create a while loop on rs.next() (if your ResultSet is called rs)
** declare variables of the correct type (see the documentation for the Product class!) for all columns (don't forget the productGroup)
** use the ResultSet to get the values of the correct type and assign to the variables (use the constants in DBHelper.ColumnId)
** use a Product.Builder to set the values, and create a product (using the builder's build() method when all fields are set)
** add this product to the products list

Note that <code>readProductsFromDatabase()</code> is declared <code>void</code> so it can't and shouldn't return anything. It will populate the products list as a [[Term:side-effect|side-effect]] from calling it from the <code>getAllProducts()</code> method.

In order to test your new code in SQLBasedProductLine, you can use the test program we have included in <code>[https://raw.githubusercontent.com/progund/exposing-data-lab-final/master/server-side/systemet-api/examples/ApiExample.java systemet-api/examples/ApiExample.java]</code>. Look at the code before you run it, so that you know what to expect should happen.

To run it, use our script compile_and_run_api_example.sh (also included in your zip file which you downloaded earlier).

Now that you have tested that the example program can get an SQLBasedProductLine, you can continue with testing that also the servlet can get an SQLBasedProduct line. Deploy the jar-file using the script and restart the servlet. Do you get all the products? Does the filtering work?

==Server-side optional - Challenge - Not mandatory!!==
If you really want a challenge, (not that it's ''that'' hard), you can write your own DBHelper class, which opens a Connection to the database, creates a Statement and executes a custom SQL query, in order to get the ResultSet with all the products.

Call some method in your own implementation (of your own DBHelper class) in order to get the ResultSet to the SQLBasedProductLine class.

As usual, don't start with this challenge (if you intend to give it a go), until you have saved a working solution from the obligatory tasks above. We don't want you to end up with something that isn't finished, or doesn't work, because you started on the challenge.

Also keep in mind, that you have a part 2 of this assignment below, the GUI part to connect to the Servlet over the network.

You can create a copy of your working code and start the challenge in a new directory, so that you still have your working solution to submit to the teachers.

'''''Note:''' For those of you who take this assignment as part of the TIG058 university course: We think that this is a good challenge for you who are finished in time, because it gives you good exercise if you plan to take the written exam about databases (which includes a JDBC section).''

We also suggest as an optional voluntary challenge to write the Client class and ProductParser class from scratch, and not just copy the code from the example.
===Super-hard challenge===
If you still think this assignment is too easy, even after doing the challenges in Lab1, Lab2 and above, you can also write code for updating the database from the Systembolaget XML file.

Your program should fetch the XML file from Systembolaget (google for the URL), and parse the XML to Java. Then either insert all products using JDBC to the database (you can delete all rows from product and productGroup first), or your program can output INSERT statements to stdout, which you can redirect to an SQL file which you can manually run on your database.

While this is not so hard (we have chapters and lectures about parsing XML - use the search box on our wiki, or look at the More programming with Java book) it contains one part which is rather tricky.

The product table has a foreign key with an id referring to the productGroup table. So each product has a productGroup id. This means that in order to know what ID to use when you insert a product to the product table, you need to first make sure the product's product group exists in the productGroup table and if so use the corresponding id, otherwise insert the product group into the productGroup table and then use the id for the new product group!

Don't start on this challenge if you don't have time (or interest) to learn about parsing XML. 

The cool thing about this challenge is that your web API will contain fresh product data - you can update the database with toda's XML every day if you feel like it.

=Client-side tasks=
==Client-side task 1 - Running the example which connects to the Web API==
We have written an example program for you, which connects to the Web API and gets a response with JSON.

You need to download these examples and put them in your <code>examples/</code> directory from Lab 2. See links below.

To compile and run the example, you need to put org.json.jar in your classpath using the <code>-cp</code> flag. The class path becomes:
* On Windows: <code>".;lib/org.json.jar"</code> (with semicolon)
* On Mac/Linux: <code>.:lib/org.json.jar</code> (with normal colon)
The command line to compile the example becomes:
* On Windows: <code>javac -cp ".;lib/org.json.jar" examples/ClientExample.java</code>
* On Mac/linux: <code>javac -cp .:lib/org.json.jar examples/ClientExample.java</code>
The command line to run the example becomes:
* On Windows: <code>java -cp ".;lib/org.json.jar" examples.ClientExample</code>
* On Mac/linux: <code>java -cp .:lib/org.json.jar examples.ClientExample</code>
===Running the example program===
The example program can be run without arguments, in which case it will fetch the JSON for all products, and report how many products were found, or it can be run with an argument of a query string in double quotes, in which case it will send that query to the Web API, and if it gets less than 40 products back, they will be printed to standard out (again, the -cp flag is MAC/Linux style, add quotes and semicolon if you are on Windows).
====Using a full database on the Web API (server-side)====
If you run against a web api with connection to the full database, you should get the following:
<source lang="Bash" highlight="1,2,3,6-19,22-23">
$ javac -cp .:lib/org.json.jar examples/ClientExample.java
$ java -cp .:lib/org.json.jar examples.ClientExample 
Found: 20876 products.

$ javac -cp .:lib/org.json.jar examples/ClientExample.java
$ java -cp .:lib/org.json.jar examples.ClientExample "max_price=9"
Found: 12 products.
Stödbenscider, 4.50%, 330 ml, 8.00 SEK Cider (),, Product number: 8849103
Bear Beer Premium, 4.60%, 330 ml, 7.90 SEK Öl (),, Product number: 134815
Åbro Export, 5.30%, 330 ml, 8.40 SEK Öl (),, Product number: 1143812
Tuborg Grön, 4.20%, 330 ml, 8.90 SEK Öl (),, Product number: 125512
Fagerhult Export, 5.30%, 330 ml, 8.90 SEK Öl (),, Product number: 148112
Dansk Fadøl, 5.00%, 330 ml, 8.90 SEK Öl (),, Product number: 146012
Guldkällan, 4.80%, 330 ml, 8.50 SEK Öl (),, Product number: 147212
Svensk Starköl, 5.10%, 330 ml, 7.90 SEK Öl (),, Product number: 169912
Småland, 5.20%, 330 ml, 8.90 SEK Öl (),, Product number: 147712
Zeunerts Original, 5.30%, 330 ml, 8.90 SEK Öl (),, Product number: 142615
Sofiero Original, 5.20%, 330 ml, 8.90 SEK Öl (),, Product number: 122215
Fem komma tvåan, 5.20%, 330 ml, 7.90 SEK Öl (),, Product number: 162515

$ javac -cp .:lib/org.json.jar examples/ClientExample.java
$ java -cp .:lib/org.json.jar examples.ClientExample "max_price=20"
Found: 715 products.
</source>

====Using 20 fake products on the Web API (server-side)====
If you run against a web api with 20 fake products, you should get the following result:
<source lang="Bash" highlight="1,2,24-26">
$ java -cp .:lib/* examples.ClientExample 
Found: 20 products.
Johanneshof Reinisch Pinot Noir, 13.00%, 750 ml, 143.00 SEK Rött vin (),, Product number: 7440201
Dobogó Tokaji Furmint, 13.50%, 750 ml, 187.00 SEK Vitt vin (),, Product number: 7598701
Château de Cazeneuve Carignan, 13.50%, 750 ml, 250.00 SEK Rött vin (),, Product number: 7105201
Engelholms Pacific Pilsner, 5.00%, 330 ml, 21.00 SEK Öl (),, Product number: 3067603
Speyside Sherry Cask 21 Years, 55.10%, 700 ml, 1537.00 SEK Whisky (),, Product number: 8591801
Giró i Giró Montaner Brut Nature Gran Reserva, 11.50%, 750 ml, 151.00 SEK Mousserande vin (),, Product number: 7723101
Clynelish No 4051 17 Years, 55.00%, 700 ml, 1583.00 SEK Whisky (),, Product number: 8515601
Albarossa, 14.00%, 750 ml, 252.00 SEK Rött vin (),, Product number: 7321901
Los Frailes Monastrell Garnacha, 13.50%, 750 ml, 115.00 SEK Rött vin (),, Product number: 7444301
Auchentoshan Maltbarn Bourbon Cask 23 Years, 52.00%, 700 ml, 1735.00 SEK Whisky (),, Product number: 8537101
Albin Jacumin Châteauneuf-du-Pape, 14.50%, 750 ml, 246.00 SEK Rött vin (),, Product number: 7551101
Bibbiano Chianti Classico, 13.50%, 750 ml, 147.00 SEK Rött vin (),, Product number: 7539001
Chapter 7 Irish Whiskey Bourbon Hogshead 14 Years, 56.70%, 700 ml, 1113.00 SEK Whisky (),, Product number: 8771001
Amour de Deutz, 12.00%, 375 ml, 796.00 SEK Mousserande vin (),, Product number: 9602502
Deutz Exclusive Gift Box, 12.00%, 1125 ml, 2397.00 SEK Mousserande vin (),, Product number: 9696209
Villa Spinosa Valpolicella Classico, 12.00%, 750 ml, 130.00 SEK Rött vin (),, Product number: 7249001
D'Aria Sauvignon Blanc, 13.00%, 750 ml, 146.00 SEK Vitt vin (),, Product number: 7337501
Fairtransport Tres Hombres 21 Años, 41.60%, 700 ml, 885.00 SEK Rom (),, Product number: 8712101
Donatella Cinelli Colombini, 14.00%, 4500 ml, 1104.00 SEK Rött vin (),, Product number: 7475209
Abrigo Giovanni Piemonte Mix 1, 13.50%, 4500 ml, 732.00 SEK Rött vin (),, Product number: 7096809


$ java -cp .:lib/* examples.ClientExample "max_price=29"
Found: 1 products.
Engelholms Pacific Pilsner, 5.00%, 330 ml, 21.00 SEK Öl (),, Product number: 3067603
</source>
'''''Hint:''' You can use compile and run (if compilation succeeded) using the following syntax: 

<code>javac -cp .:lib/org.json.jar examples/ClientExample.java && java -cp .:lib/org.json.jar examples.ClientExample</code>.'' 

Your task is to study the example, so that you will get an idea on how you can implement something similar for your GUI.

Document in your individual report how the example works, and what techniques, libraries etc are used in it.

===Overview of the example source code===
The main class, <code>ClientExample</code>, does the following in its main method:
* Check if there are any arguments
** if so, set the <code>String query</code> to the argument, otherwise set it to <code>""</code>
* Create a <code>Client</code> object for accessing the Web API, passing it a URI including the query (if any) to its constructor
* Ask the client for its JSON response and store it in a <code>String JSON</code> variable
* Create a <code>ProductParser</code>
* Declare a list of <code>Product</code> and assign it the result of calling <code>parse(JSON)</code>
* Print out how many products were parsed from the JSON
* Check if less than 40 products were parsed
** if so, print each product to standard out
 
<source lang="Java">
  public static void main(String[] args) {
    String query = "";
    if (args.length == 1) {
      query = args[0];
    }
    
    Client client = new Client("http://localhost:8080/search/products/all?" + query);
    String JSON = client.getJson();
    ProductParser parser = new ProductParser();
    List<Product> products = parser.parse(JSON);
    System.out.println("Found: " + products.size() + " products.");
    if (products.size() < 40) {
      products.stream().forEach(System.out::println);
    }
  }
</source>
The four Java classes we wrote for this example are:
* examples.ClientExample.java - main class
* examples.Client.java - Talks to the Web API over HTTP and returns the API's JSON as a Java String
* examples.ProductParser.java - Takes a Java JSON String as argument and parses the JSON to a <code>List&lt;Product&gt;</code>
* se.itu.systemet.domain.Product - a domain object representing one product from the Web API's response

==Client-side task 2 - Write a class which fetches data using REST api==

===Task 2.1===
Create en "empty" class, <code>SystemetRestApiAccess</code>, in package se.itu.systemet.rest. The class should implement the ApiAccess interface (expand?)

'''''Hint:''' look at the [https://progund.github.io/systemet-gui-client-docs/docs/ API] for information about the interface.... and look at the class FakeApiAccess "for inspiration"''.

===Task 2.2===
Copy <code>Client.java</code> to <code>se/itu/systemet/rest</code>. Make sure to change the package declaration of the Client class.

===Task 2.3===
Implement the method <code>fetch</code> to return <code>null</code>. Why do we have to implement the method fetch?.

<div class="mw-collapsible mw-collapsed">
Expand using link to the right to get a hint on why we need to implement.
<div class="mw-collapsible-content">
We're implementing the interface ApiAccess which has a method <code>fetch</code> that we must implement (or be an abstract class). 
</div></div>

Compile everything using <code>./clean.sh && ./build.sh</code> to make sure everything compiles.

'''''Important: '''Also, change the <code>ApiAccessFactory</code> so that it returns a <code>SystemetRestApiAccess</code> instance, and not the old <code>FakeApiAccess</code> instance!''

===Task 2.4===
As the first line of the <code>fetch</code> method body, declare a variable of type <code>Client</code>, and initialize it to a <code>new Client</code>. The argument to the Client constructor should be a string, from <code>url</code> + <code>query.toQueryString()</code>.

Compile everything using <code>./clean.sh && ./build.sh</code> to make sure everything compiles.

===Task 2.5===
Get JSON from the Client using the <code>getJson</code> method. Print the string out on stdout. 
<div class="mw-collapsible mw-collapsed">
Expand using link to the right to get a hint.
<div class="mw-collapsible-content">
Use <code>System.out.println</code> and pass the JSON string as argument.
</div></div>
We only print to confirm that we actually get some JSON data. No use in continuing if we don't.

Compile everything and execute the program, using <code>./clean.sh && ./build.sh && ./run_product_search_gui.sh</code>

When executing the gui you should see the printout of the JSON data of all the products. The gui may, or most likely will, crash since we're returning null. We can fake this by returning an empty ArrayList.
<div class="mw-collapsible mw-collapsed">
Expand using link to the right to get a hint.
<div class="mw-collapsible-content">
<code>return new ArrayList<>();</code>
</div></div>

===Task 2.6===
Compile everything using <code>./clean.sh && ./build.sh && ./run_product_search_gui.sh</code>

In the GUI you should fill in a maximum price of 7.9 and make sure you see this reflected in the terminal wit the JSON printout. You should see just a few products.

===Task 2.7===
Copy the class ProductParser from the example directory to the rest directory where your SystemetRestApiAccess is located. Change the package declaration to reflect this new location.

Compile everything using <code>./clean.sh && ./build.sh</code> to make sure it compiles.

===Task 2.8===
Create a new ProductParser instance. Invoke the method parse and passing the JSON string as argument. The method returns a List of Products. Print this list's size to stdout.

Compile everything using <code>./clean.sh && ./build.sh</code> to make sure it compiles.

Enter (again) 7.9 as maximum price. Make sure that you can see the list size you wrote code to print out earlier.

===Task 2.9===
Return the list of Products (instead of the new ArrayList<> we created earlier).

Compile everything using <code>./clean.sh && ./build.sh && ./run_product_search_gui.sh</code>

Make sure you can see all of our Products in the GUI. Enter (again) 7.9 as maximum price. Make sure that you can see only three products.

=Getting started with the assignment=
Here are a few pointers to get you started with the server-side tasks.
== Download the new files ==
You can find the source code you need [https://github.com/progund/exposing-data-lab-final here (github)]
===Files for server-side===
Navigate to the directory where you have your server-side code from Lab 1 and download these new files:
* <code>DBHelper.java</code> - put in <code>systemet-api/se/itu/systemet/storage/</code>
* <code>ApiExample.java</code> - put in <code>systemet-api/examples/</code> '''''Note:''' You need to replace your old version of this file, since this one forces the ProductLineFactory to use the database version of a product line''
* <code>compile_and_run_api_example.sh</code> - put in the main directory (where the other scripts are)

===Files for client-side===
Navigate to the directory where you have your GUI code from Lab 2 and download these new files:
* <code>ClientExample.java</code> - put in <code>examples/</code> 
* <code>Client.java</code> - put in <code>examples/</code>
* <code>ProductParser.java</code> - put in <code>examples/</code>

=== What to do next ===
Perform the tasks above for the server-side changes.
=== Building and running ===
====Server-side (servlet/wep-api)====
To compile and deploy the changes to <code>SQLBasedProductLine.java</code> and the new file <code>DBHelper.java</code>, use the <code>deploy_systemet_jar.sh</code> script.

To run the <code>ApiExample.java</code> to test your code without using the servlet, use the <code>compile_and_run_api_example.sh</code> script.

To restart the web API (and servlet), use the <code>run_servlet.sh</code> script.

'''''Hint:''' After having started the servlet (using <code>run_servlet.sh</code>) you should be able to use the web api e.g. using a browser [http://localhost:8080/search/products/all localhost:8080/search/products/all] or using curl <code>curl <nowiki>"http://localhost:8080/search/products/all"</nowiki></code>.

====Client-side (gui)====

=Report for the assignment=
If you have been asked to write a report for this assignment with a system's description, then think about the fact that you probably wrote two reports earlier, one for Lab1 (server) and one for Lab2 (GUI/client). You may also have been given specific instructions for this report by your teachers, e.g. via the learning platform (GUL, Canvas, etc).

You don't have to repeat yourself in the report for this final lab. Just focus on what is new:
* Where does the web api get its products from?
** How does a Java program get data from such a source?
* How does the GUI get data from the Web API?
** What techniques (protocols, formats etc) was involved?
** What did the GUI client do with the data, so that it could be presented in the GUI window?

A final '''''optional''''' part of your report could also be ('''''voluntarily''''') to review your report from Lab1 and see if you find any mistakes or errors in that report, given the knowledge you now have about the whole system. Did your explanations in the Lab1 report use the correct terms and concepts? What was slightly wrong? When did you learn or understand that your first report was a little bit off?

Note that we are not saying that all reports must have errors in them! We just ask you in this '''''voluntary''''' task to review your first report and see if you would have explained the system differently given what you have learned since then.

=Problem-shooting=
You should learn to recognize error messages from the compiler and the Java runtime. When you get a compilation error, always read the error message, and start from the top.

If your program crashes in run-time, then you should read the stack trace carefully (also starting from the top). Some common error messages from this lab are:
* ClassNotFoundException: servlets.SystemetWebAPI - what do you think it means that the JVM can't find that class? What do you usually do with your Java-files in order for the classes they represent to be accepted (and loaded) by the JVM?
** See [[Java-Errors:ClassNotFoundException]]
* ParseException/JSONException etc - What do you think is the problem if your program crashes and the error message in the stack trace complains about parsing the JSON?
** Did you really verify that the JSON your servlet produces is valid and well-formed JSON? You were supposed to verify this using the <code>jq</code> program already in Lab 1.
** Common problems with malformed JSON are that a comma is missing or appears in the wrong place. Commas have a very specific and important meaning in a JSON document.
** In Sweden, we use "decimal comma" but in most other places, people use "decimal point" for floating-point numbers
=Outro - concluding remarks=
{{#ev:vimeo|249091792}}

=Navigation=
{{Nav links|prev=Assignment:Exposing data over http lab2 GUI Client|TOC=Assignment:Exposing_data_over_http|next=Assignment:Exposing data over http lab3 integrate the subsystems}}