Assignment:Exposing data over http lab1 Web API

From Juneday education
Jump to: navigation, search

Introduction

Prerequisites - introductory workshop etc

Workshops

Before starting with this assignment, you should attend the workshop for Lab1, the manuscript for which can be found here (see below for a video version of the workshop).

All workshops for Lab 1 (see course schedule for when the workshops are held):

  1. Workshop-Introduction to Servlets - introduces the concept of Servlets
  2. Workshop Getting started with Lab 1 (only PDF) - introduces this lab and gets you started
  3. Workshop Lab 1 - Going public - Online - A networking workshop - you will connect to each other's servlets and more!

Stuff for self-studies and preparations

You should make sure that you are familiar with the following concepts:

Voluntary reading (for those who can't get enough) for reference:

Videos

Introduction videos

Workshop 1

Workshop 2

Parsing the request: Parsing the request SQL version (Full playlist) | Parsing the requst 1 | 2 | 3 | 4 | 5 | Slides: Parsing the request.pdf - SQL/Where version

Parsing the request - filter Parsing the request - filter (Full playlist) | Parsing the request - filter 1 | 2 | 3 | Slides: Parsing the request - filter/predicate version.pdf

Workshop 3

See Workshop Lab 1 - Going public - Online for information. This is a hands-on workshop and we have no videos or slides for it.

Terms and concepts

See also: Exposing_data_over_http_FAQ

Lab 1 - Finishing the web API

In this first sub assignment of the Exposing data over http laboration, we'll learn how to read data from somewhere (could be a database or something else), filter said data according to clients' requests, and, generate JSON from said data. Since we don't require that you know SQL (Structured Query Language - the language for fetching data from a database) already, we'll show you how to finish the web API using fake data which will be easy to replace with data from an actual database later.

Technologies which are the focus of this sub assignment include:

  • JSON - JavaScript Object Notation
  • A tiered architecture - how to insulate the web tier from the more machine-near tiers
  • Servlet technology - A Java app which acts like a web server, reads requests and sends responses
    • Parsing GET parameters and using them to build a filter for the data
  • Using fake data to be able to postpone some technical parts of the application, yet be able to test it

Software introduction

Your server software consists of two software modules:

  • Systemet API
  • servlet code
    • Running inside the Winstone Servlet container

You should download and unpack the source code: github.com/progund/exposing-data-lab-server/archive/master.zip. This code will be the basis for this first lab. The content of the zip file is described below.

Systemet API

The Systemet API can be found in a sub directory (systemet-api) to the big zip file you downloaded. This software module is responsible for:

  • modelling the Systembolaget product line and products
    • The Product class represents one product in the product line
    • The ProductLine interface defines the way to retrieve a list of products, filtered by some criteria
  • providing access to a set of products via the product line
    • all products
    • products filtered by some predicate (e.g. products with a price between 100 and 200 SEK)
  • loading product data from some source
    • The ProductLine interface hides the way products are retrieved (where products are loaded from)

This module is a piece of software (a library) your team is developing separately from the servlet code. The Systemet API is separated from the servlet module to allow you to:

  • reuse the Systemet API code in other pieces of software
  • focus on the problem domain

Required third party software:

  • SQLite JDBC driver (externally developed software), used in part 3 of this lab when we use a database to access the product line
    • Accessible via a script which downloads it

Servlet code

This module uses the Systemet API to get a list of Products and presents a searchable WEB API.

Required software:

  • Winstone servlet engine (externally developed software), used by the servlet code
    • Can be downloaded via script included in this assignment

Directory overview

After unzipping the zip file, enter the directory exposing-data-lab-server-master. Now you are in your working directory for this lab.

The directory you have entered contains the two important parts described above:

  • Systemet API
  • servlet code

The root directory also contains scripts for downloading Winstone and other libraries, building the Servlet part, running the Servlet and other development and deployment tasks. Here's an overview of the scripts:

  • clean.sh - Deletes all class files recursively (in current and all sub directories)
  • download_sqlite_driver.sh - Downloads the Java driver for SQLite3 (for Lab3 - ignore for now)
  • download_winstone.sh - Downloads the Servlet engine Winstone
  • run_servlet.sh - compiles the Servlet classes and runs the Servlet engine Winstone
  • deploy_systemet_jar.sh
    • compiles systemet-api
    • builds a Jar-file from the classes in systemet-api and puts it in the Servlet's lib directory

The directory layout is as follows:

.
|-- clean.sh
|-- deploy_systemet_jar.sh
|-- download_sqlite_driver.sh
|-- download_winstone.sh
|-- README.md
|-- run_servlet.sh
|-- systemet-api
|   |-- build_jar.sh
|   |-- build.sh
|   |-- examples
|   |   |-- ApiExample.java
|   |   `-- ProductsFromCSV.java
|   |-- generate_csv.sh
|   |-- products.csv
|   |-- se
|   |   `-- itu
|   |       `-- systemet
|   |           |-- domain
|   |           |   |-- package-info.java
|   |           |   `-- Product.java
|   |           `-- storage
|   |               |-- FakeProductLine.java
|   |               |-- JsonExporter.java
|   |               |-- package-info.java
|   |               |-- ProductLineFactory.java
|   |               |-- ProductLine.java
|   |               `-- SQLBasedProductLine.java
|   `-- systemet.jar
|-- test
|   `-- test-web-api.sh
|-- webroot
|   |-- api.html
|   |-- search.html
|   |-- style.css
|   `-- WEB-INF
|       |-- classes
|       |   `-- servlets
|       |       |-- FormatterFactory.java
|       |       |-- Formatter.java
|       |       |-- JsonFormatter.java
|       |       |-- ParameterParser.java
|       |       `-- SystemetWebAPI.java
|       |-- db
|       |   `-- bolaget.db
|       |-- lib
|       |   |-- sqlite-jdbc.jar
|       |   `-- systemet.jar
|       `-- web.xml
`-- winstone.jar

Part 1 - Understanding and setting up the Servlet

This first assignment has two parts. The first part is about getting a Servlet up and running, and learning about what Servlets are and how they work. The Servlet will be the central part of the web API you are building. But the Servlet can't do all the work by itself, so you'll also have to look at a Java API we have provided for you. That's the second part.

This part will be tone as a workshop, and if you don't attend this workshop, you will be on your own, but you can still see the video.

Part 2 - Adding stuff to the systemet-api (which the Servlet is using)

Once you have got the Servlet up and running, it is time to investigate and understand the architecture of the setup for the backend application we have provided you. When you feel comfortable with the layout of the backend installation, you should extend the "systemet-api" which is a Java-API for handling Product objects which come from a ProductLine. Your task is to add a class for creating fake products to the "systemet-api"

During this part of lab1, we'll give lectures and workshops about various topics like:

  • Factory for creating object of Interface type
    • Benefits from using the factory - we can fake a database!
  • HTTP
    • PDF lecture presentation exists
  • Servlets
  • Web basics
    • HTML
    • content-type
    • GET/POST from HTML FORM
  • JAR files
    • Using - putting a JAR on the class path
    • Creating - zipping Java stuff into a JAR
    • Executable JAR - Manifest file

Getting started

Getting started with part 1 - understanding the Servlet

For the part 1 in this assignment, understanding and setting up the servlet, you should focus on the webroot directory structure which is where the source code for the servlet is located, and the run_servlet.sh script.

Setting up your environment

In this section we will explain and list the steps needed to set up the development environment for this project.

These steps need only be done once:

Download the JDBC driver for SQLite

This software is not needed for the first assignment, but we have put it here since it is good to verify that you can install and use SQLite as early as possible.

$ ./download_sqlite_driver.sh

Expand using link to the right to see an example printout from the command above.

$ ./download_sqlite_driver.sh 
--2017-12-05 21:12:47--  https://bitbucket.org/xerial/sqlite-jdbc/downloads/sqlite-jdbc-3.21.0.jar
Resolving bitbucket.org (bitbucket.org)... 104.192.143.2, 104.192.143.1, 104.192.143.3, ...
Connecting to bitbucket.org (bitbucket.org)|104.192.143.2|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://bbuseruploads.s3.amazonaws.com/e23982bc-4051-4962-b78e-0d9a05c77d05/downloads/064423c2-3855-4ffd-ae0a-63440443fa8e/sqlite-jdbc-3.21.0.jar?Signature=JAed2aYVYN0Abhgm%2F5OQT70gtlw%3D&Expires=1512506305&AWSAccessKeyId=AKIAIQWXW6WLXMB5QZAQ&versionId=9s3mzI4MrPE4PdbNypGeUD_TzVnYP3w.&response-content-disposition=attachment%3B%20filename%3D%22sqlite-jdbc-3.21.0.jar%22 [following]
--2017-12-05 21:12:48--  https://bbuseruploads.s3.amazonaws.com/e23982bc-4051-4962-b78e-0d9a05c77d05/downloads/064423c2-3855-4ffd-ae0a-63440443fa8e/sqlite-jdbc-3.21.0.jar?Signature=JAed2aYVYN0Abhgm%2F5OQT70gtlw%3D&Expires=1512506305&AWSAccessKeyId=AKIAIQWXW6WLXMB5QZAQ&versionId=9s3mzI4MrPE4PdbNypGeUD_TzVnYP3w.&response-content-disposition=attachment%3B%20filename%3D%22sqlite-jdbc-3.21.0.jar%22
Resolving bbuseruploads.s3.amazonaws.com (bbuseruploads.s3.amazonaws.com)... 52.216.226.192
Connecting to bbuseruploads.s3.amazonaws.com (bbuseruploads.s3.amazonaws.com)|52.216.226.192|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6672489 (6,4M) [application/x-java-archive]
Saving to: ‘webroot/WEB-INF/lib//sqlite-jdbc.jar’

webroot/WEB-INF/lib//sqlite-jdbc.jar        100%[===========================================================================================>]   6,36M  1,63MB/s    in 6,1s    

2017-12-05 21:12:55 (1,04 MB/s) - ‘webroot/WEB-INF/lib//sqlite-jdbc.jar’ saved [6672489/6672489]

Download Winstone servlet engine

$ ./download_winstone.sh

Expand using link to the right to see an example printout from the command above.

$ ./download_winstone.sh 
--2017-12-05 21:16:52--  https://sourceforge.net/projects/winstone/files/latest/download?source=typ_redirect
Resolving sourceforge.net (sourceforge.net)... 216.34.181.60
Connecting to sourceforge.net (sourceforge.net)|216.34.181.60|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://downloads.sourceforge.net/project/winstone/winstone/v0.9.10/winstone-0.9.10.jar?r=&ts=1512505013&use_mirror=netix [following]
--2017-12-05 21:16:53--  https://downloads.sourceforge.net/project/winstone/winstone/v0.9.10/winstone-0.9.10.jar?r=&ts=1512505013&use_mirror=netix
Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 216.34.181.59
Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|216.34.181.59|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://netix.dl.sourceforge.net/project/winstone/winstone/v0.9.10/winstone-0.9.10.jar [following]
--2017-12-05 21:16:54--  https://netix.dl.sourceforge.net/project/winstone/winstone/v0.9.10/winstone-0.9.10.jar
Resolving netix.dl.sourceforge.net (netix.dl.sourceforge.net)... 87.121.121.2
Connecting to netix.dl.sourceforge.net (netix.dl.sourceforge.net)|87.121.121.2|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 327658 (320K) [application/java-archive]
Saving to: ‘winstone.jar’

winstone.jar                                100%[===========================================================================================>] 319,98K  1,21MB/s    in 0,3s    

2017-12-05 21:16:55 (1,21 MB/s) - ‘winstone.jar’ saved [327658/327658]

Deploy the systemet API

The first time you want to run the servlet (and every time have made changes to and compiled the files in the Systemet API) you will need to deploy the jar file with the following procedure:

  • execute the script deploy_systemet_jar.sh

Expand using link to the right to see an example printout from the script deploy_systemet_jar.sh.

./deploy_systemet_jar.sh 
added manifest
adding: se/(in = 0) (out= 0)(stored 0%)
adding: se/itu/(in = 0) (out= 0)(stored 0%)
adding: se/itu/systemet/(in = 0) (out= 0)(stored 0%)
adding: se/itu/systemet/domain/(in = 0) (out= 0)(stored 0%)
adding: se/itu/systemet/domain/Product.class(in = 4587) (out= 2131)(deflated 53%)
adding: se/itu/systemet/domain/Product.java(in = 9712) (out= 2918)(deflated 69%)
adding: se/itu/systemet/domain/Product$Exporter.class(in = 381) (out= 253)(deflated 33%)
adding: se/itu/systemet/domain/package-info.java(in = 144) (out= 115)(deflated 20%)
adding: se/itu/systemet/domain/Product$Builder.class(in = 1705) (out= 631)(deflated 62%)
adding: se/itu/systemet/storage/(in = 0) (out= 0)(stored 0%)
adding: se/itu/systemet/storage/JsonExporter.class(in = 2419) (out= 1172)(deflated 51%)
adding: se/itu/systemet/storage/JsonExporter.java(in = 3442) (out= 1034)(deflated 69%)
adding: se/itu/systemet/storage/SQLBasedProductLine.class(in = 1292) (out= 580)(deflated 55%)
adding: se/itu/systemet/storage/SQLBasedProductLine.java(in = 1050) (out= 451)(deflated 57%)
adding: se/itu/systemet/storage/package-info.java(in = 119) (out= 94)(deflated 21%)
adding: se/itu/systemet/storage/ProductLineFactory.java(in = 827) (out= 373)(deflated 54%)
adding: se/itu/systemet/storage/ProductLine.class(in = 458) (out= 234)(deflated 48%)
adding: se/itu/systemet/storage/ProductLine.java(in = 1207) (out= 472)(deflated 60%)
adding: se/itu/systemet/storage/FakeProductLine.java(in = 1710) (out= 639)(deflated 62%)
adding: se/itu/systemet/storage/FakeProductLine.class(in = 1543) (out= 643)(deflated 58%)
adding: se/itu/systemet/storage/ProductLineFactory.class(in = 678) (out= 393)(deflated 42%)

Build and start your servlet code

In this section we list the steps needed to build the source code. These steps need to be done every time you:

  • want to build (compile) to check that your source code edits were ok
  • start the servlet container Winstone with the latest changes
$ ./compile_servlet.sh
$ ./run_winstone.sh

Note: Expand using link to the right if you can't execute the script.

$ chmod u+x *.sh # make all scripts executable

Expand using link to the right to see an example printout from the command above.

$ 
$ ./run_winstone.sh 
Servlet URL: 
http://localhost:8080/search/products/all?min_alcohol=50&max_price=200&type=Öl&name=Guinness&max_alcohol=60
Or using an HTML form:
http://localhost:8080/search.html
Compiling and starting servlet container...
[Winstone 2017/12/05 23:27:43] - Winstone Servlet Engine v0.9.10 running: controlPort=disabled
[Winstone 2017/12/05 23:27:43] - HTTP Listener started: port=8080
[Winstone 2017/12/05 23:27:43] - AJP13 Listener started: port=8009

etc

Now you have the Servlet running inside a web server (called Winstone)! Try to access the following URLs from a browser:

Does it work? If it does not work, check the next section Deploy the systemet API. Chances are you'll need to deploy the Systemet API.

Note: searches and direct queries should result in an empty JSON array, []. It is empty since we're not yet reading from a database. We need to fake data. Creating fake data is something we will do soon.

Getting started with Part 2 - Adding stuff to the systemet-api

As you could see from the Part 1 of this assignment, there are no products returned when querying the web API. This is because there are no Products available for the web API. The Servlet doesn't read any Products from the ProductLine.

The Servlet uses the systemet.jar library file from its webroot/WEB-INF/lib/ directory, in order to access the ProductLine and query it for Product objects according to some filter criteria.

Build the Systemet API

In this section we list the steps needed to build the source code of the Systemet API. These steps need to be done every time you have made changes to the Systemet API source code.

We give you two ways of building and deploying the Systemet API.

  • The manual way
  • The scripted way

The manual way

$ cd systemet-api/
$ ./build.sh
$ ./build_jar.sh 
$ cp systemet.jar ../webroot/WEB-INF/lib/

and when you've compiled and built a jar file with your changes, put the jar file in the lib/ directory for the servlet, and want to continue developing and testing the servlet you need to go back to the directory one step above.

$ cd ..

Note: you can, if you're bold as the axis do the steps above in one: $ cd systemet-api/ && ./build.sh && ./build_jar.sh && cd ..

This might seem a bit cumbersome so we have written a script for you that both builds and deploys the Systemet API. This makes your changes (in the Systemet API) ready for use by the servlet.

The scripted way

We prefer to use the script, deploy_systemet_jar.sh to compile, create a jar file and install the jar file in a place to make it useable by the servlet. This script performs all the actions from the previous section.

$ ./deploy_systemet_jar.sh

Expand using link to the right to see an example printout from the command above.

$ ./deploy_systemet_jar.sh 
added manifest
adding: se/(in = 0) (out= 0)(stored 0%)
adding: se/itu/(in = 0) (out= 0)(stored 0%)
adding: se/itu/systemet/(in = 0) (out= 0)(stored 0%)
adding: se/itu/systemet/domain/(in = 0) (out= 0)(stored 0%)
adding: se/itu/systemet/domain/Product.class(in = 4587) (out= 2131)(deflated 53%)
adding: se/itu/systemet/domain/Product.java(in = 9712) (out= 2918)(deflated 69%)
adding: se/itu/systemet/domain/Product$Exporter.class(in = 381) (out= 253)(deflated 33%)
adding: se/itu/systemet/domain/package-info.java(in = 144) (out= 115)(deflated 20%)
adding: se/itu/systemet/domain/Product$Builder.class(in = 1705) (out= 631)(deflated 62%)
adding: se/itu/systemet/storage/(in = 0) (out= 0)(stored 0%)
adding: se/itu/systemet/storage/JsonExporter.class(in = 2419) (out= 1172)(deflated 51%)
adding: se/itu/systemet/storage/JsonExporter.java(in = 3442) (out= 1034)(deflated 69%)
adding: se/itu/systemet/storage/SQLBasedProductLine.class(in = 1292) (out= 580)(deflated 55%)
adding: se/itu/systemet/storage/SQLBasedProductLine.java(in = 1050) (out= 451)(deflated 57%)
adding: se/itu/systemet/storage/package-info.java(in = 119) (out= 94)(deflated 21%)
adding: se/itu/systemet/storage/ProductLineFactory.java(in = 827) (out= 373)(deflated 54%)
adding: se/itu/systemet/storage/ProductLine.class(in = 458) (out= 234)(deflated 48%)
adding: se/itu/systemet/storage/ProductLine.java(in = 1207) (out= 472)(deflated 60%)
adding: se/itu/systemet/storage/FakeProductLine.java(in = 1710) (out= 639)(deflated 62%)
adding: se/itu/systemet/storage/FakeProductLine.class(in = 1543) (out= 643)(deflated 58%)
adding: se/itu/systemet/storage/ProductLineFactory.class(in = 678) (out= 393)(deflated 42%)

Understanding the Systemet API

Read the online documentation for the Systemet API. You might wonder what we mean by "Systemet API". We mean, the classes in the systemet-api directory which the Servlet is using for stuff related to the products in the product line. Such classes include se.itu.systemet.domain.Product which is central here, since it represents one product from the product line. The class se.itu.systemet.storage.ProductLine is also central, since it represents all the products in Systembolaget's product line.

So, the online documentation for Systemet API, is documenting the classes that have to do with products and the product line. These classes will be packaged into a JAR file, and placed in a directory where the Servlet will find them. This way, we can separate the Servlet classes (which should focus on communicating with clients (browsers etc), and the classes representing the actual data (the product line with a lot of products).

Adding fake Product generation to the Systemet API

First, make sure you understand how a Product object is created, using its builder. There are code examples in the online documentation, here, for the Product class.

Second, make sure you understand what the purpose and responsibilities of the ProductLine interface are (you will work with the FakeProductLine class, which implements this interface).

You will in this sub task finish the FakeProductLine class, so that the Servlet later can get products from a ProductLine, filter those products using the GET parameters, and produce a JSON response.

Let's talk for a minute about why you are doing this part of the assignment. You saw that the Servlet with the Web API works, but always produces an empty JSON array, []. This is because the Servlet doesn't get any products from the ProductLine it is using. The product line factory actually gives the Servlet an instance of a FakeProductLine, but the FakeProductLine doesn't return any products yet.

Your task is thus to finish the FakeProductLine class so that it returns a list of products, even if those products are bogus and don't come from the database source.

Why do we want to use fake products? Because we haven't yet learned how to read products from the database, but we really want to test the web API anyway, so where the products come from doesn't matter at this point.

This is a common way to develop systems. You don't have to write a fully implemented system in order to test a part of the system. Postponing the reading of products from the database, allows us to focus on the Servlet and its JSON production. The database part will be done in the Lab3. This way, we can test that the Servlet is capable of filtering some list of products and create JSON from the result. If this works, we can be pretty confident that it will work with another list later, i.e. a list which originates from the database. It is important, however, that we create the list of products, and the actual products in the list, in a correct way, so that it will be comparable to a list which we generate from the database later.

Now, let's discuss various ways you could generate a list of fake products (products you have created in some way not involving the database).

In the given code for FakeProductLine, there's a private method you can use, called getFakeProduct. It is there only to give you a chance to generate Java code for products, e.g. code which could look like this:

products.add(getFakeProduct("Johanneshof Reinisch Pinot Noir","143.0","13.0","750","7440201","Rött vin",""));
products.add(getFakeProduct("Dobogó Tokaji Furmint","187.0","13.5","750","7598701","Vitt vin",""));
products.add(getFakeProduct("Château de Cazeneuve Carignan","250.0","13.5","750","7105201","Rött vin",""));
products.add(getFakeProduct("Engelholms Pacific Pilsner","21.0","5.0","330","3067603","Öl","Ljus lager"));
products.add(getFakeProduct("Speyside Sherry Cask 21 Years","1537.0","55.1","700","8591801","Whisky","Malt"));
products.add(getFakeProduct("Giró i Giró Montaner Brut Nature Gran Reserva","151.0","11.5","750","7723101","Mousserande vin","Vitt Torrt"));
products.add(getFakeProduct("Clynelish No 4051 17 Years","1583.0","55.0","700","8515601","Whisky","Malt"));
products.add(getFakeProduct("Albarossa","252.0","14.0","750","7321901","Rött vin",""));
products.add(getFakeProduct("Los Frailes Monastrell Garnacha","115.0","13.5","750","7444301","Rött vin",""));
products.add(getFakeProduct("Auchentoshan Maltbarn Bourbon Cask 23 Years","1735.0","52.0","700","8537101","Whisky","Malt"));
products.add(getFakeProduct("Albin Jacumin Châteauneuf-du-Pape","246.0","14.5","750","7551101","Rött vin",""));
products.add(getFakeProduct("Bibbiano Chianti Classico","147.0","13.5","750","7539001","Rött vin",""));
products.add(getFakeProduct("Chapter 7 Irish Whiskey Bourbon Hogshead 14 Years","1113.0","56.7","700","8771001","Whisky","Malt"));
products.add(getFakeProduct("Amour de Deutz","796.0","12.0","375","9602502","Mousserande vin","Vitt Torrt"));
products.add(getFakeProduct("Deutz Exclusive Gift Box","2397.0","12.0","1125","9696209","Mousserande vin",""));
products.add(getFakeProduct("Villa Spinosa Valpolicella Classico","130.0","12.0","750","7249001","Rött vin",""));
products.add(getFakeProduct("D'Aria Sauvignon Blanc","146.0","13.0","750","7337501","Vitt vin",""));
products.add(getFakeProduct("Fairtransport Tres Hombres 21 Años","885.0","41.6","700","8712101","Rom","Mörk"));
products.add(getFakeProduct("Donatella Cinelli Colombini","1104.0","14.0","4500","7475209","Rött vin",""));
products.add(getFakeProduct("Abrigo Giovanni Piemonte Mix 1","732.0","13.5","4500","7096809","Rött vin",""));

To generate code lines like that is much, much simpler than to generate code using the Builder for products. Note, however, that having a method like getFakeProduct which takes seven parameters effectively defeats the purpose of the builder pattern! One of the motivations behind using a builder, is to remove constructors with a lot of arguments (which is very hard to use). But given that this is a private method, not exposed to any client code, we can live with the fact that getFakeProduct takes seven arguments. It makes it easy for us to generate code lines as the one above.

If you don't want to use getFakeProduct for the above reason or some other reason, you can use a CSV file as the source of the fake product generation. There's an example Java file showing you how you could do that. There's also a sample CSV file you can use, and even a script for generating CSV files from the database. You don't have to understand how the script and database works, in order to run the script.

If you opt for the CSV as a source for fake products, you will have to make sure that the Servlet can find the CSV file when it is running. We suggest that you put the CSV file in the Servlet directory webroot/WEB-INF/db/ when you deploy the new systemet.jar file.

Using CSV (Challenge - not mandatory)

If you want, you can optionally create fake prouducts from the CSV file we have provided for you. There's an example showing how you can produce products from a CSV file, in systemet-api/examples/ProductsFromCSV.java. When you deploy systemet.jar to the Servlet, put the CSV file in webroot/WEB-INF/db/ and make sure the file path you are using points to that place.

Example:

// Read products from CSV file:
String fileName = "webroot/WEB-INF/db/products.csv";
List<Product> products = new ArrayList<>();
try {
  BufferedReader reader =
  Files.newBufferedReader(Paths.get(fileName), StandardCharsets.UTF_8);
  ...
  ...

Knowing when you are done

You are done with this task when the following is true:

  • You understand how to
    • build
    • deploy
    • run
    the Servlet Web API
  • Can query your Web API via HTTP and see that the filtering works

You shall write in your diary (see workshop for details) what you have written and how that code works. You shall also explain how the overall system works and is intended to work.

So, how do you know that you can query the Web API and that it does the right thing?

It is important that you have created enough fake products for the Web API to work on, and that the Fake products are different enough for you to test that the filters work.

Note that you haven't written the filters yourself (the teachers were kind enough to write them for you), but you should still test that the setup with the fake products works and filters you fake products in accordance with the Web API online documentation (Which can be reached from the Servlet itself, see above).

We suggest that you test to query your Web API using various combinations of queries, which, according to your fake product store contents, will filter out a predictable number of products.

If we take the products from the CSV provided for you, the following queries could be tested:

  • All products unfiltered
    • e.g. no parameters at all
    • should return 20 products as well-formed JSON
  • products with price >=100 and price <=200
    • parameters: min_price=100&max_price=200
    • should return 7 products
  • products with alcohol >=40 and price <=900
    • parameters: min_alcohol=40&max_price=900
    • should return 1 product
  • products with alcohol >=10 and price <=200
    • parameters: min_alcohol=10&max_price=200
    • should return 7 products

It is good if you figure out more cases to test. For instance cases which return no products at all. How does the Web API handle bogus parameters which it doesn't understand?

How can you test all those cases? Well, you could enter the criteria in the web form HTML page included with the Servlet setup. But that quickly gets boring and hard. We recommend that you use some command line tools to do the testing. We have a script, test/test-web-api.sh, doing this. The script is included in the zip file you downloaded to get started with this assignment.

$  test/test-web-api.sh 
Checking "": OK
Checking "min_alcohol=50": OK
Checking "min_price=100&max_price=200": OK
Checking "min_alcohol=40&max_price=900": OK
Checking "min_alcohol=10&max_price=200": OK

You should also test the JSON you get, to see that it is valid JSON. You can use tools for that, such as jq (available for GNU/Linux, MacOS and Cygwin). There are also online resources for validating JSON. We recommend the command line tools, though, since they can be scripted.

$ ./test/test-web-api.sh -j
Checking "": OK
Checking "min_alcohol=50": OK
Checking "min_price=100&max_price=200": OK
Checking "min_alcohol=40&max_price=900": OK
Checking "min_alcohol=10&max_price=200": OK

Include in your diary what tests you wrote and explain how they work.

When the tests pass, you have finished the first part of this tree-part assignment! Lab1 is done, and you now have a Web API which returns JSON with products according to user-provided criteria.

In the next assignment part, Lab2, we'll write a small (and ugly) GUI client, which is able to display a table of products, and also is able to create the URLs necessary to request the Web API for products filtered with some criteria.

If you think this assignment was too easy

Note: We got some feedback from students that the assignment was too easy (some thought so, others thought it was just OK while others still thought it was too hard). If you think it was too easy, you can optionally do all the challenges. We will not require that every student does the challenges, nor will we give much feedback on the challenges while correcting the code you send in. The challenges are totally voluntary and only serve as a tool for those who want to challenge themselves and learn more. Also, make sure your code works and save and send in working code. You don't need to send in code with the challenges implemented.

If you found this assignment too easy, then you should look at the challenge above, using a CSV file for loading the products. Note that loading a lot of products using a CSV file, means that the test script above won't work, since you might have more and different products in your system than the 20 fake products we suggested. You are free to change the test script so that it tests the products in your CSV file.

If you still think this is too easy, you can do one more thing. Look at the code and API documentation for the Product class in the systemet-api directory tree. It has a field (instance variable and accessor method) for "type". The type signifies a sub category of the product group. A "type" of "Beer" (Swedish: Öl) can be "Stout" (e.g. Guinness is a beer of type Stout).

Now look at the JsonExporter class (also in the systemet-api directory tree). This class is responsible for converting one product to a JSON object. But the resulting JSON doesn't contain "type". Your challenge is obviously to add the "type" to the JSON output.

Make sure that the JSON including the new key-value for type doesn't break the rules for JSON. It must still list all the key-value pairs in the JSON-object, and separate the key-values using a comma, and the key-value text for type should look something like:

{
  "name": "Vodka especial",
  "type": "Mexican vodka",
  "price": 100.90,
... etc etc
},

Another voluntary challenge could be to add functionality to the web api to accept one more parameter, "name". This should the ParameterParser (in the same package as the servlet) add to the list of accepted parameters and also take into account when creating the Predicate<Product>.

A parameter of name=carlsberg should match all products that contain the string "carlsberg" anywhere in the name.

You could look at the API documentation for String to see how you can check if a String contains another String (as a substring). You can test this in a stand-alone Java-program where the following should be true:

String carl = "carlsberg";
String fullName "Super Carlsberg beer";
// test that carl is a part of fullName

Also you can add a parameter for product_group, so that the API can return JSON for a specified product group, like "Öl" (Swedish for beer) or "Vin" (Swedish for Wine).

If you add a parameter "product_group=xxx" to the web api, then the Servlet must decode the string from the GET parameter, since e.g. "Öl" with the Swedish character "Ö" will be encoded by the HTTP request to %C3%96l, where %C3%96 is the URL encoding for "Ö". There are methods available to decode URL parameters, in the Java API.

Expand using link to the right to see some hints for filtering on name.

The way the parser for the parameters works, is by only allowing valid parameters from a list of valid parameter names. You should add "name" to that list, in order to allow that parameter.

But, then, the parser also checks that all values (the part after the =) can be parsed to double values.

If you allow name as a valid parameter, the parser won't work anymore, since the name probably can't be parsed as a double value, and it will be removed.

We suggest the following strategy:

First add "name" do the list of valid parameters. Then, in the parse method, handle the case that there is a name parameter present, and create a predicate for the name and add it to the list of predicates. Then remove the name parameter. This way, you don't have to change the way the parser handles double values.

This is how you can check if there is a name parameter present:

  private List<Predicate<Product>> parse() {
    List<Predicate<Product>> predicates = new ArrayList<>();
    List<String> validArgs = new ArrayList<>(Arrays.asList(args));

    // you need to import java.util.Optional!
    Optional<String> name = validArgs.stream() 
      .filter(s -> s.startsWith("name=") && s.split("=").length == 2)
      .findFirst();
    if (name.isPresent()) {
      // you can now create the predicate using name.get().spllit("=") etc
      // add the predicate to the list of predicates
    }
    // the next line will remove the name parameter since it's not a double
    validArgs.removeIf(s->!isValidKey(s) || !isDouble(s.split("=")[1]));
    // But that's OK, since we can keep the rest of the parsing as is...

Navigation

« PreviousBook TOCNext »