ITIC:Introduction to Bash scripting - Exercises

From Juneday education
Jump to: navigation, search

Your first script

Write a script that prints out today's date and time. Run it and confirm that it works.

Hints:

  • Always start new tasks in a new directory - create one for these exercises and cd down to it
  • Remember that the first line should be the shebang #!/bin/bash
  • Remember to change the mode of the file to add execute permission
    chmod u+x filename
  • Remember that to run a script, you need a path to the script, e.g. ./scriptname
  • Start your editor from the command line - easier to create and find the script, prevents confusion later

Expand using link to the right to see a suggested solution.

$ chmod u+x datetime.sh

$ ./datetime.sh
ons  7 aug 2019 10:55:15 CEST

$ cat datetime.sh
#!/bin/bash

date
rikard@newdelli:~/opt/progund/intro-it/bash-scripting-intro/exercises$

Now, to practice on date formats, you should make a script that prints today's date and time in this format: YYYY-mm-dd HH:MM:SS, e.g. 2019-08-07 22:57:39.

Hints:

  • To get a list of date format symbols, use man date (navigate with arrow keys, quit with "q") or date --help | grep %
  • To use a date format, e.g. %Y, do date "+%Y"
  • Try your commands in the interactive shell before you put them in the script, to speed up development time (you don't have to edit-save-run between each test - just run date in the terminal until you are pleased)

Expand using link to the right to see a suggested solution.

$ chmod u+x datetime_formatted.sh

$ ./datetime_formatted.sh 2019-08-07 10:59:31

$ cat ./datetime_formatted.sh

  1. !/bin/bash

date "+%Y-%m-%d %H:%M:%S"

$

Using environment variables

Write a script that prints out the value of the following environment variables (some may be empty):

  • HOME
  • USER
  • EDITOR
  • TERM
  • SHELL
  • PATH
  • LANG
  • LC_TIME

Each variable should be printed on a separate line with a label before. Example:

HOME: /home/rikard
USER: rikard
etc etc

Hints:

  • Remember that you should use a dollar sign before a variable name, to expand the value
  • Use quotes around text (a good habit but not a requirement)
  • You can mix text and variable values, e.g.
    echo "M-m-m-m-my Sharona: $SHARONA"

Expand using link to the right to see a suggested solution.

$ chmod u+x environment.sh

$ ./environment.sh
HOME: /home/rikard
USER: rikard
EDITOR: 
TERM: xterm-256color
SHELL: /bin/bash
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/rikard/bin/
LANG: en_US.UTF-8
LC_TIME: sv_SE.UTF-8

$ cat ./environment.sh
#!/bin/bash

echo "HOME: $HOME"
echo "USER: $USER"
echo "EDITOR: $EDITOR"
echo "TERM: $TERM"
echo "SHELL: $SHELL"
echo "PATH: $PATH"
echo "LANG: $LANG"
echo "LC_TIME: $LC_TIME"

$ ######## Fancy version using an associative array below ###########

$ chmod u+x environment_map.sh

$ ./environment_map.sh
SHELL: /bin/bash
USER: rikard
HOME: /home/rikard

$ cat ./environment_map.sh
#!/bin/bash

#  An associative array
#+ with key-value pairs
declare -A VARS
VARS[HOME]="$HOME"
VARS[USER]="$USER"
VARS[SHELL]="$SHELL"
# etc etc

# Loop over each key:
for var in "${!VARS[@]}"
do
    echo "$var: ${VARS[$var]}"
done

$

Alternative solution, declare the associative array and its key-values directly:

declare -A map=(
    [HOME]="$HOME"
    [USER]="$USER"
    [SHELL]="$SHELL"
)

for var in "${!map[@]}"
do
    echo "$var: ${map[$var]}"
done

Print some network information

Write a script that prints out some network information about your computer.

Hints:

  • You can use command substitution/command expansion, e.g.
    echo "Default gw: $(ip route | grep default | cut -d ' ' -f3)"
  • Some useful commands can be found in the module about network tools
    • ip
    • host
    • uname
  • If you are on VirtualBox your result may vary from your actual computer - but that's OK - ask your teacher or supervisor if you want to know why
    • If you have Bash on your default OS, try the script there too - you may need to install some commands or find replacements

Expand using link to the right to see a suggested solution.

Simple script:

$ ./simple-network-info.sh
Default gateway: default via 10.0.0.1 dev wlp58s0
IP number(s): 10.0.116.35 192.168.56.1 2001:6b0:2:2801:36bd:fb4f:417b:b503 
Hostname: newdelli

$ cat ./simple-network-info.sh
#!/bin/bash

GW=$(ip r | grep default | cut -d ' ' -f 1-5)
# take line with "default", take columns 1-5 separated with space

IP=$(hostname -I)

echo "Default gateway: $GW"
echo "IP number(s): $IP"
echo "Hostname: $HOSTNAME" # environment variable

More fancy/advanced script (just for those who want to learn more):

$ ./network-info.sh
Pinging ftp.sunet.se...Done!
Counting hops to pts.se...Done!

Network cards:  lo  wlp58s0  vboxnet0 
IP addresses:  lo: 127.0.0.1/8 wlp58s0: 10.0.116.35/16 vboxnet0: 192.168.56.1/24
Default gateway: default via 10.0.0.1 dev wlp58s0
Computer hostname: newdelli
Ping to ftp.sunet.se successful.
There were  16 hops to pts.se

$ cat ./network-info.sh
#!/bin/bash

GW=$(ip r | grep default | cut -d ' ' -f 1-5)
NETWORK_CARDS=$(ip link | grep ^[0-9] | cut -d ':' -f2 | tr '\n' ' ' )
IPS=$(for card in $NETWORK_CARDS
      do
          addr=$(ip address |
                        grep -A2 ${card}: |
                        grep 'inet ' |
                        awk '{print $2;}')
          echo -n " $card: $addr"
      done
   )
CAN_PING_SUNET=false

echo -n "Pinging ftp.sunet.se..."
ping -c 1 -w 3 ftp.sunet.se &> /dev/null && CAN_PING_SUNET=true
echo "Done!"

echo -n "Counting hops to pts.se..."
HOPS_TO_PTS=$(mtr -r -c 1 pts.se 2> /dev/null | tail -1 | cut -d '.' -f1 )
echo "Done!"
echo
echo "Network cards: $NETWORK_CARDS"
echo "IP addresses: $IPS"
echo "Default gateway: $GW"
echo "Computer hostname: $HOSTNAME" # environment variable
if $CAN_PING_SUNET
then
    echo "Ping to ftp.sunet.se successful."
else
    echo "Could not ping ftp.sunet.se. Do you have a network connection?"
fi

if [[ -z "$HOPS_TO_PTS" ]]
then
    echo "Could not count hops to pts.se . Network problem?"
else
    echo "There were $HOPS_TO_PTS hops to pts.se"
fi

Write a script that downloads a file

The script should take two arguments:

  • The URL to the file (use quotes around the URL both on the command line and inside the script)
  • The filename to use when saving

It is OK if the script downloads the file to the current directory (the directory where the user runs the script).

You can try the script with these URLs:

To verify that your script worked, use file to investigate the file type, then use an application to open the file from the command line. Some tools:

  • evince (opens PDF files - GNU/Linux)
  • xpdf (opens PDF files - GNU/Linux)
  • eog (Eye of Gnome - opens images - GNU/Linux/Gnome)
  • Use a search engine to find other tools you may need, and information on how to install them

Expand using link to the right to see a suggested solution.

Here's one suggested solution, which handles wrong number of arguments etc:

$ chmod u+x download_file.sh

$ ./download_file.sh "http://wiki.juneday.se/mediawiki/images/6/6f/TildeGrillbar.png" grillbar.png
Saved grillbar.png from http://wiki.juneday.se/mediawiki/images/6/6f/TildeGrillbar.png

$ file grillbar.png 
grillbar.png: PNG image data, 999 x 1338, 8-bit/color RGB, non-interlaced

$ ./download_file.sh "http://wiki.juneday.se/mediawiki/images/6/6f/TildeGrillbar.png" 
Usage:
 ./download_file.sh url filename
Where:
 url is an HTTP url to a file (use double quotes around the url)
 filename is the filename to use for the resulting file

$ ./download_file.sh "http://wiki.juneday.se/mediawiki/images/6/6f/TildeGrillbar.png" grillbar.png aoa
Usage:
 ./download_file.sh url filename
Where:
 url is an HTTP url to a file (use double quotes around the url)
 filename is the filename to use for the resulting file

$ ./download_file.sh 
Usage:
 ./download_file.sh url filename
Where:
 url is an HTTP url to a file (use double quotes around the url)
 filename is the filename to use for the resulting file

$ ./download_file.sh "http://totally.wrong.url/bad/monkey" grillbar.png
Could not download file.
See /tmp/download_27665.log for details.

$ cat /tmp/download_27665.log
--2019-08-07 16:08:58--  http://totally.wrong.url/bad/monkey
Resolving totally.wrong.url (totally.wrong.url)... failed: Name or service not known.
wget: unable to resolve host address ‘totally.wrong.url’

$ cat ./download_file.sh
#!/bin/bash

URL="$1"
FILE="$2"
LOG_FILE="/tmp/download_$$.log" # for error msg

usage()
{
    echo "Usage:" >&2
    echo " $0 url filename" >&2
    echo "Where:" >&2
    echo " url is an HTTP url to a file (use double quotes around the url)" >&2
    echo " filename is the filename to use for the resulting file" >&2
    exit 1
}

if (( $# != 2 ))
then
    usage
fi

wget "$URL" -O "$FILE" &> "$LOG_FILE" # both std out and std err go to logfile
if (( $? != 0 ))
then
    echo "Could not download file." >&2
    echo "See $LOG_FILE for details." >&2
    exit 1
else
    echo "Saved $FILE from $URL"
fi

If you think that was to advanced, here's a simpler version which doesn't handle errors as well, and is more verbose:

$ chmod u+x ./download_simple.sh

$ ./download_simple.sh "http://kursplaner.gu.se/pdf/kurs/sv/TIG015" curriculum-tig015.pdf
--2019-08-07 16:14:11--  http://kursplaner.gu.se/pdf/kurs/sv/TIG015
Resolving kursplaner.gu.se (kursplaner.gu.se)... 130.235.62.23
Connecting to kursplaner.gu.se (kursplaner.gu.se)|130.235.62.23|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/pdf]
Saving to: ‘curriculum-tig015.pdf’

curriculum-tig015.pdf                  [ <=>                                                             ] 145,91K  --.-KB/s    in 0,02s   

2019-08-07 16:14:11 (5,75 MB/s) - ‘curriculum-tig015.pdf’ saved [149407]


$ ./download_simple.sh "http://totally.wrong.url/bad/monkey" grillbar.png
--2019-08-07 16:15:31--  http://totally.wrong.url/bad/monkey
Resolving totally.wrong.url (totally.wrong.url)... failed: Name or service not known.
wget: unable to resolve host address ‘totally.wrong.url’

$ ./download_simple.sh "http://totally.wrong.url/bad/monkey" 
No such file or directory

$ ./download_simple.sh 
No such file or directory

$ cat ./download_simple.sh
#!/bin/bash

URL="$1"
FILE="$2"

wget "$URL" -O "$FILE"

Time to Christmas?

Here's a little harder challenge.

In Sweden, we celebrate the Christmas Eve at December 24. We also celebrate New year's eve at December 31. You can choose one of these dates, or some other date of importance to you if you don't care about Swedish holidays. Perhaps you can use the date for the exam of your course (if you have one).

Your task is to write a script that calculates the number of days from today to this date.

Look at the example at the end of the last page for inspiration.

Hints:

  • The number of days from today to some other day, is the day-in-year number of the other date minus the day-of-year number of today.
    • If this number is negative, the day has passed
    • If this number is zero it is today
    • Otherwise it is the number of days remaining
  • You can get day-in-year from date using the format string %j:
    date +%j
  • You can give date an argument of -d somedate to use somedate instead of today:
    date -d 2019-08-06 +%j
  • You can use a simple if-statement with double parentheses for comparisons:
    if (( a < b )); then echo "a is less"; else echo "a is greater or equal";fi

Expand using link to the right to see a suggested solution.

$ ./days-to-christmas.sh 
Christmas is in 139 days

$ cat days-to-christmas.sh
#!/bin/bash

YEAR=$(date "+%Y")
CHRISTMAS="$YEAR-12-24"
TODAY_NR=$(date "+%j")
CHRISTMAS_NR=$(date -d "$CHRISTMAS" +%j)

DAYS_TO_CHRISTMAS=$((CHRISTMAS_NR - TODAY_NR));

if (( $DAYS_TO_CHRISTMAS > 0 ))
then
    echo "Christmas is in $DAYS_TO_CHRISTMAS days";
elif (( $DAYS_TO_CHRISTMAS < 0 ))
then
    echo "Christmas has already been";
else
    echo "Merry Christmas!";     
fi

$ date
ons  7 aug 2019 14:44:02 CEST

$

Here's a more advanced and fancy script, that will default to the number of days until Christmas if no arguments are given, but also allows the user to provide a date, e.g. "10-24" (October 24) and show the number of days to that date.

$ cat days-to-date.sh
#!/bin/bash

YEAR=$(date "+%Y")
DEFAULT_DATE="$YEAR-12-24"
TODAY_NR=$(date "+%j")
days_to_date ()
{
    DATE=$(date -d "$1" +%j);
    DAYS_TO_DATE=$((DATE - TODAY_NR));
    if (( $DAYS_TO_DATE > 0 ))
    then
        echo "$1 is in $DAYS_TO_DATE days";
    elif (( $DAYS_TO_DATE < 0 ))
    then
        echo "$1 has already been";
    else
        echo "$1 is today!";
    fi
}

usage()
{
    echo "Usage: $0 [mm-dd]" >&2
    echo " No arguments shows day to Christmas" >&2
    echo " An argument of e.g. 12-31 shows" >&2
    echo "the number of days to that date this year" >&2
    exit 1
}

# Action begins here...

# Sanity check - invalid date, show usage
date -d "$YEAR-$1" &> /dev/null || usage

if (( $# == 1 )) # One argument?
then
    days_to_date "$YEAR-$1"
elif (( $# == 0 )) # no arguments, use Christmas
then
    days_to_date "$DEFAULT_DATE"
else # too many arguments, show usage
    usage
fi
#### examples of the use below ####

$ ./days-to-date.sh
2019-12-24 is in 139 days

$ ./days-to-date.sh 11-24
2019-11-24 is in 109 days

$ ./days-to-date.sh today
2019-today is today!

$ ./days-to-date.sh tomorrow
2019-tomorrow is in 1 days

$ ./days-to-date.sh "Next month"
2019-Next month is in 31 days

$ ./days-to-date.sh "Next Friday"
2019-Next Friday is in 2 days

$ date
ons  7 aug 2019 15:46:07 CEST

Latest file uploads to wiki.juneday.se

There's a page on the wiki listing the latest file uploads. Your task is to write a script to list the URLs to the most recent uploaded files.

Let's first analyze the HTML for the file, to see what lines contain the links. Here's an excerpt from the HTML for the page, showing one file listing:

<tr>
<td class="TablePager_col_img_timestamp">19:07, 9 June 2019</td>
<td class="TablePager_col_img_name"><a href="/mediawiki/index.php/File:Operator.pdf" title="File:Operator.pdf">Operator.pdf</a> (<a href="/mediawiki/images/f/f9/Operator.pdf">file</a>)</td>
<td class="TablePager_col_thumb"><a href="/mediawiki/index.php/File:Operator.pdf" class="image"><img alt="" src="/mediawiki/resources/assets/file-type-icons/fileicon-pdf.png" width="120" height="120" /></a></td>
<td class="TablePager_col_img_size">51 KB</td>
<td class="TablePager_col_img_user_text"><a href="/mediawiki/index.php/User:Rikard" title="User:Rikard">Rikard</a></td>
<td class="TablePager_col_img_description">New white style/theme. Some typos and formatting.</td>
<td class="TablePager_col_count">2</td>
</tr>

You can use the class for the cell, TablePager_col_img_name to get all lines with the URLs for the files, using grep.

To get the page, you can use lwp-request -m GET "<URL>" (where URL is the URL to the page).

To get the URL from the line, you can use cut with the delimiter set to double quotes ("). The URL you want from the example above, is /mediawiki/images/f/f9/Operator.pdf.

Now, once you have figured out a way to get a list of URLs from the page, all you need to do is to loop over those URLs and print them out using a prefix "http://wiki.juneday.se". We suggest that you create a variable with the prefix, and use it as part of the printing of the URL.

To loop over the URLs, you can use something like:

for url in $(command-that-downloads-page-and-filters-out-urls)
do
  print-the-url-and-prefix
done

Of course, you need to figure out the commands yourself as part of the exercise.

Hints:

  • Create a variable with the prefix
  • Create a variable with the page url
  • loop with some variable over the command to download the page and parse out (filter out) the recent file urls
    • use command substitution: for variable in $(commands)
  • in the loop, print the prefix

Expand using link to the right to see a suggested solution.

#!/bin/bash

PREFIX="http://wiki.juneday.se"
FILES_URL="http://wiki.juneday.se/mediawiki/index.php/Special:ListFiles"

# GET is an alias for "lwp-request -m GET"

for file_url in $(GET "$FILES_URL" |
                         grep TablePager_col_img_name |
                         cut -d '"' -f8)
do
    echo "$PREFIX$file_url"
done

Links

Source code

Further reading - more exercises

Where to go next

The next page is HTML which introduces basic HTML markup for a simple webpage.

« PreviousBook TOCNext »