Revision as of 08:31, 20 September 2019

Shells - there are many shells out there, and Bash is one of them

TODO: Write exercises before October 9 2019 (at the very latest).

There are exercises linked below to other parts of this wiki, as a backup (if we don't have time to write new ones) and also as a bonus for the ambitious students, if we do have time to write new ones.

Exercises

Basics

A shell is a program that can be run interactively, e.g. in a terminal (or terminal emulator). Describe the terms (use a search engine, our Swedish compendium or the previous page if you need help):

shell
terminal
command line
command line interface

File system navigation

Start a new terminal. Use pwd to notice where you are in the file system. What is this directory called in general terms?

Issue the following command:

$ echo ~

Notice the output from echo. It seems that Bash has expanded the ~ to a path. What path is that?

Change directory to one of the directories that are in your current directory. You use cd to change directory. But what do you need to write after the cd command (as an argument) for it to know what to do?

What happens if you, from any directory, simply issues the command cd without any arguments? Try it.

Change directory again, from your home directory to some other directory. Issue the command pwd. Then issue the command cd - (that is, cd with the argument of a single dash -). Where did you end up? Issue cd - again. Where did you end up this time? Do it again, and again, until you figure it out.

In your home directory, issue the command tree without arguments. What does it list? Next, issue the command pwd again, but give it an argument of one of the directories in your current directory (e.g. Documents, Desktop, Music, Downloads etc - try all of them or if you don't have any of them, use an existing directory as the argument).

In your home directory, use ls with the appropriate flag (or flags) to list all files and directories in a way that shows you which ones are files, and which ones are directories.

Next, issue the command ls (without any arguments) and notice what files and directories are listed. Next, issue the command ls -a and notice what additional files and directories are listed. The additional files and folders you see, look at their names and make a note of what they have in common.

Next, create a directory (you may use mkdir to do this) called .this-is-a-test (note the leading dot in the file name). Run ls without arguments. Is the new directory listed? Use ls -a to list all files again. Was the new directory listed?

Use rmdir to delete the new directory.

Issuing commands

Issuing commands means that you write the name of a command and press Enter. Often, the command needs more information. Such extra information comes in two flavors, options (also known as flags) and arguments.

Options/flags typically start with a dash (the minus sign). Arguments are normal strings. You can have both flags and arguments.

Use ls -l to see what directories you have in your current directory (your home is a good place to do this). The -l (dash lowercase L) is an option (or a flag - that's the same thing).

Use ls to list a directory. The directory is the argument to ls telling it what to list.

Use ls both with a flag (try -a, -l and -L for instance) and an argument of the same directory again. Notice the result of mixing flags and arguments.

Use echo to print the following (notice the different kinds of quotations):

$ echo ~

$ echo $HOME

$ echo "$HOME"

$ echo '$HOME'

As usual, the $ symbolizes the prompt, and shouldn't be written by you.

Create two directories, called first and second. Try the following commands (note the quotes):

$ ls -L first second

$ ls -L "first second"

$ rmdir "first second"

$ rmdir first second

Try to explain what happened. What commands worked, and what commands didn't work? Why do you think that happened?

Next, we'll see how we can temporarily change the environment a command executes in (you could think of it as tricking the command to think it executes in a different setting).

First enter the following:

$ date

What language and regional settings does it look like the date command is using?

Next, issue the following command:

$ echo "$LC_TIME"

Do you think there's a correlation between how date prints the date and time and the output from printing the environment variable LC_TIME?

Now, issue the following commands exactly as we print them below (you may use copy-paste - you can paste into the terminal using Ctrl-Insert or Ctrl-Shift-V):

$ LC_TIME="en_US.UTF-8" date

$ LC_TIME="sv_SE.utf8" date

You can see what locales are installed by typing:

$ locale -a

What happened above was that Bash set the LC_TIME environment variable to a new value temporarily for the date command. You should conclude that the behavior (output) from date relies on the value of the variable LC_TIME.

Working with text files

Preparations

Create a directory structure

Before we proceed with the exercises we need to create directories like this:

tig015/
`-- itic
    `-- working-in-the-shell
        |-- 10
        |-- 11
        |-- 12
        |-- 13
        |-- 14
        |-- 15
        |-- 16
        |-- 17
        |-- 18
        |-- 19
        |-- 20
        `-- gutenberg

You can create this directory structure in many ways but we suggest the following command:

mkdir -p tig015/itic/working-in-the-shell/{10..20} tig015/itic/working-in-the-shell/gutenberg

You can check your directory structure like this:

$ tree -d --charset=ascii tig015/
tig015/
`-- itic
    `-- working-in-the-shell
        |-- 10
        |-- 11
        |-- 12
        |-- 13
        |-- 14
        |-- 15
        |-- 16
        |-- 17
        |-- 18
        |-- 19
        |-- 20
        `-- gutenberg

14 directories

Create a file with text

Next thing we should do is to create a file with some text in it. The file, called book.txt shall be created in the tig015/itic/working-in-the-shell folder. The content of the file should be:

First of all, we're happy you're using our course material over at wiki.juneday.se.

The authors of these exercises are Rikard and Henrik.... and we suck at just about everything.

The hardest thing with these exercises was to actually come up with the text for this very text file. But since we're creative, innovative and painters of digital landscapes we're not afraid of trying. So here it is. This is the text we're going to use in some exercises.

If you're not familiar with tools to create a file with the content above here are some instructions:

Copy the text content above.

In bash enter the directory tig015/itic/working-in-the-shell with the following command:

cd tig015/itic/working-in-the-shell

In bash assuming you're in the tig015/itic/working-in-the-shell folder type the following command:

cat > book.txt

Bash (actually the command cat) is now waiting for text and to redirect that to the file book.txt.

Paste the text you copied earlier to the terminal window. Use a menu in your terminal.

Press enter and then Ctrl-d (that is first press down the key Ctrl and then press and release d and after this release the Ctrl key).

By now bash should by now display the normal prompt ending with a dollar sign (e g hesa@allan:~/tig015/itic/working-in-the-shell$ )

To make sure you have correctly created the file you can use ls:

$ ls -al book.txt 
-rw-r--r-- 1 hesa hesa 453 sep 12 15:36 book.txt

The date will and the user (hesa) will differ when you execute the ls command.

Add some real books

Enter the gutenberg directory (cd tig015/itic/working-in-the-shell/gutenberg/) and issue the following commands (assuming you have curl installed):

curl -LJO "https://www.gutenberg.org/ebooks/16328.txt.utf-8"
curl -LJO "https://www.gutenberg.org/files/1342/1342-0.txt"
curl -LJO "https://www.gutenberg.org/files/84/84-0.txt"
curl -LJO "https://www.gutenberg.org/ebooks/1080.txt.utf-8"
curl -LJO "https://www.gutenberg.org/ebooks/60271.txt.utf-8"
curl -LJO "https://www.gutenberg.org/files/2701/2701-0.txt"
curl -LJO "https://www.gutenberg.org/files/1952/1952-0.txt"
curl -LJO "https://www.gutenberg.org/files/43/43-0.txt"
curl -LJO "https://www.gutenberg.org/files/1661/1661-0.txt"
curl -LJO "https://www.gutenberg.org/files/11/11-0.txt"
curl -LJO "https://www.gutenberg.org/ebooks/851.txt.utf-8"
curl -LJO "https://www.gutenberg.org/files/98/98-0.txt"
curl -LJO "https://www.gutenberg.org/ebooks/2542.txt.utf-8"
curl -LJO "https://www.gutenberg.org/ebooks/345.txt.utf-8"
curl -LJO "https://www.gutenberg.org/ebooks/1232.txt.utf-8"
curl -LJO "https://www.gutenberg.org/ebooks/13701.txt.utf-8"
curl -LJO "https://www.gutenberg.org/files/46/46-0.txt"
curl -LJO "https://www.gutenberg.org/ebooks/5200.txt.utf-8"

Checking your preparations

If you enter the directory where you created the tig015 directory you should have a directory and file structure as below:

$ tree --charset=ascii tig015
tig015
`-- itic
    `-- working-in-the-shell
        |-- 10
        |-- 11
        |-- 12
        |-- 13
        |-- 14
        |-- 15
        |-- 16
        |-- 17
        |-- 18
        |-- 19
        |-- 20
        |-- book.txt
        `-- gutenberg
            |-- 1080.txt.utf-8
            |-- 11-0.txt
            |-- 1232.txt.utf-8
            |-- 1342-0.txt
            |-- 13701.txt.utf-8
            |-- 16328.txt.utf-8
            |-- 1661-0.txt
            |-- 1952-0.txt
            |-- 2542.txt.utf-8
            |-- 2701-0.txt
            |-- 345.txt.utf-8
            |-- 43-0.txt
            |-- 46-0.txt
            |-- 5200.txt.utf-8
            |-- 60271.txt.utf-8
            |-- 84-0.txt
            |-- 851.txt.utf-8
            `-- 98-0.txt

15 directories, 19 files

Final preparation step

Now, At Last! (this is a really great album!), we should do the last preparation. Enter the tig015/itic/working-in-the-shell/:

$ cd tig015/itic/working-in-the-shell/

Displaying content

Display the content of the file book.txt in the terminal.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ cat book.txt 
First of all, we're happy you're using our course material over at wiki.juneday.se.

The authors of these exercises are Rikard and Henrik.... and we suck at just about everything.

The hardest thing with these exercises was to actually come up with the text for this very text file. But since we're creative, innovative and painters of digital landscapes we're not afraid of trying. So here it is. This is the text we're going to use in some exercises.

Display the content of the file gutenberg/1342-0.txt in the terminal.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ cat gutenberg/1342-0.txt

Note: we do not display the content of the file displayed since this files is 13247 lines long.

Display the previous file, but this in in reversed order. That is the last line will be display first, then the second last line .... Display the content of the file gutenberg/1342-0.txt in the terminal.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ tac gutenberg/1342-0.txt

Note: we do not display the content of the file displayed since this files is still 13247 lines long.

Counting

Counting lines

How many lines are there in the file book.txt?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Let's use wc to count the lines. Using the option -l we instruct wc to count lines.

$ wc -l book.txt 
5 book.txt

So there are 5 lines.

How many lines are there in the file gutenberg/1342-0.txt?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Let's use wc to count the lines. Using the option -l we instruct wc to count lines.

$ wc -l gutenberg/1342-0.txt 
13427 gutenberg/1342-0.txt

So there are 13427 lines.

How many lines are there in all the files gutenberg directory?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Let's use wc to count the lines. Using the option -l we instruct wc to count lines. Make sure to use * to make wc (actually bash supplying all files conforming to the *):

$ wc -l gutenberg/*
    732 gutenberg/1080.txt.utf-8
   3736 gutenberg/11-0.txt
   5063 gutenberg/1232.txt.utf-8
  13427 gutenberg/1342-0.txt
   8862 gutenberg/13701.txt.utf-8
   7004 gutenberg/16328.txt.utf-8
  12310 gutenberg/1661-0.txt
   1197 gutenberg/1952-0.txt
   4325 gutenberg/2542.txt.utf-8
  22333 gutenberg/2701-0.txt
  15973 gutenberg/345.txt.utf-8
   2949 gutenberg/43-0.txt
   4238 gutenberg/46-0.txt
   2362 gutenberg/5200.txt.utf-8
   3307 gutenberg/60271.txt.utf-8
   7834 gutenberg/84-0.txt
   2066 gutenberg/851.txt.utf-8
  16272 gutenberg/98-0.txt
 133990 total

How many lines are there in all the files ending with txt the gutenberg directory? Why does the amount of files different from in the previous exercise?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Let's use wc to count the lines. Using the option -l we instruct wc to count lines. Make sure to use *.txt to make wc (actually bash supplying all files conforming to the *.txt)

$ wc -l gutenberg/*.txt
   3736 gutenberg/11-0.txt
  13427 gutenberg/1342-0.txt
  12310 gutenberg/1661-0.txt
   1197 gutenberg/1952-0.txt
  22333 gutenberg/2701-0.txt
   2949 gutenberg/43-0.txt
   4238 gutenberg/46-0.txt
   7834 gutenberg/84-0.txt
  16272 gutenberg/98-0.txt
  84296 total

The total amount of lines in the files differ from the previous exercise since all files in the gutenberg directory do not end with .txt. Some files end with txt.utf-8. Try the following command wc -l gutenberg/*.txt*.

Counting words

How many words are there in the file book.txt?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Let's use wc to count the words. Using the option -w we instruct wc to count words.

$ wc -w book.txt 
78 book.txt

So there are 78 words.

How many lines are there in the file gutenberg/1342-0.txt?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Let's use wc to count the words. Using the option -w we instruct wc to count words.

$ wc -w gutenberg/1342-0.txt
124592 gutenberg/1342-0.txt

So there are 124592 words.

Now let's count the total number of words in all the files in the gutenberg directory.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

We'll continue to use wc to count the words. Using the option -w we instruct wc to count words.

$ wc -w gutenberg/*
   6456 gutenberg/1080.txt.utf-8
  29465 gutenberg/11-0.txt
  52636 gutenberg/1232.txt.utf-8
 124592 gutenberg/1342-0.txt
  90011 gutenberg/13701.txt.utf-8
  42089 gutenberg/16328.txt.utf-8
 107602 gutenberg/1661-0.txt
   9103 gutenberg/1952-0.txt
  29482 gutenberg/2542.txt.utf-8
 215830 gutenberg/2701-0.txt
 164424 gutenberg/345.txt.utf-8
  28662 gutenberg/43-0.txt
  31568 gutenberg/46-0.txt
  25186 gutenberg/5200.txt.utf-8
  29406 gutenberg/60271.txt.utf-8
  78098 gutenberg/84-0.txt
  23219 gutenberg/851.txt.utf-8
 138883 gutenberg/98-0.txt
1226712 total

So there are 1226712 words.

Finding text in text

Ever wondered how many lines contain the letter 'a' in the file books.txt. Same here, we have never thought about that... until we wrote this exercise. Let's start with printing the lines containing an a.

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

$ grep a book.txt 
First of all, we're happy you're using our course material over at wiki.juneday.se.
The authors of these exercises are Rikard and Henrik.... and we suck at just about everything.
The hardest thing with these exercises was to actually come up with the text for this very text file. But since we're creative, innovative and painters of digital landscapes we're not afraid of trying. So here it is. This is the text we're going to use in some exercises.

So how many lines contain letters the letter 'a'?

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

We're using the grep option -c which instructs grep to not print the actual lines containing an 'a' but instead counts the number of lines.

$ grep -c a book.txt 
3

So there's 3 lines containing an 'a'.

Try the same with the letter z.

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

$ grep -c z book.txt 
0

So there's 0 lines containing an 'z'.

Let's move on to finding things in the downloaded files. After all, you could almost have counted the number of occurrences of z manually. Now, how many times can you find the word "this" in the downloaded books:

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

$ grep -c this gutenberg/*
gutenberg/1080.txt.utf-8:59
gutenberg/11-0.txt:148
gutenberg/1232.txt.utf-8:295
gutenberg/1342-0.txt:414
gutenberg/13701.txt.utf-8:942
gutenberg/16328.txt.utf-8:124
gutenberg/1661-0.txt:438
gutenberg/1952-0.txt:60
gutenberg/2542.txt.utf-8:124
gutenberg/2701-0.txt:1301
gutenberg/345.txt.utf-8:523
gutenberg/43-0.txt:149
gutenberg/46-0.txt:144
gutenberg/5200.txt.utf-8:116
gutenberg/60271.txt.utf-8:201
gutenberg/84-0.txt:372
gutenberg/851.txt.utf-8:128
gutenberg/98-0.txt:527

Did we just find the occurrences of the word this or the letter this? Actually, when searching for this with the command $ grep -c this gutenberg/* we're looking for every occurrence of this including for example sympathise. If we want to search for this as a word we need to add an option to grep. Try yourself.

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

$ grep -c -w this gutenberg/*
gutenberg/1080.txt.utf-8:59
gutenberg/11-0.txt:146
gutenberg/1232.txt.utf-8:295
gutenberg/1342-0.txt:408
gutenberg/13701.txt.utf-8:942
gutenberg/16328.txt.utf-8:124
gutenberg/1661-0.txt:438
gutenberg/1952-0.txt:60
gutenberg/2542.txt.utf-8:124
gutenberg/2701-0.txt:1297
gutenberg/345.txt.utf-8:521
gutenberg/43-0.txt:149
gutenberg/46-0.txt:144
gutenberg/5200.txt.utf-8:116
gutenberg/60271.txt.utf-8:201
gutenberg/84-0.txt:360
gutenberg/851.txt.utf-8:128
gutenberg/98-0.txt:525

Are we done. No. What if a sentence begins with this. Then we have a capital t (T) in the beginning of this. Try this yourself.

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

We can do this with a regular expression:

$ grep -c -w "[Tt]his" gutenberg/* 
gutenberg/1080.txt.utf-8:67
gutenberg/11-0.txt:168
gutenberg/1232.txt.utf-8:355
gutenberg/1342-0.txt:476
gutenberg/13701.txt.utf-8:1097
gutenberg/16328.txt.utf-8:172
gutenberg/1661-0.txt:525
gutenberg/1952-0.txt:70
gutenberg/2542.txt.utf-8:141
gutenberg/2701-0.txt:1404
gutenberg/345.txt.utf-8:660
gutenberg/43-0.txt:185
gutenberg/46-0.txt:161
gutenberg/5200.txt.utf-8:136
gutenberg/60271.txt.utf-8:269
gutenberg/84-0.txt:440
gutenberg/851.txt.utf-8:142
gutenberg/98-0.txt:584

... or we can do it with two expressions:

$ grep -c -w -e "this" -e "This" gutenberg/* 
gutenberg/1080.txt.utf-8:67
gutenberg/11-0.txt:168
gutenberg/1232.txt.utf-8:355
gutenberg/1342-0.txt:476
gutenberg/13701.txt.utf-8:1097
gutenberg/16328.txt.utf-8:172
gutenberg/1661-0.txt:525
gutenberg/1952-0.txt:70
gutenberg/2542.txt.utf-8:141
gutenberg/2701-0.txt:1404
gutenberg/345.txt.utf-8:660
gutenberg/43-0.txt:185
gutenberg/46-0.txt:161
gutenberg/5200.txt.utf-8:136
gutenberg/60271.txt.utf-8:269
gutenberg/84-0.txt:440
gutenberg/851.txt.utf-8:142
gutenberg/98-0.txt:584

Parts of a a file

Display the first two lines of the file book.txt

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

$ head -2 book.txt 
First of all, we're happy you're using our course material over at wiki.juneday.se.

Display the last three lines of the file book.txt

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

$ tail -3 book.txt 
The authors of these exercises are Rikard and Henrik.... and we suck at just about everything.

The hardest thing with these exercises was to actually come up with the text for this very text file. But since we're creative, innovative and painters of digital landscapes we're not afraid of trying. So here it is. This is the text we're going to use in some exercises.

Note: This exercise is a bit of peak preview on the coming exercises on pipes. A bit harder question this time. Display line number two and only that line from the file . Image if we could do this in two steps:

display the first two lines
and from them display the last

This corresponds to:

head -2
tail -1

By using so called pipes we can achieve this. Try issuing the following command:

$ head -2 gutenberg/2701-0.txt 

The Project Gutenberg EBook of Moby Dick; or The Whale, by Herman

By using pipes we can make the output of the command above (the two lines) become input to the next command. Let's try:

$ head -2 gutenberg/2701-0.txt | tail -1
The Project Gutenberg EBook of Moby Dick; or The Whale, by Herman

Adjusting text

Sorting

Sort the lines in the file gutenberg/2701-0.txt alphabetical order (lexicographic order). We can't come up with a reason for you to do that apart from the obvious reason - you need to practice.

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

$ sort gutenberg/2701-0.txt

We're not showing the display text. After all, the files contains more than 22 000 lines.

Sort the lines in in the file gutenberg/2701-0.txt reversed alphabetical order. Again, we can't come up with a reason for you to do that apart from the obvious reason - you need to practice.

Expand using link to the right to see an explanation and a suggested solution for how to handle this problem.

$ sort -r gutenberg/2701-0.txt

We're not showing the display text now either.

Combining commands with pipes

Earlier we search for the word this (including This) using grep -c -w "[Tt]his" gutenberg/* . We then got the number of occurrences per files. This time we should search for [Tt]his in all files in the gutenberg directory and count the total amount of occurrences.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ grep -w "[Tt]his" gutenberg/*  | wc -l
7052

How many times are thou (including Thou) mentioned?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ grep -w "[Tt]hou" gutenberg/*  | wc -l
362

Of the lines containing thou (including Thou), print the lines that also contains the word rather (including Rather)?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ grep -w "[Tt]hou" gutenberg/*  | grep "[Rr]ather"
gutenberg/2701-0.txt:thou rather work in clay?
gutenberg/84-0.txt:I ought to be thy Adam, but I am rather the fallen angel, whom thou

Count the number of lines in the previous exercise (no not manually, using pipes and wc).

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ grep -w "[Tt]hou" gutenberg/*  | grep "[Rr]ather" | wc -l
2

Search for [Tt]his in all files in the gutenberg directory. Use only grep so we get an individual count per file. Sort the list alphabetically (lexicographical order) after file name.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ grep -c -w "[Tt]his" gutenberg/*  | sort
gutenberg/1080.txt.utf-8:67
gutenberg/11-0.txt:168
gutenberg/1232.txt.utf-8:355
gutenberg/1342-0.txt:476
gutenberg/13701.txt.utf-8:1097
gutenberg/16328.txt.utf-8:172
gutenberg/1661-0.txt:525
gutenberg/1952-0.txt:70
gutenberg/2542.txt.utf-8:141
gutenberg/2701-0.txt:1404
gutenberg/345.txt.utf-8:660
gutenberg/43-0.txt:185
gutenberg/46-0.txt:161
gutenberg/5200.txt.utf-8:136
gutenberg/60271.txt.utf-8:269
gutenberg/84-0.txt:440
gutenberg/851.txt.utf-8:142
gutenberg/98-0.txt:584

This time sort them in order after the number of occurrences (the last part of the line) and find the file with most occurrences. We're going to do this in a few steps. Let's start with putting the number of lines first in the line (using rev)

Hint: the proposed solution will turn out to be problematic.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ grep -c -w "[Tt]his" gutenberg/*  | rev 
76:8-ftu.txt.0801/grebnetug
861:txt.0-11/grebnetug
553:8-ftu.txt.2321/grebnetug
674:txt.0-2431/grebnetug
7901:8-ftu.txt.10731/grebnetug
271:8-ftu.txt.82361/grebnetug
525:txt.0-1661/grebnetug
07:txt.0-2591/grebnetug
141:8-ftu.txt.2452/grebnetug
4041:txt.0-1072/grebnetug
066:8-ftu.txt.543/grebnetug
581:txt.0-34/grebnetug
161:txt.0-64/grebnetug
631:8-ftu.txt.0025/grebnetug
962:8-ftu.txt.17206/grebnetug
044:txt.0-48/grebnetug
241:8-ftu.txt.158/grebnetug
485:txt.0-89/grebnetug

But hey, even the numbers are reversed. That is 76 is now 67 and this will of course impact the sorting.

Note: One of the authors (Henrik!!!) was quite happy with the solution above. Until someone else (Rikard!!!) pointed out how stupid Henrik is.

Now use sort alone instead. We can use the fact that none of the files contain a comma (":") and grep uses comma to separate the filename and the line count.

Hint: we need to instruct sort to use comma to separate fields instead of using blank/space. Doing this we can sort on the second column (the line count).

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ grep -c -w "[Tt]his" gutenberg/* | sort -t ":" -k 2
gutenberg/13701.txt.utf-8:1097
gutenberg/5200.txt.utf-8:136
gutenberg/2701-0.txt:1404
gutenberg/2542.txt.utf-8:141
gutenberg/851.txt.utf-8:142
gutenberg/46-0.txt:161
gutenberg/11-0.txt:168
gutenberg/16328.txt.utf-8:172
gutenberg/43-0.txt:185
gutenberg/60271.txt.utf-8:269
gutenberg/1232.txt.utf-8:355
gutenberg/84-0.txt:440
gutenberg/1342-0.txt:476
gutenberg/1661-0.txt:525
gutenberg/98-0.txt:584
gutenberg/345.txt.utf-8:660
gutenberg/1080.txt.utf-8:67
gutenberg/1952-0.txt:70

1097, 136, 1404 .... does not seem correct? We need to sort numerically.

$ grep -c -w "[Tt]his" gutenberg/* | sort -n -t ":" -k 2
gutenberg/1080.txt.utf-8:67
gutenberg/1952-0.txt:70
gutenberg/5200.txt.utf-8:136
gutenberg/2542.txt.utf-8:141
gutenberg/851.txt.utf-8:142
gutenberg/46-0.txt:161
gutenberg/11-0.txt:168
gutenberg/16328.txt.utf-8:172
gutenberg/43-0.txt:185
gutenberg/60271.txt.utf-8:269
gutenberg/1232.txt.utf-8:355
gutenberg/84-0.txt:440
gutenberg/1342-0.txt:476
gutenberg/1661-0.txt:525
gutenberg/98-0.txt:584
gutenberg/345.txt.utf-8:660
gutenberg/13701.txt.utf-8:1097
gutenberg/2701-0.txt:1404

That's more like it.

Now use tail to display only the last line (i e the files with most occurrences of the word this (including This).

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ grep -c -w "[Tt]his" gutenberg/* | sort -n -t ":" -k 2 | tail -1
gutenberg/2701-0.txt:1404

Editing the commands line and some other tricks

List the files and directories in the current folder.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ ls
10  11	12  13	14  15	16  17	18  19	20  book.txt  gutenberg

List the files and directories in the current folder. But this time you're only allowed to use the arrow keys (and Enter).

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Press arrow-up and you should see the previous command (ls). Press Enter.

List the files and directories in the current folder. But this time you're only allowed to use ! and Enter.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Type !! and press Enter.

Display what directory you're in.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ pwd

List the files and directories in the current folder. But this time you must use ! and l and, of course, Enter.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Type !l and press Enter.

$ !l
ls
10  11	12  13	14  15	16  17	18  19	20  book.txt  gutenberg

Display what directory you're in using ! and p (and Enter).

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ !p
pwd
/home/hesa/tig015/itic/working-in-the-shell

List the files and directories in the current folder. But this time you must search for a command and Enter.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

Type Ctrl-r and type the letter 'l'. You should now see something like (reverse-i-search)`l': ls. Press Enter.

TODO: Exercises on editing the command line - moving around, deleting and pasting parts of the command line etc. Esc-. For now, you will have to read the previous page and use the examples there as an exercise.

Exit status

Let's say you want to list the files on your home directory. This can be done in different ways.

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ cd
$ ls

Use ls and supply your home directory:

$ ls ~

The question of this exercise is - what exit status do you get from ls when listing directories and files on your home directory?

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls  ~/
 Android		 bin	   Desktop     edu     Pictures    tig015  'VirtualBox VMs'
 AndroidStudioProjects	 cert	   Documents   Music   Public	   tmp
 Arduino		 crypted   Downloads   opt     Templates   Videos
$ echo $?
0

The exit status is 0. This means the command succeeded. Note: the files listed are files in one of the author's home directory. This will most likely differ from your home directory.

What exit status do you get from ls when listing directories and files in your home directory and in a non existing directory?

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls  ~/ ~/this-most-likely-does-not-exist
ls: cannot access '/home/hesa/this-most-likely-does-not-exist': No such file or directory
/home/hesa/:
 Android		 bin	   Desktop     edu     Pictures    tig015  'VirtualBox VMs'
 AndroidStudioProjects	 cert	   Documents   Music   Public	   tmp
 Arduino		 crypted   Downloads   opt     Templates   Videos
hesa@berio:~/tig015/itic/working-in-the-shell$ echo $?
2

The exit status is 2. This means the command did NOT succeed.

Write a command line that lists the files in your home directory and:

if it fails, echoes the text "FAILURE"
if it succeeds, echoes the text "SUCCESS"

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls  ~/  && echo SUCCESS || echo "FAILURE"
 Android		 bin	   Desktop     edu     Pictures    tig015  'VirtualBox VMs'
 AndroidStudioProjects	 cert	   Documents   Music   Public	   tmp
 Arduino		 crypted   Downloads   opt     Templates   Videos
SUCCESS

If you chance the order of the following && echo SUCCESS || echo "FAILURE" into || echo "FAILURE" && echo "SUCCESS" it will work in this very case.

Use the same command but for a non-existing directory, ~/this-most-likely-does-not-exist:

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls  ~/this-most-likely-does-not-exist  && echo SUCCESS || echo "FAILURE"
ls: cannot access '/home/hesa/this-most-likely-does-not-exist': No such file or directory
FAILURE

So if the command (ls ~/this-most-likely-does-not-exist) fails the part after || is executed. If you (again) chance the order of the following && echo SUCCESS || echo "FAILURE" into || echo "FAILURE" && echo "SUCCESS" it will NOT work since echo "FAILURE" will be executed (since ls failed) and that echoing itself works fine so <code>echo "SUCCESS" is also executed:

$ ls  ~/this-most-likely-does-not-exist || echo "FAILURE" && echo "SUCCESS"
ls: cannot access '/home/hesa/this-most-likely-does-not-exist': No such file or directory
FAILURE
SUCCESS

Confused about the last things?. Don't worry, you'll get over it. It may take a while though.

Globbing and expansion

Globbing

We've practiced using globbing already (e g when we listed files using *.txt*). Let's practice some more on globbing.

List (long listing using -l) all the files in the gutenberg directory (no globbing needed).

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls -l gutenberg/
total 6972
-rw-r--r-- 1 hesa hesa   39700 sep.  13 18:51 1080.txt.utf-8
-rw-r--r-- 1 hesa hesa  173595 sep.  13 18:51 11-0.txt
-rw-r--r-- 1 hesa hesa  305853 sep.  13 18:51 1232.txt.utf-8
-rw-r--r-- 1 hesa hesa  724726 sep.  13 18:51 1342-0.txt
-rw-r--r-- 1 hesa hesa  529453 sep.  13 18:52 13701.txt.utf-8
-rw-r--r-- 1 hesa hesa  301063 sep.  13 18:51 16328.txt.utf-8
-rw-r--r-- 1 hesa hesa  607788 sep.  13 18:51 1661-0.txt
-rw-r--r-- 1 hesa hesa   51185 sep.  13 18:51 1952-0.txt
-rw-r--r-- 1 hesa hesa  165572 sep.  13 18:51 2542.txt.utf-8
-rw-r--r-- 1 hesa hesa 1276201 sep.  13 18:51 2701-0.txt
-rw-r--r-- 1 hesa hesa  883160 sep.  13 18:51 345.txt.utf-8
-rw-r--r-- 1 hesa hesa  163463 sep.  13 18:51 43-0.txt
-rw-r--r-- 1 hesa hesa  182057 sep.  13 18:52 46-0.txt
-rw-r--r-- 1 hesa hesa  141420 sep.  13 18:52 5200.txt.utf-8
-rw-r--r-- 1 hesa hesa  178224 sep.  13 18:51 60271.txt.utf-8
-rw-r--r-- 1 hesa hesa  450783 sep.  13 18:51 84-0.txt
-rw-r--r-- 1 hesa hesa  125852 sep.  13 18:51 851.txt.utf-8
-rw-r--r-- 1 hesa hesa  804335 sep.  13 18:51 98-0.txt

List (long listing using ) all files beginning with 1 in the gutenberg directory (globbing needed).

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls -l gutenberg/1*
-rw-r--r-- 1 hesa hesa  39700 sep.  13 18:51 gutenberg/1080.txt.utf-8
-rw-r--r-- 1 hesa hesa 173595 sep.  13 18:51 gutenberg/11-0.txt
-rw-r--r-- 1 hesa hesa 305853 sep.  13 18:51 gutenberg/1232.txt.utf-8
-rw-r--r-- 1 hesa hesa 724726 sep.  13 18:51 gutenberg/1342-0.txt
-rw-r--r-- 1 hesa hesa 529453 sep.  13 18:52 gutenberg/13701.txt.utf-8
-rw-r--r-- 1 hesa hesa 301063 sep.  13 18:51 gutenberg/16328.txt.utf-8
-rw-r--r-- 1 hesa hesa 607788 sep.  13 18:51 gutenberg/1661-0.txt
-rw-r--r-- 1 hesa hesa  51185 sep.  13 18:51 gutenberg/1952-0.txt

List (long listing) all files beginning with 1 and ending with .txt in the gutenberg directory (globbing needed).

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls -l gutenberg/1*.txt
-rw-r--r-- 1 hesa hesa 173595 sep.  13 18:51 gutenberg/11-0.txt
-rw-r--r-- 1 hesa hesa 724726 sep.  13 18:51 gutenberg/1342-0.txt
-rw-r--r-- 1 hesa hesa 607788 sep.  13 18:51 gutenberg/1661-0.txt
-rw-r--r-- 1 hesa hesa  51185 sep.  13 18:51 gutenberg/1952-0.txt

List (long listing) all files beginning with 9 somewhere in the name in the gutenberg directory (globbing needed).

Hint: 9 somewhere means the file name could begin with anything, followed by 9 and ending with anything.

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls -l gutenberg/*9*
-rw-r--r-- 1 hesa hesa  51185 sep.  13 18:51 gutenberg/1952-0.txt
-rw-r--r-- 1 hesa hesa 804335 sep.  13 18:51 gutenberg/98-0.txt

List (long listing) all files beginning with 9 OR 2 somewhere in the name in the gutenberg directory (globbing needed).

Hint: 2 OR 9 somewhere means the file name could begin with anything, followed by 9 or 2 and ending with anything.

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls -l gutenberg/*[29]*
-rw-r--r-- 1 hesa hesa  305853 sep.  13 18:51 gutenberg/1232.txt.utf-8
-rw-r--r-- 1 hesa hesa  724726 sep.  13 18:51 gutenberg/1342-0.txt
-rw-r--r-- 1 hesa hesa  301063 sep.  13 18:51 gutenberg/16328.txt.utf-8
-rw-r--r-- 1 hesa hesa   51185 sep.  13 18:51 gutenberg/1952-0.txt
-rw-r--r-- 1 hesa hesa  165572 sep.  13 18:51 gutenberg/2542.txt.utf-8
-rw-r--r-- 1 hesa hesa 1276201 sep.  13 18:51 gutenberg/2701-0.txt
-rw-r--r-- 1 hesa hesa  141420 sep.  13 18:52 gutenberg/5200.txt.utf-8
-rw-r--r-- 1 hesa hesa  178224 sep.  13 18:51 gutenberg/60271.txt.utf-8
-rw-r--r-- 1 hesa hesa  804335 sep.  13 18:51 gutenberg/98-0.txt

How many files begins with 9 OR 2 somewhere in the name in the gutenberg directory (globbing and pipes needed).

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ ls -l gutenberg/*[92]*| wc -l
9

So there are 9 such files.

Think we've covered globbing now. Let's move on to brace expansion.

Brace expansion

We've seen in the previous chapter how we could create files with the brace expansion. Is that all we can do? No, but it gives you a visible result. So let's do that for a short while.

Create 10 files, beginning with example_ followed by the numbers 0 to 9.

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ touch example_{0..9}
$ ls -l
total 52
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 10
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 11
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 12
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 13
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 14
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 15
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 16
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 17
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 18
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 19
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:49 20
-rw-r--r-- 1 hesa hesa  453 sep.  13 18:50 book.txt
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_0
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_1
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_2
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_3
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_4
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_5
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_6
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_7
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_8
-rw-r--r-- 1 hesa hesa    0 sep.  17 14:35 example_9
drwxr-xr-x 2 hesa hesa 4096 sep.  13 18:52 gutenberg

So, 10 files we're created.

Let's create a directory structure for 100 students (student0, student1, ... student99). The structure per student shall look the following.

student/
|-- bin
|-- doc
|-- report
`-- src

The student directories should be placed in a directory called students.

Expand using link to the right to see an explanation and some suggested solutions for how to do this.

Enter your home directory and use ls:

$ mkdir -p students/student{0..99}/{bin,doc,report,src}
$ tree --charset=ascii -d students/
students/
|-- student0
|   |-- bin
|   |-- doc
|   |-- report
|   `-- src
|-- student1
|   |-- bin
|   |-- doc
|   |-- report
|   `-- src
....... snip
|-- student97
|   |-- bin
|   |-- doc
|   |-- report
|   `-- src
|-- student98
|   |-- bin
|   |-- doc
|   |-- report
|   `-- src
`-- student99
    |-- bin
    |-- doc
    |-- report
    `-- src

500 directories

We cut away (see the snip note above) most of the directory listed using tree. The displayed text looks correct. This and similar commands can save a lot of time for us....but we know it is not every day you're asked to create this many directories.

Think about the time it would take to create these directories manually using some GUI. Now imagine create more directories and for more students and possibly also for more classes..... We figure you get the point.

Environment variables and custom variables (and their expansion)

What is your LANG variable set to?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ echo $LANG
en_US.UTF-8

The author's LANG variable is set to en_US.UTF-8. Your may be set to something else. The important thing here is to know hoe to display the variable.

Display the current date.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ date
Tue Sep 17 17:47:34 CEST 2019

The author's environment variable LC_TIME is set to en_US.UTF-8. This influences the way date prints the date so if you get another printout.

Display the current date the American way - using LC_TIME.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

We can set the variable LC_TIME is set to en_US.UTF-8 just when executing the command date

$ LC_TIME=en_US.UTF-8 date
Tue Sep 17 17:47:59 CEST 2019

Display the current date the Swedish way - using LC_TIME.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

We can set the variable LC_TIME is set to en_US.UTF-8 just when executing the command date

$ LC_TIME=sv_SE.UTF-8 date
tis 17 sep 2019 17:53:31 CEST

Note: you may have to manually install the files for the Swedish translations. To do this, execute the following (assuming you're using Ubuntu): sudo apt-get install language-pack-sv language-pack-sv-base.

Display the current date the Azerbaijani - using LC_TIME.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

We can set the variable LC_TIME is set to az_AZ.UTF-8 just when executing the command date

$ LC_TIME=az_AZ.UTF-8 date
çax Sen 17 17:57:03 CEST 2019

Note: you may have to manually install the files for the translations for the Azerbaijani language. To do this, execute the following (assuming you're using Ubuntu): sudo apt-get install language-pack-az language-pack-az-base.

Now your task is to use the number of students you have in the students folder you created in previous exercises. We know this and can answer directly that the answer is 100. Create a variable, NR_STUDENTS, to keep this value.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ NR_STUDENTS=100

If you weren't sure about how many students you had in the folder you could, of course, bash and friends to count for you.

$ ls -1 students/| wc -l
100

You could also set the value directly. Like this:

$ NR_STUDENTS=$(ls -1 students/| wc -l)

What we're doing is to instruct bash to place the output to stdout (100 in our specific case) from ls -1 students/| wc -l in the variable NR_STUDENTS using $().

Print the value of the variable you just created. NR_STUDENTS, to keep this value.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ echo $NR_STUDENTS
100

Update the variable NR_STUDENTS to 1000 and print the value of the variable you just created. NR_STUDENTS, to keep this value.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ NR_STUDENTS=1000
$ echo $NR_STUDENTS 
1000

Here's a tip: Let's say you started to write echo $NR and you're too bored writing the remaining _STUDENTS. Then you can bash's completion by pressing the tab key. If you're lucky (only one variable starting with NR) then bash will fill in (complete) the rest for you.

Display the root user's home directory using ~.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ echo ~root
/root

Enter the directory students. You could back to the previous directory using cd - (see earlier exercises). Here's a small sample session:

$ pwd
/home/hesa/tig015/itic/working-in-the-shell
$ cd students/
$ pwd
/home/hesa/tig015/itic/working-in-the-shell/students
$ cd -
/home/hesa/tig015/itic/working-in-the-shell
$ pwd
/home/hesa/tig015/itic/working-in-the-shell

You saw the environment variable OLDPWD in the previous chapter ITIC:Working in the shell - Introduction to Bash. Enter the students directory and check (display) the value of the variable OLDPWD. What is the connection between this value and the directory you'll get to when typing cd -?

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ cd students/
$ echo $OLDPWD
/home/hesa/tig015/itic/working-in-the-shell

The connection is that you will be taken to the directory stored in OLDPWD when issuing the command cd -.

Stay in the students directory and update the variable OLDPWD with the value /tmp. What do you think will happen if you do cd -? Try it!

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ OLDPWD=/tmp
$ cd -
/tmp
$ pwd
/tmp

The connection is that you will be taken to the directory stored in OLDPWD when issuing the command cd -.

Now, do cd - one more time. Before you do this - spend some time thinking about where you think you will be taken (to which directory) and why.

Expand using link to the right to see an explanation and a suggested solution for how to do this.

$ pwd
/tmp
$ echo $OLDPWD
/home/hesa/tig015/itic/working-in-the-shell/students
$ cd -
/home/hesa/tig015/itic/working-in-the-shell/students
$ pwd
/home/hesa/tig015/itic/working-in-the-shell/students

You will be taken to the previous directory which is the directory stored in OLDPWD.

Network

We will add exercises here when we have the time. See below for now.

Or we save this to the Network commands chapter.

Swedish exercises

Here are links to our Swedish compendium (which contains a lot of Bash), and the companion Swedish exercise PDF (which has a lot of Bash exercises).

Datorkunskap för IT-studenter.pdf (102 pages)
- Övningshäfte_-_Datorkunskap_för_IT-studenter (Swedish) (44 pages)

Bonus exercises

These are some recommended exercise pages from other parts of this wiki:

Links

Where to go next

The next page is Networks_and_protocols.

« Previous • Book TOC • Next »

@@ Line 928: / Line 928: @@
 </div>
 </div>
+TODO: Exercises on editing the command line - moving around, deleting and pasting parts of the command line etc. Esc-. For now, you will have to read the previous page and use the examples there as an exercise.
 == Exit status ==

Difference between revisions of "ITIC:Working in the shell - Introduction to Bash - Exercises"

Revision as of 08:31, 20 September 2019

Contents

Exercises

Basics

File system navigation

Issuing commands

Working with text files

Preparations

Create a directory structure

Create a file with text

Add some real books

Checking your preparations

Final preparation step

Displaying content

Counting

Counting lines

Counting words

Finding text in text

Parts of a a file

Adjusting text

Sorting

Combining commands with pipes

Editing the commands line and some other tricks

Exit status

Globbing and expansion

Globbing

Brace expansion

Environment variables and custom variables (and their expansion)

Network

Swedish exercises

Bonus exercises

Links

Where to go next

Navigation menu

Search