---===[ Message from the authors: This wiki might be taken down due to maintenance ]===----


Difference between revisions of "ITIC:Introduction to Bash scripting"

From Juneday education
Jump to: navigation, search
m (Our first script)
(Some more sections)
Line 96: Line 96:
 
[[File:Screenshot nano first script.png|400px|thumb|center|The nano editor and the script]]
 
[[File:Screenshot nano first script.png|400px|thumb|center|The nano editor and the script]]
  
 +
Adding execute permissions: <source lang="Bash" inline>$ chmod u+x welcome.sh</source> and then running it:
 +
<source lang="Bash">
 +
$ ./welcome.sh
 +
Welcome rikard
 +
Your IP address is 10.0.116.35 192.168.56.1 2001:6b0:2:2801:36bd:fb4f:417b:b503
 +
newdelli
 +
</source>
 +
 +
Oops! We forgot the date! Edit the file again and add the date line. This is what the script could look like:
 +
<source lang="Bash">
 +
$ cat ./welcome.sh
 +
#!/bin/bash
 +
echo "Welcome $USER"
 +
echo -n "Time and date is "
 +
date
 +
echo -n "Your IP address is "
 +
hostname -I
 +
uname -n
 +
 +
$ ./welcome.sh
 +
Welcome rikard
 +
Time and date is tis  6 aug 2019 11:57:24 CEST
 +
Your IP address is 10.0.116.35 192.168.56.1 2001:6b0:2:2801:36bd:fb4f:417b:b503
 +
newdelli
 +
</source>
 +
 +
A few notes. We don't need to use semicolons in the script, since we can put commands on separate lines to make it easier to read. So we split up the <code>echo -n "Time and date is "</code> and <code>date</code> commands on separate lines. You can keep the semicolon and have them on the same line if you prefer.
 +
 +
A few alternatives are:
 +
<source lang="Bash">
 +
$ echo -n "Time and date is ";date
 +
Time and date is tis  6 aug 2019 12:01:53 CEST
 +
 +
$ echo "Time and date is $(date)"
 +
Time and date is tis  6 aug 2019 12:02:25 CEST
 +
</source>
 +
We think that's a matter of style and you can choose whichever feels more natural to you. Or try all three styles:
 +
* <code>echo -n "some text ";some_command</code>
 +
* <code>echo -n "some text "</code><br><code>some_command</code>
 +
* <code>echo "some text $(some_command)"</code>
 +
 +
==Evolving the script - using variables==
 +
As you may recall from the module on Bash introduction, you can create variables in Bash. This is very common in scripts, so let's evolve our script to use some variables.
 +
 +
Using variables have many advantages. Here are a few:
 +
* You can initialize a variable to a value in one place and use it in many places in your script
 +
* If you need to change the value, you now only have to change it in one place
 +
* Variables makes printing easier, in particular when you want to print something generic but a part of the text may vary
 +
 +
This is how to make a few variables for our script:
 +
<source lang="Bash">
 +
GREETING="Welcome $USER"
 +
TIME=$(date)
 +
IP=$(hostname -I)
 +
HOSTNAME=$(uname -n)
 +
echo "$GREETING"
 +
# ...etc
 +
</source>
 +
Above, we had the environment variable <code>$USER</code> as part of the new <code>GREETING</code> variable. When the value of a variable is used, you put a dollar sign before the variable name. Variables can be nested like the above (the value of a variable can contain the value of another variable etc). Then we created three additional new variables, <code>TIME</code>, <code>IP</code> and <code>HOSTNAME</code>.
 +
 +
The naming of variables is kind of flexible, but constants and environment variables usually have all caps (uppercase). A constant is a variable that you give a value once, and then only use it, never change it.
 +
 +
Normal variables are such that we give them new values often. Such variables usually have all lowercase names.
 +
<source lang="Bash">
 +
# i is a variable whose value changes
 +
$ for i in 1 2 3 4; do echo $i; done
 +
1
 +
2
 +
3
 +
4
 +
# BACKUP_DIR is a constant, never to be changed in the script
 +
BACKUP_DIR="/home/rikard/backups"
 +
# when used:
 +
for file in $(find music/ -name '*.mp3')
 +
do
 +
mv $file $BACKUP_DIR
 +
done
 +
</source>
 +
 +
This is what the script could look like if we used variables instead:
 +
<source lang="Bash">
 +
$ cat welcome.sh
 +
#!/bin/bash
 +
 +
GREETING="Welcome $USER"
 +
TIME=$(date)
 +
IP=$(hostname -I)
 +
HOST=$(uname -n)
 +
 +
# the action begins here
 +
echo "$GREETING"
 +
echo "Time and date is $TIME"
 +
echo "Your IP address is $IP and hostname is $HOST"
 +
</source>
 +
 +
You should note that you need to declare and initialize a variable ''before its value is used''. With "before", we simply means on a command line above the variable's use. First you use the following command line to declare and initialize the variable <code>GREETING</code>:
 +
<source lang="Bash">
 +
GREETING="Welcome $USER"
 +
</source>
 +
 +
After that, on a few lines below, you use the variable in another command line:
 +
<source lang="Bash">
 +
echo "$GREETING"
 +
</source>
 +
 +
Declaring and initializing a variable is a command line like any other. You can do it in the shell interactively too. You can "forget" a variable using <code>unset</code>:
 +
<source lang="Bash">
 +
$ WELCOME="welcome.sh"
 +
$ ./$WELCOME
 +
Welcome rikard
 +
Time and date is tis  6 aug 2019 12:24:54 CEST
 +
Your IP address is 10.0.116.35 192.168.56.1 2001:6b0:2:2801:36bd:fb4f:417b:b503  and hostname is newdelli
 +
$ unset WELCOME
 +
$ echo "$WELCOME"
 +
 +
$
 +
</source>
 +
 +
==Using today's date as part of a filename==
 +
When making backups, it is useful to give the backup file a name that contains the date it was being backed up. How can we do that? We'll use command substitution (command expansion)!
 +
 +
Let's pretend we are writing a small script which backs up some files. We want the backup copies to contain the backup date as part of their names, so that we can distinguish between backups and get a file version from a certain date if we need.
 +
 +
Before we begin, we need to learn a little about formatting the output from the <code>date</code> command. If we can make it output the date in a nice format (without spaces in particular, because spaces in a filename is like asking for trouble), then we can use command expansion to create a filename with today's date as part of the name.
 +
 +
To format the output from <code>date</code> we use a date format string. The format string for four digit year is <code>%Y</code>, for two digit month is <code>%m</code> and for two digit day is <code>%d</code>. We can add any text or characters between the parts of the date format symbols. So if we want a date format for e.g. 2019-08-06, we'd use the following date format string: <code>%Y-%m-%d</code>. Now, you can instruct <code>date</code> to use such a format string, by giving it as an argument starting with a plus directly followed by the format string:
 +
<source lang="Bash">
 +
$ date "+%Y-%m-%d"
 +
2019-08-06
 +
</source>
 
=Links=
 
=Links=
 
==Workshop slides==
 
==Workshop slides==

Revision as of 12:54, 6 August 2019

What is a script

Remember that Bash is a command line interpreter (a shell). The word line is central here. Bash only processes complete commands if they end with a newline character (typically you press Enter to issue the command for processing by Bash).

Bash can work interactively like this. You enter commands to be processed when you press Enter. But Bash can read files with commands too, and it can read from standard in. Here are some examples of bash working interactively, reading from standard in (using a pipe) and reading from a file called file-with-echo:

$ cat file-with-echo
echo hej
$ echo "echo hej" | bash
hej
$ bash < file-with-echo 
hej
$
$ cat file-with-echo | bash
hej
$

In the example above, we started by showing you the contents of the file file-with-echo using cat. The text inside the file was echo hej. But, hey, that's a valid command line! So if we told bash to read that file from standard in, using redirection and then also a pipe, wouldn't it execute the command line in the file? Yes it would and it did. We can even use echo and a pipe to bash to have bash execute the command from echo, as you saw in the example: echo "echo hej" | bash. Bash read echo hej from standard in, and since it ended with a newline (default behavior from echo), it just executed it. So we created a new instance of Bash, which executed the command echo hej and then exited.

You should be aware that in all of the examples above, we are running bash interactively but start a new instance of bash which reads commands and executes them, then dies (so we are back in interactive mode).

But Bash can also execute a file that contains a special first line thing called shebanb and that has execute permissions for the user. Let's look at such a file:

$ cat say_hej.sh
#!/bin/bash

echo hej

The first line is the shebang and it tells Bash how to execute this file. In fact, the shebang line tells Bash to use Bash to execute it. The rest of the file is just one single command line with echo hej. By the way, "hej" means "hi" in Swedish.

We named the file say_hej.sh using the .sh suffix. Not because we had to, but because it is a shell script, and it's a convention to name such files with the suffix .sh so that we later can expect the file to be a script, just by looking at its name. As far as Bash is concerned, the file could be called anything, like say_hej.bananas or even say_hej.pdf but that makes less sense to us humans. Some operating systems (looking at you, Windows) have put some magic into filenames, letting the suffix decide what file type they are. We, the authors, have never understood how a name can change the contents of a file (it can't, by the way). Just as if Rikard changed his name to Rebecca (which happens to be the name of his sister), it wouldn't make him a woman physically. Or if Henrik changed his name to Rikard, it wouldn't make them the same person. We think that the contents of the file should decide its file type, and so does most reasonable operating systems as well as Bash.

If we had an MP3 file called say_hej.sh, Bash would not be able to "run" it. Because Bash only understand plain text and command lines. Bash would still be able to tell an MP3 player application like mpg321 to play the MP3 file, and mpg321 would still be able to play it (if it really is an MP3 file), regardless of its name ending with .mp3 or not.

Anyway, files with execute permissions, and a shebang and some command lines are what we call scripts (or Bash scripts if the shebang is exactly #!/bin/bash.

So let's add the execute permission to the file and execute it:

$ chmod u+x say_hej.sh
$ ./say_hej.sh 
hej
$

To add execute permission for the user that owns the file, we use chmod u+x and the file as argument. To execute the file, we need to use a relative path. If we are in the same directory, the relative path will be ./filename, in our case ./say_hej.sh . This has to do with the fact that, if we want to use our script as a command, bash must know where to find the command. Bash uses the environment variable PATH to decide where to look for commands. This is actually a good thing. If we happened to have five programs called ls in different places on our computer, how would Bash know which one to use? We need the list of directories in the PATH variable to find the correct one. If you want to execute your own ls program you will have to be explicit about it and give Bash the relative or absolute path to your command. You cannot expect Bash to read your mind and understand that you meant another version of ls from the standard one in /bin/ls .

So, if our script is not in one of the directories listed in our PATH variable, we need to use a relative or absolute path to the script to help Bash find it.

In conclusion:

  • Bash scripts are just plain text files with command lines
  • They can be executed as commands if
    • they have the execute permission
    • they start with #!/bin/bash
    • we help Bash to find the script using a path to it

Our first script

Let's create a script together. The script should:

  • Print a welcome message with you user name and the current time and date
  • Print your computer's name and IP address

Now, a script is just a bunch of command lines, right? So we can actually try the command lines in the shell interactively, before we make it into a script. If they work in the shell interactively, they will work in the script.

Before we start, we'll have to tell you a few useful techniques:

  • You can tell echo not to make a newline by using the flag -n
  • You can create text (e.g. for echo to print) by using command substitution:
    • echo "Today is $(date +%A)" - prints Today is Monday if it is Monday
    • $(date +%A) will be expanded to the result of the command
  • Always use quotes around text (will spare you some headache)
  • Using a semicolon allows you to issue two commands on the same command line

When trying the commands for our script in the interactive shell, we'll use the following commands:

  • echo - to print stuff
  • hostname -I - to get the IP address
  • uname -n - to get the computer name

And the environment variable $USER contains your username.

$ echo "Welcome $USER"
Welcome rikard
$ date
tis  6 aug 2019 11:30:15 CEST
$ echo -n "Your IP address is ";hostname -I
Your IP address is 10.0.116.35 192.168.56.1 2001:6b0:2:2801:36bd:fb4f:417b:b503 
$ uname -n
newdelli
$

Feel free to try the same commands yourself. You should get different results, of course.

Now, let's make a script out of those command lines. If you want to follow what we do here, create a directory first and cd to that directory. You don't want to pollute your home directory with a lot of scripts and unrelated files.

Use an editor to create a file called welcome.sh and put the shebang on the first line of the file. Put the commands in the same order as we tested them in the file and save it. Set the execute permission on the file and execute it with ./welcome.sh. If this is the only file in the directory, you may use tab completion when executing it. Type ./w and press TAB, then enter.

This is what the file looks like when edited using nano:

The nano editor and the script

Adding execute permissions: $ chmod u+x welcome.sh and then running it:

$ ./welcome.sh 
Welcome rikard
Your IP address is 10.0.116.35 192.168.56.1 2001:6b0:2:2801:36bd:fb4f:417b:b503 
newdelli

Oops! We forgot the date! Edit the file again and add the date line. This is what the script could look like:

$ cat ./welcome.sh
#!/bin/bash
echo "Welcome $USER"
echo -n "Time and date is "
date
echo -n "Your IP address is "
hostname -I
uname -n

$ ./welcome.sh 
Welcome rikard
Time and date is tis  6 aug 2019 11:57:24 CEST
Your IP address is 10.0.116.35 192.168.56.1 2001:6b0:2:2801:36bd:fb4f:417b:b503 
newdelli

A few notes. We don't need to use semicolons in the script, since we can put commands on separate lines to make it easier to read. So we split up the echo -n "Time and date is " and date commands on separate lines. You can keep the semicolon and have them on the same line if you prefer.

A few alternatives are:

$ echo -n "Time and date is ";date
Time and date is tis  6 aug 2019 12:01:53 CEST

$ echo "Time and date is $(date)"
Time and date is tis  6 aug 2019 12:02:25 CEST

We think that's a matter of style and you can choose whichever feels more natural to you. Or try all three styles:

  • echo -n "some text ";some_command
  • echo -n "some text "
    some_command
  • echo "some text $(some_command)"

Evolving the script - using variables

As you may recall from the module on Bash introduction, you can create variables in Bash. This is very common in scripts, so let's evolve our script to use some variables.

Using variables have many advantages. Here are a few:

  • You can initialize a variable to a value in one place and use it in many places in your script
  • If you need to change the value, you now only have to change it in one place
  • Variables makes printing easier, in particular when you want to print something generic but a part of the text may vary

This is how to make a few variables for our script:

GREETING="Welcome $USER"
TIME=$(date)
IP=$(hostname -I)
HOSTNAME=$(uname -n)
echo "$GREETING"
# ...etc

Above, we had the environment variable $USER as part of the new GREETING variable. When the value of a variable is used, you put a dollar sign before the variable name. Variables can be nested like the above (the value of a variable can contain the value of another variable etc). Then we created three additional new variables, TIME, IP and HOSTNAME.

The naming of variables is kind of flexible, but constants and environment variables usually have all caps (uppercase). A constant is a variable that you give a value once, and then only use it, never change it.

Normal variables are such that we give them new values often. Such variables usually have all lowercase names.

# i is a variable whose value changes
$ for i in 1 2 3 4; do echo $i; done
1
2
3
4
# BACKUP_DIR is a constant, never to be changed in the script
BACKUP_DIR="/home/rikard/backups"
# when used:
for file in $(find music/ -name '*.mp3')
do
 mv $file $BACKUP_DIR
done

This is what the script could look like if we used variables instead:

$ cat welcome.sh
#!/bin/bash

GREETING="Welcome $USER"
TIME=$(date)
IP=$(hostname -I)
HOST=$(uname -n)

# the action begins here
echo "$GREETING"
echo "Time and date is $TIME"
echo "Your IP address is $IP and hostname is $HOST"

You should note that you need to declare and initialize a variable before its value is used. With "before", we simply means on a command line above the variable's use. First you use the following command line to declare and initialize the variable GREETING:

GREETING="Welcome $USER"

After that, on a few lines below, you use the variable in another command line:

echo "$GREETING"

Declaring and initializing a variable is a command line like any other. You can do it in the shell interactively too. You can "forget" a variable using unset:

$ WELCOME="welcome.sh"
$ ./$WELCOME
Welcome rikard
Time and date is tis  6 aug 2019 12:24:54 CEST
Your IP address is 10.0.116.35 192.168.56.1 2001:6b0:2:2801:36bd:fb4f:417b:b503  and hostname is newdelli
$ unset WELCOME
$ echo "$WELCOME"

$

Using today's date as part of a filename

When making backups, it is useful to give the backup file a name that contains the date it was being backed up. How can we do that? We'll use command substitution (command expansion)!

Let's pretend we are writing a small script which backs up some files. We want the backup copies to contain the backup date as part of their names, so that we can distinguish between backups and get a file version from a certain date if we need.

Before we begin, we need to learn a little about formatting the output from the date command. If we can make it output the date in a nice format (without spaces in particular, because spaces in a filename is like asking for trouble), then we can use command expansion to create a filename with today's date as part of the name.

To format the output from date we use a date format string. The format string for four digit year is %Y, for two digit month is %m and for two digit day is %d. We can add any text or characters between the parts of the date format symbols. So if we want a date format for e.g. 2019-08-06, we'd use the following date format string: %Y-%m-%d. Now, you can instruct date to use such a format string, by giving it as an argument starting with a plus directly followed by the format string:

$ date "+%Y-%m-%d"
2019-08-06

Links

Workshop slides

Summary lecture slides

  • TODO

Videos and video slides

  • Video: TODO
  • Video slides TODO

Further reading

Where to go next

  • TODO