This tutorial is a three part series as an introduction to Bash Shell Scripting. Part I is mainly basics of shell scripting and is generic, Part II will gradually move towards some more advanced techniques and focus mostly on the bash shell and finally in Part III we will try to use all the techniques to create working program.

A basic understanding of Linux shell is required for this tutorial. Please refer to Introduction to Linux Shell and Introduction to Shell Environment to learn more about Linux shell.

 

Shell Scripting

The Linux shell includes a full-fledged programming language, A high level language that enables the user to execute sequence of commands, parse their output and change it’s flow based on the command’s output. These programs are referred as shell scripts. Unlike most other programming languages like C, the shell programs are not compiled but are run directly by the interpreter which is in this case is the shell.

Uses of Shell Scripts

Shell scripts uses common utilities that come with the system thus reducing the development time and that are why they are good for rapid prototyping. Apart from this they are also good for automation e.g. System update, Backup etc.
Despite their less development time shell scripts aren’t used for deployment work or final product as they tends to be less efficient compared to the corresponding compiled program. Also shell scripts cause the problem of portability. As the shell uses the system utilities they are highly dependent on the system, and their environment, they were created on.

Understanding Shell Commands

To understand shell scripting lets starts with some information on how shell interprets the command you enter at the prompt.  Any input received at the prompt is considered a command. There can be two kind of commands and both are interpreted differently.

Internal Command or Built-in command

These are commands that are built in to shell and are a part of the shell source code. They mostly work to change the shell environment (variables). To make any change to these commands we have to recompile the shell e.g.

$ cd /home

This command changes the current working directory* to /home.
* The current working directory is kept in the environment in PWD.

External Command

External commands are the executable files that are part of the file system. We can simply replace the executable with the new one or remove one without affecting the functionality of the shell e.g.

$ ls program.c

This command tells the shell to execute the executable file ls with arguments program.c. If the input command is not a built-in command the shell looks in all the directories listed in the PATH environment variable in order and when it finds one it executes it with the arguments provided with the command*. These commands can be anything from binary executable to text files called scripts.

* The shell also checks for the executable bit in the file permission.

The command “type” displays information about command type. Try help type for more information.

Executing Shell Scripts

Any Script like other executable must have its executable bit set. It can be executed in two ways.

Executing in current shell

The script can be executed under the current shell by using the dot (.) command or source commands.
e.g.

$ source script_name.sh

or

$ . script_name.sh

The advantage of running script in the context of current shell is that the change in environment is persistent. For this method the script does not need its executable bit to be set.

Executing in a sub shell

To execute a script in a sub shell, enter the script full name on the prompt. In this way a new child shell is created and the script is run in the context of this shell. Another way to run the script is to pass is as an argument to shell.
e.g.

$ bash script_name.sh

When script is running in a sub shell the change in environment variables are not persistent.

e.g. Consider following script named chdir.sh with only one statement.

cd " $1"

If I run the following command the current working directory will not change

$ bash chdir.sh /home

But following command will change the current working directory to /home

$ . chdir.sh /home

Shell Scripting Basics

Comments

To make comments in a shell we use # symbol*. Anything after # is considered a comment and is ignored by the shell except when it is in quote.
e.g. Consider the following script which prints the files in the users home directory

#!/bin/bash
# This is an example of the comment in a script
# cd /home
ls $USERNAME

If we run above scripts it will fail because the third line won’t run and the directory won’t change to /home. Instead this script will run $USERNAME file in the current working directory.
* Not just shell scripts, even config file in Linux mostly uses # for comments.

SHA-BANG

In above script the first line is a special line which in not a comment. It is called SHA-BANG. The symbol #! Tells the shells path to the interpreter which will be used to run the script. Here are some of the examples of other scripts.

#!/bin/perl , #!/bin/tcsh , #!/bin/ksh

Variables

Like other programming languages shell has variables too. But unlike programming languages they do not have any type attached to them. We do not even need to declare the variables. All the variables hold the data in the form of text. e.g.

x=10

The variable x has the value 10 but its not the integer 10 but rather it’s the text 10. To access the variables value use the dollar sign in front of the variables name. e.g. $x or ${x}
e.g. To print the value of x

$ echo $x

Environment Variables

Environment variables or the global variables are the variables that are available to all the shell scripts. They are defined in your shell configuration file like bashrc for bash e.g. PATH, HOME. Command env or printenv prints all the environment variables.

Local Variables

Local variables are confined to the scripts they are defined in, set command without any arguments prints all the global and local variables defined in the context of current shell.

Special Variables

Apart from global and local variables that we defined there are some automatic variables that get their values automatically.

Command Line Arguments

To access command line arguments to your script use $1, $2, $3 …
e.g.

echo $1
echo $2
echo $3

will prints the first, second and third arguments. We can access up to 9 arguments this way. To use more than 9 argument use shift command. Shift command discard the 1st argument and convert 2nd argument to 1st, 3rd to 2nd, 4th to 3rd and so on. e.g.

echo $1
shift
echi $1
shift 
echi $1

This script will show the same results as shown by the first script.
The shift command does not change the 0th argument. 0th argument always holds the name of the script.

Conditional Execution

If command

Like the other programming languages shell programming has if command for conditional execution.

if command0
then

    command1

    command2

fi

The above program works as follows :

• The if command runs the command0 and if it is successful only then run the commands command1 and command2.
• We can include as many command before fi .
• The fi terminates the if block.
• As you know in Linux any command on successful execution returns 0 and any other value for some error. So if command only tests the output comand0 to 0.

To understand lets create a simple script which will enter in to a directory and delete file specified on the command line.

#!/bin/bash
#remove.sh
# The first argument is name of directory and second and third are file names
dir=$1
filename1=$2
filename2=$3
if cd $dir
then

    if rm $filename1

    then

        echo "$filename1 deleted successfully"

    fi

    if rm $filename2

    then

        echo "$filename1 deleted successfully"

    fi

fi

Let’s analyze the above script. We are first changing in to the directory specified in the first arguments and if the directory exists only then delete the files and print the Success.

test command

test is a command which is used to evaluates expressions and returns 0 if it is true and non-zero if it is false. This command comes in really handy when working with if command. There are different tests that can be done with this command. Some of them are:

Integer tests.

 

String test

 

Lets modify the above script to use test command

#!/bin/bash
# remove.sh
# The first argument is name of directory and second and third are file names
dir=$1
filename1=$2
filename2=$3
cd $dir
if test $? –eq 0 
then

    rm $filename1

    if test $? –eq 0

    then

        echo "$filename1 delete successfully"

    fi

    rm $filename2

    if test $? –eq 0

    then

       echo "$filename1 delete successfully"

    fi

fi

In above script we are first executing the commands and then checking the exit status of command through special variable $?.

File Test

These are some file or directory tests

Let’s create a script which will check if the argument is file or directory and delete it using appropriate command.

#!/bin/bash
#remove.sh
# The usage: ./remove.sh filename
filename=$1
if test –d $filename
then

    CMD=rdirm $filename

fi
if test –f $filename
then

    CMD=rm $filename

fi
$CMD
if test $? –eq 0
then

    echo "$filename deleted successfully"

fi

Explanation

• First assign first argument to variable filename.
• After that check if it’s a directory if it’s a directory then set CMD to rmdir $filename
• Check if its regular file if it is then set CMD to rm $filename
• After that run the CMD and checks its status.

Other Operators

There are some logical operators

 

[] command

[] command is just another way of using the test command. E.g.

If [ $? –eq 0 ] 
then

    command1

    command2

fi

Pay special attention to spaces around the brackets. It is an error to not place a space around brackets.

Else

With If command we can also use the else command which will run if test evaluates to false.
e.g.

if command
then

    command1

else

    command2

fi

Above if command returns successfully then command1 will run otherwise command2 will run.

Elif command

Elif command is a combination of else if in one single command, its use is as follows.

if command0
then

    command1

elif command2
then

    command3

else

    command4

fi

Explanation:

• Run Command0, if it is successful run command1
• Else run command2 and if it is successful run command3
• Else run command4

Loop

The bash shell has number of ways to creates loops.

While Command

While command keep on executing the commands placed between do and done until its arguments returns non zero.

while command0
do

    Command1

    Command2

    Command3

done

Like if command we can also use the test command here. Let’s creates a script which will print its arguments.

#!/bin/bash
#print.sh
# The usage: ./print.sh arg1 arg2 arg3 arg4 ……..
while test –n "$1" 
do

    echo "$1"

    shift

done

Explanation

• Check if the first argument is non-zero, if this test fails then exit loop.
• Print the first argument.
• Shift the arguments, second argument becomes first.
• Move to step 1

For command

For command is used to traverse over a list.
e.g.

for i in 1 2 3 4 5 6
do

    echo $i

done

The above program will print number from 1 to 6.

Until command

Until command is a compliment of while loop .i.e. it breaks out of loop only when its argument is True.

until command
do

    Commands

done

let’s create a program which will wait for given user to login and print status.

#!/bin/bash
#user_log.sh
# The usage: ./user_log.sh username
if [ -z "$1" ];then
echo "usage: $0 username"

    exit 1

fi 
Username=$1
until who | grep $Username
do

    # user is not logged in sleep for 1 seconds

    sleep 1

done
echo "$Username has loggedin"

break and continue

Break causes to exit out of loop immediately.
Continue cause the loop to jump directly at the next iteration.
Let’s create a script which will prints its argument unto the “last” argument and skip the argument they came after the “skip”

#!/bin/bash
#print.sh
# The usage: ./print.sh arg1 arg2 arg3 arg4 last……..
while [ –n "$1" ] 
do

    if [ "$1" = "last" ];then

        break

    elif [ "$1" = "skip" ];then

        shift;shift

        continue

    else

       echo $1

       shift

    fi

   echo "$1"

   shift

done

Explanation

• Check if first argument is non-zero length, exit out of loop if it’s not
• Check if its last arguments, exit out of loop if it is
• Check if it is skip, if it is then skip this and next arguments and continue to next iteration
• Else print the arguments

Switch Case

The equivalent of C Switch case in scripting is “case”. It helps in combining the multiple if else statements to write in a more elegant and efficient way. Its use is to check if the argument matches one of the given patterns and execute some commands.

case argument in

   Pattern1)

       Command1

       Command2

   ;;

   Pattern2)

       Command4

       Command5

   ;;

   *)

       Command7

       Command8

   ;;

esac

esac points the end of case statement. Pattern1 and Pattern2 can be simple strings or regular expression. Double semicolon (;;) marks the end of block of that pattern.

I/O and Redirection

Input output can be through terminal or through files. For output we have already seen the use of echo command. Echo command prints its argument on the standard output that is terminal for default and appends a line break after that. To suppress appending of new line character use –n option. Default echo does not interpret the special characters like line break but we can change its behavior by using the –e option.

Run following commands in your shell.

echo -n"Hello World\n" 
echo –en " Hello World\n"

If we need to take input from user we use read command. Read command can be very handy when combined with redirection. Try following commands.

read –p"Enter Value:" x
echo $x

Redirection is a very powerful tool that makes it easy to handle input output. For example when we are using the pipe | we are redirecting the command output to second command. Apart from pipe there are other redirections also.

 

Lets Create a script which reads /etc/passwd file and prints the shell and home directory of each user.

#!/bin/bah
# Print Shell and Home directories of user
# user-info.sh 
while read line;do

   username=`echo $line| cut-d":" –f1`

   home=`echo $line| cut-d":" –f6`

   shell=$(echo $line| cut-d":" –f7)

   echo $shell | egrep –e ‘false’

   if [ $? –ne 0 ];

       echo –e "Username\t:\t$username"

       echo –e "Home\t:\t$home"

       echo –e "Shell\t:\t$shell"

   fi

done < /etc/passwd

Explanation:

• In first line we are reading a input from user and using its return value as a test for while loop.
• As In the last line we have redirected the input of from file /etc/passwd so read command will read from this file instead of stdin.
• When it reaches at the end of file read will return exit status other than 0 and while loop will break.
• In second line we have used special constrict back tick. Its use is to run the commands given in a sub shell and returns the output of these commands.
• We can replace back tick with $(command) which is simple to use and less error prone.
• We are running the command in a sub shell and assigning the output to variables.
• The cut command is use to divide the line in to columns based on the delimiter character given by option –d.
• In fifth line we are checking if this is a standard user account which has its shell other than /bin/false.
• Then we are printing the output.

Notice here that we have use the semicolon to combine the command while and do and in the same way if and then, it is not necessary to do so, it is more of a personal choice whether to use a new line for then and do or the same.

When we run the above script using ./user-info.sh  the output is shown on the terminal. Now lets use following command

$ ./user-info.sh > filename

This will cause the output of the script user-info.sh to be redirected to file.

Here Document

Here document or heredoc is a way to specify script input in a literal string in the script itself. To use it we define a delimiter which tells the interpreter the end of the input. e.g.

echo <<END

This is an example of here document. We use to it to redirect the command’s input to some literal string.
END. Here END is delimiter; we can use anything as delimiter. One point to keep in mind is that delimiter must come in a new line and that line must not have anything beside the delimiter.

<<- delimiter syntax helps in suppressing the tabs before each lines which are used to format the code.

Arithmetic Operations

In programs we sometime need to do some arithmetic operations on our input, shell also give us tools to do it.

Let Command

Let command is a very simple command that can do some easy arithmetic operations on in arguments and returns the results. It is a shell built-in command so to get help use help command. e.g.

x=10
let x++
echo $x

The output will be 11. Here is a list of operators that can be used with the Let command. (From the help let command)

The downside of using let command is, it does not output the results, we have to use a variable to store the results. E.g to add two numbers

Let "x=1+2"
echo $x

Lets write script which will work as the seq command ( output sequence)

#!/bin/bah
# output sequence of numbers
# seq.sh <start> <end>
case $# in

    1)

        last=$1

        start=0

    ;;

    2)

        start=$1

        last=$2

    ;;

    *)

        start=0

        last=0

    ;;

esac

if [ $last- lt $start ];then
   echo "usage:$0[ <start>] <end>"
fi

while [ $last –lt $start ];do

echo $start

let start++

done

Expr command

Like Let command expr command helps for handling arithmetic operations but it’s an external command and unlike let command it directly outputs the results. It evaluates the expression given as an argument and prints the output. e.g.

$ expr "1 + 2"

will output 3

The spaces between elements of an expression are critical. The problem you will experience with is multiplication operator, as the shell uses it for wildcard substitution, for multiplication use \* instead. Both let and expr only handles fixed points integer expressions.

Bc command

Bc is a small interactive program which comes with the system utilities, we can also use it to handle our arithmetic expressions using pipe. Unlike let or expr it is capable of handling real numbers. It is very powerful tool which can handle all the arithmetic work for us. It can also be used to convert from one number system to other.
Examples:

echo "2+3" | bc                        :     Add 2 and 3
echo "7*8" | bc                        :     Multiply 7 and 8
echo "obase=16;100" | bc               :     Convert 100 to hexadecimal
echo "scale=2;13/7"  | bc              :     Floating point calculations

Try man bc to learn more about this wonderful tool.

Debugging Scripts

Like any other programming language some time shell scripting also have the debugging capability. But unlike them we do not need any other tool to do that. The same shell which is used run the scripts can be used to debug them. To debug scripts run the script with –x option. For example to debug my script test.sh I will use the command bash –x test.sh.
I can also change the very first statement of my script to include this flag.  e.g

#!/bin/bash –x

With –x option shell will print all the statements it is executing with replacing the variables with their values.

Getting Help

Shell is a very powerful tool and it takes a lot of time and effort to master it. You can not remember all the commands so it comes with various handy utilities for help.

Help Command

Help command displays help information for built in commands.
Try help help to learn more.

Man Command

Man is a tool to read and display man pages which comes with the utilities we install on our system. For external commands use man is the tool to get help. If you are new to Linux man is your friend, not only for commands but also for the system calls and other libraries. Try man man to learn more

Info command

Info command gives more detailed information about the command. It reads the info documents of that utility.
Try man info to learn more.

-h, –help Options

In Linux almost all the commands implement these command line options to display a small text of help for the command. This is command specific so not all command may have this. To use it try command name with –help option or -h option.

Apropos command

All the above commands will help only if you know the command name exactly. Some time we forgot the exact command name but but know what it does,  then apropos is the command that helps us.  Try man apropos for more info.

An example script

Before concluding lets create a script which will run in the background and monitors the disk space usage of all users and mail them if disk usage becomes more then quota.

For this we have to do following:

• Find users and their home directories reading /etc/passwd
• Find his disk usage with du command
• Compare his usage with the quota.

#!/bin/bash
# Users Disk Space Monitor
# Run every friday midnight

USER_QUOTA="10" # GB

# Traverse over all the lines of /etc/passwd
# Read one line 
while read line;do

    # Parse user info

    home_dir=$(echo $line|cut -d":" -f6)

    user=$(echo $line|cut -d":" -f1)

    shell=$(echo $line|cut -d":" -f7)

    echo $shell| grep false >/dev/null 2>&1

    # Check user is real user

    if [ $? -ne 0 ];then

        # Check user home directory exist

        if [ -n "$home_dir" -a -d "$home_dir" ];then

            # Calculate the disk usage

            DISK_USAGE=$(du -sh "$home_dir" | cut -f1)

            # Check the disk usage are in the GB

           if echo $DISK_USAGE | grep "G";then

               # Get the number and remove G

               DISK_USAGE=$(echo $DISK_USAGE | tr -d '[A-Z]')

               # Check if user has crossed his quota

               if [ $DISK_USAGE -gt $USER_QUOTA ];then

                   # mail user

                   # Using Heredoc to redirect mail input

                   # Use - construct to supress tabs

                   mail -s "Disk Usaga" $user <<-END

                   Dear $user,

                   Your disk usage($DISK_USAGE) crossed $USER_QUOTA GB limit.

                   Please delete unnecessary files"

END

               fi

            fi

        fi

    fi

done < /etc/passwd
# Reschedule the command to run at midnight each Friday
at midnight Friday <<!
$0
!
# End of Script

In this part of the series we have learned some basics of shell scripting. In part II we will learn about arrays, functions, file descriptors, networking and async I/O etc.

 

REFERENCES

Bash Guide for Beginners

Advanced Bash-Scripting Guide

Introduction to Linux Shell

Introduction to Shell Environment