Table of Contents

Bash - Bourne-Again SHell

bash, the GNU Bourne-Again SHell, is a commonly used shell a.k.a. command-line interpreter a.k.a. terminal.

Configuration

A shell can either be a login-shell or a non-login-shell. The idea behind this concept is, that the first shell instance after login (login-shell) is configured with various configuration files setting variables etc.

All other shell started from the login-shell are sub-shells (non-login-shell). Sub-shells automatically inherit the configuration of its parent, therefore there is no need for reading the configuration files again.

Configuration Files

/etc/environment         # one environment variable assignment per line; read at login only
/etc/profile             # shell script; only executed by login-shell (in theory)
/etc/bash.bashrc         # shell script; executed by each sub-shell (bash only!)

Global configuration files/scripts for all users

~/.pam_environment       # one environment variable assignment per line; read at login only 
~/.profile               # shell script; only executed by login-shell (in theory)
~/.bashrc                # shell script; executed by each sub-shell (bash only!)

Configuration for a single user, that overrides global configuration.

Limitations

No login-shell is created and therefore /etc/profile and ~/.profile are not executed when: (source)

/etc/bash.bashrc and ~/.bashrc are bash-specific and do not work with other shells like /bin/sh.

See details in the documentation for Ubuntu environment variables

Variables

There are two types of variables: - shell variables, which are only visible in the shell they were defined in - environment variables, which are passed to child processes

Here is how (un)setting variables works:

foo='bar bar'      # define a shell variable (no spaces between variable name and content)
echo $foo          # print a shell or environment variable
export foo         # make a shell variable to an environment variable
export foo=bar     # define an environment variable
export foo=${foo}:bar # append ':bar' to an environment variable
export -n foo      # remove the environment variable foo, but keep the shell variable
unset foo          # remove foo

Bash features a number of predefined variables. See the full list in the section “special variables” in man bash. Here is a small selection:

echo $0            # name of the shell or shell script.
echo $?            # print exit status of the most recently executed foreground pipeline.
echo $!            # print process ID of the most recently executed background  (asynchronous) command.

To list currently defined variables:

export             # list current environment variables (= export -p)
env                # does the same (but unsorted)

To set paths for the future (for your user) you can put them into e.g. ~/.bashrc:

export JBOSS_HOME=/opt/jboss
export PATH=${PATH}:/opt/bin

The shell builtin command 'source' can then be used to update the environment of the current shell by executing the changed file:

source ~/.bashrc

Shell Options

Shell options can be set with the built-in set command. They are boolean flags and can be activated with - and deactivated with + (yes, this way round!) as follows:

set -o verbose      # (or set -v) activate verbose mode
set +o verbose      # (or set +v) deactivate verbose mode

The verbose mode prints the input command again before executing it. Other useful options are noclobber (avoid overwriting of regular files with stream redirection) or noglob (disable globbing - the expansion of the wildcards ? and *).

Currently set options are stored in the variable $-.

Usage

Running Programs

To run/execute a program or script that lies in one of the directories of $PATH simply type its name and press enter.

ls

To execute other scripts or directories supply the whole path or prepend ./ if it resides in the current working dirctory (pwd).

/path/to/executable
./executable_in_current_directory

The bash builtin exec will replace itself (the shell) with the command to execute. The program will have the same process id (pid) and you do not get the shell back after the command finishes. This can be useful for wrapper scripts around programs or more advanced stuff.

echo "This is a wrapper script"
# do some vodoo here, probably change the arguments etc.
exec "myprogram" "$@"

The bash builtins source and . execute all lines of a script in the current shell. The script is not required to be executable!

source /path/to/myscript.sh
. /path/to/myscript.sh

With the program env the set of environment variables when running a program can be manipulated (but not changed globally).

env                                    # print environment variables
env -i env                             # start env with a cleaned / empty environment
env -u VAR1 -u VAR2 VAR3=X VAR4=Y env  # unset VAR1+2, set VAR3+4 

Shell Expansion

Have a look at the manual for shell expansions like the tilde character (~) or brace expansion.

History

The command history for each user is stored in ~/.bash_history. It can be viewed with history.

Live-search is activated by typing CTRL-r and then a part of the command. Typing CTRL-r cycles through matching commands, ENTER executes it.

These shortcuts can be used to execute recent commands:

!!              #run last command executed in bash (= !-1)
!-2             #run command before last command
!120            #run command with nr 120 as shown by ''history''
!commandname    #run last command starting with a certain commandname

Exit Code

When a program or script exits it returns an integer, which is called “exit status”, “exit code”, “return status”,.. By convention 0 stands for success and everything else for some kind of error.

The exit code is stored in a special variable and can be printed like this:

echo $?

Escaping

Whitespace and some additional characters are treated special by the shell. If they are not destined for the shell, they must be 'escaped' properly.

$&;(){}[]*?!<>"'

Escaping works by either preceding characters with a backslash or by putting strings in quotes:

foo=uname
echo '$foo'     # prints "$foo"     ' completely preserves the string
echo "$foo"     # prints "uname"    " preserves string except, dollar ($), backticks (`), backslash (\).
echo `$foo`     # prints "Linux"    ` executes string as a command (this is called command substitution)
echo $($foo)    # prints "Linux"    another way to use command substitution

Command Chaining

The Unix philosophy is: “Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.”

Command chaining allows us to use these small tools to build bigger ones by executing them in a predefined order or letting execution depend on the success of previously executed commands.

The following chaining parameters should be used like e.g. cmd1 | cmd2

|           pipe: STDOUT of cmd1 goes into STDIN of cmd2
;           cmd2 is executed after cmd1
&&          cmd2 is executed after cmd1 was successful (return code = 0)
||          cmd2 is only executed, if cmd1 was not successful
&           cmd2 is executed after cmd1 was started in background

()          group commands to override operator precedence

Stream Redirection

Many programs support reading input from STDIN and writing output to STDOUT or STDERR. This is how the flow of streams can be controlled with redirection.

In the example we use find because it writes to STDOUT (names of all readable files) as well as STDERR (unaccessible files).

1> or >     redirect STDOUT     find /etc/apt/ /root > /dev/null
2>          redirect STDERR     find /etc/apt/ /root 2> /dev/null
&>          redirect both       find /etc/apt/ /root &> /dev/null

Redirecting with a single greater than sign > causes the target file to be overwritten. By replacing it with a double greater than sign » in all examples above new content is appended to the target.

1>> or >>   redirect STDOUT     find /etc/apt/ /root >> /var/log/my.log

Output can also be redirected into another stream instead of directly to a file:

2>&1        redirect STDERR into same stream as STDOUT      find /etc/apt/ /root > /dev/null 2>&1
1>&2        redirect STDOUT into same stream as STDERR      find /etc/apt/ /root 2> /dev/null 1>&2

Input redirection:

<           STDIN not from keyboard but from file           sort < file.txt
<< EOT      STDIN from keyboard, but instead of CTRL-d the character sequence EOT in one line ends input (this is called: HERE document)

To avoid accidential overwriting of files with >, the noclobber-option can be set in bash. This has the same effect as the append-only file attribute (see chattr), except it is valid for every file:

set +o noclobber

Also remember the program tee, which acts as a T-junction: STDIN goes to STDOUT and into a file.

Aliases

Aliases are used to define shortcuts to commands invocations - including their parameters - that are used often. They are typically defined in a script like ~/.bashrc

alias ll='ls -alF'
alias o='xdg-open'

Call alias without parameters to see which aliases are currently defined.

Scripting

Variables

Assignment

Variable assignment uses the = character. There must be no spaces inbetween.

x=10        # assign with an equal sign without spaces
x="a b c"   # strings with spaces must be quoted
x=$(ls -l)  # (or x=`ls -l`)command substitution - variable stores result of command execution
x=          # set variable to null
unset x     # unset variable

Variables in bash are global except when specifically defined as local. Local variables shadow global ones.

globalVariable=5
local localVariable=6

Using Variables

Prefixing a variable name with $ dereferences it. To clarify where the variable name ends curly braces {} can be used. Quoting the use of variables preserves whitespace.

x="tasty   ice"
echo $x          # prints "tasty ice"
echo ${x}cream   # prints "tasty icecream"
echo "${x}cream" # prints "tasty   icecream"

Evaluate Expressions

test and [

test or [ is a program used to e.g. evaluate comparisons typically required for if statements.

It returns either true (return code 0) or false (return code 1).

test 10 -eq 10     # long form
[ 10 -eq 10 ]      # short form (spaces to brackets are mandatory)

Integer comparisons:

  1. -eq, -ge, -le: equal, greater equal, less equal
  2. -ne, -gt, -lt: not equal, greater than, less than

String comparisons:

  1. =, !=
  2. -z: empty string (length is zero)
  3. -n: non-empty string

File tests (evaluated for the calling user):

  1. -r, -w, -x: read, write and executable permissions
  2. -e, -s: file exists, file exists and is greater than zero
  3. -nt, -ot: newer / older than
  4. -f, -d, -L/-h: file exists and is a regular file, directory or link
  5. -c, -b, -p, -S: file exists and is a character device, block device, pipe, socket

Note, that most tests will return true for empty input, which may not be the expected result:

x=                                                               # x is an empty string
[ -e $x ] && echo file $x exists || echo file $x does not exist  # output: 'file exists'

Expression can be grouped with parentheses () (which must be escaped!), negated with ! and combined with-o (or) and -a (and)

x=5
[ $x -gt 2 -a ! $x -eq 10 ]; echo $?
[ $x -gt 2 -a \( ! $x -eq 10 \) ]; echo $?   # with grouping
Other Comparison and Arithmetic Functions

Apart from test ([) there is an extended test command [[ available in bash. Read more about it here and here.

Integer Arithmetics

bc

bc is an arbitrary precision calculator language.

echo "5*4 + 1" | bc
bc <<< "5*4 + 1"
let

let is a bash-built-in that can do integer arithmetics. It can evaluate several expressions at once (from left to right). When using double quotes spaces can be used within the expression(s). Other variables are referenced without the dollar-sign.

let x=5*4 x=x/=2; echo $x         # several calculations resulting in 10
a=10
let x="15 % (a + 2)"; echo $x    # prints 3
Arithmetic Expansion

Another possibility to do integer arithmetic is the use of $((. Within the double parentheses spaces and additional parentheses to group expressions can be used freely. Other variables are referenced without the dollar-sign. See here, here, and here.

a=10
x=$(( 15 % (a + 2) )); echo $x   # prints 3

Flow Control

If else

Multiple line example

if [ -f $file ]
then
    echo $file "is a regular file"
elif [ -d $file  ]
then
    echo $file "is a directory"
else
    echo $file "is neither a file nor a directory"
fi

One-liner:

if [ -f $file ]; then echo $file "is a regular file"; elif [ -d $file  ]; then echo $file "is a directory"; else echo $file "is neither a file nor a directory"; fi
While / Until

A while loop runs as long as the exit status of the command in the loop head returns 0.

An until loop runs as long as the exit status of the command in the loop head returns 1.

Multiple line example

i=1
while [[ $i < 6 ]]; do
    echo $((i++))
done
 
i=1
until [[ $i > 5 ]]; do
    echo $((i++))
done

One-liner

i=1
while [[ $i < 6 ]]; do echo $((i++)); done
 
i=1
until [[ $i > 5 ]]; do echo $((i++)); done
For

Multiple line example

for i in $(seq 1 5); do
    echo $i
done
for i in var1 var2 var3; do
    echo $i
done

One-liner

for i in $(seq 1 5); do echo $i; done
Case

After the first match, case terminates with the exit status of the last command that was executed. So there is no need for breaks after each switch.

i=5
case $i in
     5)
     echo five
     ;;
     banana)
     echo monkey
     ;;
     *)
     echo "something else - $i to be precise"
     ;;
esac
Break & Continue

break and continue can be used to break out of a loop / case or to continue at the next iteration of a loop.

Exit

To stop the script immediately and decleare a return code (exit value) use exit.

exit 0   # exit with success
exit 1   # exit with failure (any code except 0)

Functions

Bash supports basic functions with arguments and an exit status (integer only). The function definition must precede the first call to it (but using a not yet defined function within another function is OK - only when calling the function both must be defined).

unset -f will get rid of a defined function.

function myfunc() {
    echo "hello"
}
 
function myfuncArgs() {
    echo $1 $2
    return 0              # exit status of the function
}
 
myfunc
myfuncArgs a b c # will print a b, because $3 is not used.
 
unset -f myfunc

Command sequences

Parentheses ( () ) execute commands in a subshell. The current shell environment is therefore not influenced by the executed commands, e.g. a directory change

(cd /tmp; touch file)

Braces ( {} ) execute commands in the current shell context. The braces must be separated with a space and a semicolon before the closing brace is required.

{ cd /tmp; touch file; }

Check out this nice overview of what brackets, braces and parentheses can mean in bash.

Scripts

Scripts should be executable files and start with a sha-bang (#!) in the first line. The sha-bang defines which program should execute the script, e.g.

#!/bin/bash                  # using absolute location of program
#!/usr/bin/env bash          # using platform-independent way to find program

This simple script prints '1234' and uses local/global variables and a function:

#!/bin/bash
 
#global variable
x=1
echo -n $x 
 
#simple function
function myFunction() {
     #local variable
     local y=2
     echo -n $y
 
     #changing the global variable
     x=3
 
     #creating another global variable
     z=4
}
 
#invocing function
myFunction
 
echo -n $x
echo -n $z

More resources: Advanced Bash-Scripting Guide Bash Guide

Debugging

With sleep a script can be paused for a certain amount of time:

sleep 10           # sleep for 10 seconds
sleep 1h 10m 59s   # sleep for 1 hour, 10 minutes and 59 seconds

time can measure the execution time of a program and is available as bash built-in or program (that can display more information) in /usr/bin/time.

time program              # print summary of execution time (bash buit-in)
/usr/bin/time -v program  # print more information with program time

Trace Mode

To trace scripts use the bash option 'x', which prints commands and their arguments as they are executed.

set -x # enable trace-mode
var=1
echo $var
set +x # disable trace-mode