Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Mastering Bash

You're reading from   Mastering Bash A Step-by-Step Guide to working with Bash Programming and Shell Scripting

Arrow left icon
Product type Paperback
Published in Jun 2017
Publisher Packt
ISBN-13 9781784396879
Length 502 pages
Edition 1st Edition
Languages
Tools
Arrow right icon
Author (1):
Arrow left icon
Giorgio Zarrelli Giorgio Zarrelli
Author Profile Icon Giorgio Zarrelli
Giorgio Zarrelli
Arrow right icon
View More author details
Toc

Table of Contents (15) Chapters Close

Preface 1. Let's Start Programming FREE CHAPTER 2. Operators 3. Testing 4. Quoting and Escaping 5. Menus, Arrays, and Functions 6. Iterations 7. Plug into the Real World 8. We Want to Chat 9. Subshells, Signals, and Job Controls 10. Lets Make a Process Chat 11. Living as a Daemon 12. Remote Connections over SSH 13. Its Time for a Timer 14. Time for Safety

Variables

What is a variable? We could answer that it is something not constant; nice joke, but it would not help us so much. Better to think of it as a bucket where we can store some information for later processing: at a certain point of your script you get a value, a piece of info that you do not want to process at that very moment, so you fit it into a variable that you will recall later in the script. This is, in an intuitive way, the use of a variable, a way to allocate a part of the system memory to hold your data.

So far, we have seen that our scripts could retrieve some pieces of information from the system and had to process them straight away, since, without the use of a variable, we had no way to further process the information except for concatenating or redirecting the output to another program. This forced us to have a linear execution, no flexibility, no complexity: once you get some data, you process it straight away redirecting the file descriptors, one link in the chain after the other.

A variable is nothing really new; a lot of programming languages use them to store different types of data, integers, floating, strings, and you can see many different kinds of variables related to different kinds of data they hold. So, you have probably heard about casting a variable, which means, roughly, changing its type: you get a value as a string of numbers and you want to use it as an integer, so you cast it as an int and proceed processing it using some math functions.

Our shell is not so sophisticated, and it has only one type of variable or, better, it has none: whatever you store in it can be later processed without any casting. This can be nice because you do not have to pay attention to what type of data you are holding; you get a number as a string and can process it straight away as an integer. Nice and easy, but we must remember that restrictions are in place not just to prevent us from doing something, but also to help us not do something that would be unhealthy for our code, and this is exactly the risk in having flat variables, to write some piece of code that simply does not work, cannot work.

Assigning a variable

As we just saw, a variable is a way to store a value: we get a value, assign it to a variable and refer to the latter to access the former. The operation of retrieving the content of a variable is named variable substitution. A bit like, if you think about descriptors, the way that you use them to access files. The way you assign a variable is quite straightforward:

LABEL=value  

LABEL can be any string, can have upper and lowercase, start with or contain numbers and underscores, and it is case sensitive.

The assignment is performed by the = character, which, be wary, is not the same as the equal to == sign; they are two different things and are used in different contexts. Finally, whatever you put at the right of the assignment operator becomes the value of the variable. So, let's assign some value to our first variable:

gzarrelli:~$ FIRST_VARIABLE=amazing  

Now we can try to access the value trying to perform an action on the variable itself:

gzarrelli:~$ echo FIRST_VARIABLE
FIRST_VARIABLE

Not exactly what we expected. We want the content, not the name of the variable. Have a look at this:

gzarrelli:~$ echo $FIRST_VARIABLE
amazing

This is better. Using the $ character at the beginning of the variable name identified this as a variable and not a plain string, so we had access to the content. This means that, from now on, we can just use the variable with any commands instead of referring to the whole content of it. So, let us try again:

gzarrelli:~$ echo $first_variable    
gzarrelli:~$

The output is null, and not 0; we will see later on that zero is not the same as null, since null is no value but zero is indeed a value, an integer. What does the previous output mean? Simply that our labels are case sensitive, change one character from upper to lower or vice versa, and you will have a new variable which, since you did not assign any value to it, does not hold any value, hence the null you receive once you try to access it.

Keep the variable name safe

We just saw that $label is the way we reference the content of a variable, but if you have a look at some scripts, you can find another way of retrieving variable content:

${label}  

The two ways of referencing the content of a variable are both valid, and you can use the first, more compact, in any case except when concatenating the variable name to any characters, which could change the variable name itself. In this case, it becomes mandatory to use the extended version of the variable substitution, as the following example will make clear.

Let's start printing our variable again:

gzarrelli:~$ echo $FIRST_VARIABLE
amazing

Now, let's do it again using the extended version of substitution:

gzarrelli:~$ echo ${FIRST_VARIABLE}
amazing

Exactly the same output since, as we said, these two methods are equivalent. Now, let us add a string to our variable name:

gzarrelli:~$ echo $FIRST_VARIABLEngly    
gzarrelli:~$

Nothing, and we can understand why the name of the variable changed; so we have no content to access to. But now, let us try the extended way:

gzarrelli:~$ echo ${FIRST_VARIABLE}ly
amazingly

Bingo! The name of the variable has been preserved so that the shell was able to reference its value and then concatenated it to the ly string we added to the name.

Keep this difference in mind, because the graphs will be a handy way to concatenate strings to a variable to spice your scripts up and, as a good rule of thumb, refer to variables using the graphs. This will help you avoid unwanted hindrances.

Variables with limited scope

As we said before, variables have no type in shell, and this makes them somehow easy to use, but we must pay attention to some sorts of limits to their use.

  • First, the content of a variable is accessible only after the value has been assigned
  • An example will make everything clearer:
gzarrelli:~$ cat disk-space.sh 
#!/bin/bash
echo -e "\n"
echo "The space left is ${disk_space}"
disk_space=`df -h | grep /$ | awk '{print $4}'`
echo "The space left is ${disk_space}

We used the variable disk space to store the result of the df command and try to reference its value on the preceding and following lines. Let us run it in debug mode:

gzarrelli:~$ sh -x disk-space.sh 
+ echo -e \n
-e
+ echo The space left is
The space left is
+ awk {print $4}
+ grep /dm-0
+ df -h
+ disk_space=3.0G
+ echo The space left is 3.0G
The space left is 3.0G

As we can see, the flow of execution is sequential: you access the value of the variable only after it is instanced, not before. And bear in mind that the first line actually printed something: a null value. Well, now let us print the variable on the command line:

gzarrelli:~$ echo ${disk_space}    
gzarrelli:~$

The variable is instanced inside the script, and it is confined there, inside the shell spawned to invoke the command and nothing passed to our main shell.

We can ourselves impose some restrictions to a variable, as we will see with the next example. In this new case, we will introduce the use of a function, something that we are going to look at in more detail further in this book and the keyword local:

gzarrelli:~$ cat disk-space-function.sh
#!/bin/bash
echo -e "\n"
echo "The space left is ${disk_space}"
disk_space=`df -h | grep /dm-0 | awk '{print $4}'`
print () {
echo "The space left inside the function is ${disk_space}"
local available=yes
last=yes
echo "Is the available variable available inside the function? ${available}"
}
echo "Is the last variable available outside the function before it is invoked? ${last}"
print
echo "The space left outside is ${disk_space}"
echo "Is the available variable available outside the function? ${available}"
echo "Is the last variable available outside the function after it is invoked? ${last}"

Now let us run it:

gzarrelli:~$ cat di./pace-function.sh
The space left is
Is the last variable available outside the function before it is invoked?
The space left inside the function is 3.0G
Is the available variable available inside the function? yes
The space left outside is 3.0G
Is the available variable available outside the function?
Is the last variable available outside the function after it is invoked? yes

What can we see here?

The content of variable disk_space is not available before the variable itself is instanced. We already knew this.

The content of a variable instanced inside a function is not available when it is defined in the function, but when the function itself is invoked.

A variable marked by the keyword local and defined inside a function is available only inside the function and only when the function is invoked. Outside the block of code defined by the function itself; the local variable is not visible to the rest of the script. So, using local variables can be handy to write recursive code, even though not recommended.

So, we just saw a few ways to make a variable really limited in its scope, and we also noted that its content is not available outside the script it was instanced in. Wouldn't it be nice to have some variables with a broader scope, capable of influencing the execution of each and every script, something at environment level? It would, and from now on we are going to explore the environment variables.

Environment variables

As we discussed earlier, the shell comes with an environment, which dictates what it can do and what not, so let's just have a look at what these variables are using the env command:

zarrelli:~$ env    
...
LANG=en_GB.utf8
...
DISPLAY=:0.0
...
USER=zarrelli
...
DESKTOP_SESSION=xfce
...
PWD=/home/zarrelli/Documents
...
HOME=/home/zarrelli
...
SHELL=/bin/bash
...
LANGUAGE=en_GB:en
...
GDMSESSION=xfce
...
LOGNAME=zarrelli
...
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
_=/usr/bin/env

Some of the variables have been omitted for the sake of clarity; otherwise, the output would have been too long, but still we can see something interesting. We can have a look at the PATH variable content, which influences where the shell will look for a program or script to execute. We can see which shell is being currently used, by which user, what the current directory is and the previous one.

But environment variables can not only be read; they can be instanced using the export command:

zarrelli:~$ export TEST_VAR=awesome  

Now, let us read it:

zarrelli:~/$ echo ${TEST_VAR}
awesome

That is it, but since this was just a test, it is better to unset the variable so that we do not leave unwanted values around the shell environment:

zarrelli:~$ unset TEST_VAR  

And now, let us try to get the content of the variable:

zarrelli:~/$ echo ${TEST_VAR}
zarrelli:~/$

No way! The variable content is no more, and as you will see now, the environment variables disappear once their shell is no more. Let's have a look at the following script:

zarrelli:~$ cat setting.sh 
#!/bin/bash
export MYTEST=NOWAY
env | grep MYTEST
echo ${MYTEST}

We simply instance a new variable, grep for it in the environment and then print its content to the stdout. What happens once invoked?

zarrelli@:~$ ./setting.sh ; echo ${MYTEST}
MYTEST=NOWAY
NOWAY
zarrelli:~$

We can easily see that the variable was grepped on the env output, so this means that the variable is actually instanced at the environment level and we could access its content and print it. But then we executed the echo of the content of MYTEST outside the script again, and we could just print a blank line. If you remember, when we execute a script, the shell forks a new shell and passes to it its full environment, thus the command inside the program shell can manipulate the environment. But then, once the program is terminated, the related shell is terminated, and its environment variables are lost; the child shell inherits the environment from the parent, the parent does not inherit the environment from the child.

Now, let us go back to our shell, and let us see how we can manipulate the environment to our advantage. If you remember, when the shell has to invoke a program or a script, it looks inside the content of the PATH environment variable to see if it can find it in one of the paths listed. If it is not there, the executable or the script cannot be invoked just with their names, they have to be called passing the full path to it. But have a look at what this script is capable of doing:

#!/bin/bash    
echo "We are into the directory"
pwd

We print our current user directory:

echo "What is our PATH?"
echo ${PATH}

And now we print the content of the environment PATH variable:

echo "Now we expand the path for all the shell"
export PATH=${PATH}:~/tmp

This is a little tricky. Using the graphs, we preserve the content of the variable and add a, which is the delimiter for each path inside the list held by PATH, plus the ~/tmp, which literally means the tmp directory inside the home directory of the current user:

echo "And now our PATH is..."
echo ${PATH}
echo "We are looking for the setting.sh script!"
which setting.sh
echo "Found it!"

And we actually found it. Well, you could also add some evaluation to make the echo conditional, but we will see such a thing later on. Time for something funny:

echo "Time for magic!"
echo "We are looking for the setting.sh script!"
env PATH=/usr/bin which setting.sh
echo "BOOOO, nothing!"

Pay attention to the line starting with env; this command is able to overrun the PATH environment variable and to pass its own variable and related value. The same behavior can be obtained using export instead of env:

echo "Second try..."
env PATH=/usr/sbin which setting.sh
echo "No way..."

This last try is even worse. We modified the content of the $PATH variable which now points to a directory where we cannot find the script. So, not being in the $PATH, the script cannot be invoked by just its name:

zarrelli:~$ ./setenv.sh   

We are in the directory:

/home/zarrelli/Documents/My books/Mastering bash/Chapter 1/Scripts  

What is our PATH?

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games  

Now we expand the path for all the shell.

And now our PATH is:

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home
/zarrelli/tmp

We are looking for the setting.sh script!

/home/zarrelli/tmp/setting.sh  

Found it!

Time for magic!

We are looking for the setting.sh script!

BOOOO, nothing!

Second try...

env: 'which': No such file or directory

No way...

Environment variable

Use

BASH_VERSION

The version of the current Bash session

HOME

The home directory of the current user

HOSTNAME

The name of the host

LANG

The locale used to manage the data

PATH

The search path for the shell

PS1

The prompt configuration

PWD

The path to the current directory

USER

The name of the currently logged in user

LOGNAME

Same as user

We can also use env with the -i argument to strip down all the environment variables and just pass to the process what we want, as we can see in the following examples. Let's start with something easy:

zarrelli:~$ cat env-test.sh 
#!/bin/bash
env PATH=HELLO /usr/bin/env | grep -A1 -B1 ^PATH

Nothing too difficult, we modified the PATH variable passing a useless value because HELLO is not a searchable path, then we had to invoke env using the full path because PATH became useless. Finally, we piped everything to the input of grep, which will select all the rows (^) starting with the string PATH, printing that line and one line before and after:

zarrelli:~$ ./env-test.sh     
2705-XDG_CONFIG_DIRS=/etc/xdg
2730:PATH=HELLO
2741-SESSION_MANAGER=local/moveaway:@/tmp/.ICE-unix/888,unix/moveaway:/tmp/.ICE-unix/888

Now, let's modify the script, adding -i to the first env:

zarrelli:~$ cat env-test.sh 
#!/bin/bash
env -i PATH=HELLO /usr/bin/env | grep -A1 -B1 ^PATH

And now let us run it:

zarrelli:~/$ ./env-test.sh 
PATH=HELLO
zarrelli:~/$

Can you guess what happened? Another change will make everything clearer:

env -i PATH=HELLO /usr/bin/env   

No grep; we are able to see the complete output of the second env command:

zarrelli:~$ env -i PATH=HELLO /usr/bin/env
PATH=HELLO
zarrelli:~$

Just PATH=HELLO env with the argument -i passed to the second env process, a stripped down environment with only the variables specified on the command line:

zarrelli:~$ env -i PATH=HELLO LOGNAME=whoami/usr/bin/env
PATH=HELLO
LOGNAME=whoami/usr/bin/env
zarrelli:~$

Because we are engaged in stripping down, let us see how we can make a function disappear with the well-known unset -f command:

#!/bin/bash    
echo -e "\n"
echo "The space left is ${disk_space}"
disk_space=`df -h | grep vg-root | awk '{print $4}'`
print () {
echo "The space left inside the function is ${disk_space}"
local available=yes
last=yes
echo "Is the available variable available inside the function? ${available}"
}
echo "Is the last variable available outside the function before it is invoked? ${last}"
print
echo "The space left outside is ${disk_space}"
echo "Is the available variable available outside the function? ${available}"
echo "Is the last variable available outside the function after it is invoked? ${last}"
echo "What happens if we unset a variable, like last?"
unset last
echo "Has last a referrable value ${last}"
echo "And what happens if I try to unset a while print functions using unset -f"
t
print
unset -f print
echo "Unset done, now let us invoke the function"
print

Time to verify what happens with the unset command:

zarrelli:~$ ./disk-space-function-unavailable.sh   

The space left is:

Is the last variable available outside the function before it is invoked? 
The space left inside the function is 202G
Is the available variable available inside the function? yes
The space left outside is 202G
Is the available variable available outside the function?
Is the last variable available outside the function after it is invoked? yes
What happens if we unset a variable, like last?
Has last a referrable value
And what happens if I try to unset a while print functions using
unset -f

The space left inside the function is 202G
Is the available variable available inside the function? yes

Unset done, now let us invoke the function:

zarrelli:~$   

The print function works well, as expected before we unset it, and also the variable content becomes no longer available. Speaking about variables, we can actually unset some of them on the same row using the following:

unset -v variable1 variable2 variablen  

We saw how to modify an environment variable, but what if we want to make it read-only so to protect its content from an unwanted modification?

zarrelli:~$ cat readonly.sh 
#!/bin/bash
echo "What is our PATH?"
echo ${PATH}
echo "Now we make it readonly"
readonly PATH
echo "Now we expand the path for all the shell"
export PATH=${PATH}:~/tmp

Look at the line readonlyPATH, and now let's see what the execution of this script leads us to:

zarrelli:~$ ./readonly.sh 
What is our PATH?
/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
Now we make it readonly
Now we expand the path for all the shell
./readonly.sh: line 10: PATH: readonly variable
zarrelli:~$

What happened is that our script tried to modify the PATH variable that was just made readonly a few lines before and failed. This failure then led us out of the screen with a failure, and this is confirmed by printing the value of the $? variable, which holds the exit state of the last command invoked:

zarrelli:~$ echo $?
1
zarrelli:~$ echo $?
0

We will see the use of such a kind of variable later, but now what interests us is to know what that 0 and 1 mean: the first time we issued the echo command, right after invoking the script, it gave us the exit code 1, which means failure, and this makes sense because the script exited abruptly with an error. The second time we ran echo, it showed 0, which means that the last command executed, the previous echo went well, without any errors.

Variable expansion

The variable expansion is the method we have to access and actually change the content of a variable or parameter. The simplest way to access or reference the variable value is as in the following example:

x=1 ; echo $x    
zarrelli:~$ x=1 ; echo $x
1

So, we assigned a value to the variable x and then referenced the value preceding the variable name with the dollar sign $. So, echo$x prints the content of x, 1, to the standard output. But we can do something even more subtle:

zarrelli:~$ x=1 ; y=$x; echo "x is $x" ; echo "y is $y"
x is 1
y is 1

So, we gave a value to the variable x, then we instanced the variable y referencing the content of the variable x. So, y got its assignment referencing the value of x through the $ character, not directly using a number after the = char. So far, we saw two different ways to reference a variable:

$x
${x}

The first one is terser, but it would be better to stick to the second way because it preserves the name of the variable and, as we saw a few pages before, it allows us to concatenate a string to the variable without losing the possibility of referencing it.

We just saw the simplest among different ways to manipulate the value held by a variable. What we are going to see now is how to thinker with a variable to have default values and messages, so we make the interaction with the variable more flexible. Before proceeding, just bear in mind that we can use two notations for our next example and they are equivalent:

${variable-default}
${variable:-default}

So, you could see either of the two in a script, and both are correct:

${variable:-default} ${variable-default}  

Simply, if a variable is not set, return a default value, as we can see in the following example:

#!/bin/bash
echo "Setting the variable x"
x=10
echo "Printing the value of x using a default fallback value"
echo "${x:-20}"
echo "Unsetting x"
unset -v x
echo "Printing the value of x using a default fallback value"
echo "${x:-20}"
echo "Setting the value of x to null"
x=
echo "Printing the value of x with x to null"
echo "${x:-30}

Now, let's execute it:

zarrelli:~$ ./variables.sh 
Setting the variable x
Printing the value of x using a default fallback value
10
Unsetting x
Printing the value of x using a default fallback value
20
Setting the value of x to null
Printing the value of x with x to null
30

As mentioned before, the two notations, with or without the colon, are quite the same. Let us see what happens if in the previous script we substitute ${x:-somenumber} with ${x-somenumber}.

Let's run the modified script:

Setting the variable x
Printing the value of x using a default fallback value
10
Unsetting x
Printing the value of x using a default fallback value
20
Setting the value of x to null
Printing the value of x with x to null
zarrelli:$

Everything is fine, but the last line. So what is the difference at play here? Simple:

  • *${x-30}: The notation with a colon forces a check on the existence of a value for the variable and this value may well be null. In case you have a value, it does print the value of the variable, ignoring the fallback.
    • unset -f x: It unsets the variable, so it has no value and we have a fallback value
    • x=: It gives a null to x; so the fallback does not come in to play, and we get back the variable value, for example, null
  • ${x:-30}: This forces a fallback value in case the value of a variable is null or nonexistent
    • unset -f x: It unsets the variable, so it has no value and we have a fallback value
    • x=: It gives a null to x, but the fallback comes in to play and we get a default value

Default values can be handy if you are writing a script which expects an input or the customer: if the customer does not provide a value, we can use a fallback default value and have our variable instanced with something meaningful:

#!/bin/bash        
echo "Hello user, please give me a number: "
read user_input
echo "The number is: ${user_input:-99}"

We ask the user for an input. If he gives us a value, we print it; otherwise, we fallback the value of the variable to 99 and print it:

zarrelli:~$ ./userinput.sh 
Hello user, please give me a number:
10
The number is: 10
zarrelli:~/$
zarrelli$ ./userinput.sh
Hello user, please give me a number:
The number is: 99
zarrelli:~/$
${variable:=default} ${variable=default}

If the variable has a value, it is returned; otherwise, the variable has a default value assigned. In the previous case, we got back a value if the variable had no value; or null, here the variable is actually assigned a value. Better to see an example:

#!/bin/bash    
#!/bin/bash
echo "Setting the variable x"
x=10
echo "Printing the value of x"
echo ${x}
echo "Unsetting x"
unset -v x
echo "Printing the value of x using a default fallback value"
echo "${x:-20}"
echo "Printing the value of x"
echo ${x}
echo "Setting the variable x with assignement"
echo "${x:=30}"
echo "Printing the value of x again"
echo ${x}

We set a variable and then print its value. Then, we unset it and print its value, but because it is unset, we get back a default value. So we try to print the value of x, but since the number we got in the preceding operation was not obtained by an assignment, x is still unset. Finally, we use echo "${x:=30}" and get the value 30 assigned to the variable x, and indeed, when we print the value of the variable, we get something. Let us see the script in action:

Setting the variable x
Printing the value of x
10
Unsetting x
Printing the value of x using a default fallback value
20
Printing the value of x
Setting the variable x with assignement
30
Printing the value of x again
30

Notice the blank line in the middle of the output: we just got a value from the preceding operation, not a real variable assignment:

${variable:+default} ${variable+default}  

Force a check on the existence of a non null value for a variable. If it exists, it returns the default value; otherwise it returns null:

#!/bin/bash    
#!/bin/bash
echo "Setting the variable x"
x=10
echo "Printing the value of x"
echo ${x}
echo "Printing the value of x with a default value on
assigned value"

echo "${x:+100}"
echo "Printing the value of x after default"
echo ${x}
echo "Unsetting x"
unset -v x
echo "Printing the value of x using a default fallback value"
echo "${x:+20}"
echo "Printing the value of x"
echo ${x}
echo "Setting the variable x with assignement"
echo "${x:+30}"
echo "Printing the value of x again"
echo ${x}

Now, let us run it and check, as follows:

Setting the variable x
Printing the value of x
10
Printing the value of x with a default value on assigned value
100
Printing the value of x after default
10
Unsetting x
Printing the value of x using a default fallback value
Printing the value of x
Setting the variable x with assignement
Printing the value of x again
zarrelli:~$

As you can see, when the variable is correctly instanced, instead of returning its value, it returns a default 100 and this is double-checked in the following rows where we print the value of x and it is still 10: the 100 we saw was not a value assignment but just a default returned instead of the real value:

${variable:?message} ${variable?message}
#!/bin/bash
x=10
y=
unset -v z
echo ${x:?"Should work"}
echo ${y:?"No way"}
echo ${y:?"Well"}

The results are quite straightforward:

zarrelli:~$ ./set-message.sh 
10
./set-message.sh: line 8: y: No way

As we tried to access a void variable, but for the unset would have been the same, the script exited with an error and the message we got from the variable expansion. All good with the first line, x has a value and we printed it but, as you can see, we cannot arrive to the third line, which remains unparsed: the script exited abruptly with a default message printed.

Nice stuff, isn't it? Well, there is a lot more, we have to look at the pattern matching against variables.

Pattern matching against variables

We have a few ways to fiddle with variables, and some of these have a really interesting use in scripts, as we will see later on in this book. Let's briefly recap what we can do with variables and how to do it, but remember we are dealing with values that are returned, not assigned back to the variable:

${#variable)  

It gives us the length of the variable, or if it is an array, the length of the first element of an array. Here is an example:

zarrelli:~$ my_variable=thisisaverylongvalue
zarrelli:~$ echo ${#my_variable}
20

And indeed thisisaverylongvalue is made up of 20 characters. Now, let us see an example with arrays:

zarrelli:~$ fruit=(apple pear banana)  

Here, we instantiated an array with three elements apple, pear, and banana. We will see later in this book how to work with arrays in detail:

zarrelli@moveaway:~$ echo ${fruit[2]}
banana

We printed the third element of the array. Arrays start with an index of 0, so the third element is at index 2, and it is banana, a 6 characters long word:

zarrelli@moveaway:~$ echo ${fruit[1]}
pear

We print the second element of the array: pear,a 4 characters long word:

zarrelli@moveaway:~$ echo ${fruit[0]}
apple

And now, the first element, that is, apple is 5 characters long. Now, if the example we saw is true, the following command should return 5.

zarrelli:~$ echo ${#fruit}
5

And indeed, the length of the word apple is 5 characters:

${variable#pattern) 

If you need to tear out your variable, for a part of it you can use a pattern and remove the shortest occurrence of the pattern from the beginning of the variable and return the resulting value. It is not a variable assignment, not so easy to grasp, but an example will make it clear:

zarrelli:~$ shortest=1010201010
zarrelli:~$ echo ${shortest#10}
10201010
zarrelli:~$ echo ${shortest}
1010201010
${variable##pattern)

This form is like the preceding one but with a slight difference, the pattern is used to remove its largest occurrence in the variable:

zarrelli:~$ my_variable=10102010103  

We instanced the variable with a series of recurring digits:

zarrelli:~$ echo ${my_variable#1*1}
02010103

Then, we tried to match a pattern, which means any digit between a leading and ending 1, the shortest occurrence. So it took out 10102010103:

zarrelli:~$ echo ${my_variablet##1*1}
03

Now, we cut away the widest occurrence of the pattern, and so 10102010103, resulting in a meager 03 as the value returned:

${variable%pattern)  

Here, we cut away the shortest occurrence of the pattern but now from the end of the variable value:

zarrelli:~$ ending=10102010103
zarrelli:~$ echo ${ending%1*3}
10102010

So, the shortest occurrence of the 1*3 pattern counted from the end of the file is 10102010103 so we get 10102010 back:

${variable%%pattern)  

Similar to the previous example, with ##, in this case, we cut away the longest occurrence of the pattern from the end of the variable value:

zarrelli:~$ ending=10102010103
zarrelli:~$ echo ${ending}
10102010103
zarrelli:~$ echo ${ending%1*3}
10102010
zarrelli:~$ echo ${ending%%1*3}
zarrelli:~$

Quite clear, isn't it? The longest occurrence is 1*3 is 10102010103, so we tear away everything and we return nothing, as this example which makes use of the evaluation of -z (is empty) will show:

zarrelli:~$ my_var=${ending%1*3}
zarrelli:~$ [[ -z "$my_var" ]] && echo "Empty" || echo "Not empty"
Not empty
zarrelli:~$ my_var=${ending%%1*3}
zarrelli:~$ [[ -z "$my_var" ]] && echo "Empty" || echo "Not empty"
Empty
${variable/pattern/substitution}

The reader familiar with regular expressions probably already understood what the outcome is: replace the first occurrence of the pattern in the variable by substitution. If substitution does not exist, then delete the first occurrence of a pattern in variable:

zarrelli:~$ my_var="Give me a banana"
zarrelli:~$ echo ${my_var}
Give me a banana
zarrelli:~$ echo ${my_var/banana/pear}
Give me a pear
zarrelli:~$ fruit=${my_var/banana/pear}
zarrelli:~$ echo ${fruit}
Give me a pear

Not so nasty, and we were able to instance a variable with the output of our find and replace:

${variable//pattern/substitution}  

Similar to the preceding, in this case, we are going to replace the occurrences of a pattern in the variable:

zarrelli@moveaway:~$ fruit="A pear is a pear and is not a banana"
zarrelli@moveaway:~$ echo ${fruit//pear/watermelon}
A watermelon is a watermelon and is not a banana

Like the preceding example, if substitution is omitted, a pattern is deleted from the variable:

${variable/#pattern/substitution}  

If the prefix of the variable matches, then replace the pattern with substitution in variable, so this is similar to the preceding but matches only at the beginning of the variable:

zarrelli:~$ fruit="a pear is a pear and is not a banana"
zarrelli:~$ echo ${fruit/#"a pear"/}
is a pear and is not a banana
zarrelli:~$ echo ${fruit/#"a pear"/"an apple"}
an apple is a pear and is not a banana

As usual, omitting means deleting the occurrence of the pattern from the variable.

${variable/%pattern/substitution}  

Once again, a positional replacement, this time at the end of the variable value:

zarrelli:~$ fruit="a pear is not a banana even tough I would 
like to eat a banana"

zarrelli:~$ echo ${fruit/%"a banana"/"an apple"}
a pear is not a banana even though I would like to eat an apple

A lot of nonsense, but it makes sense:

${!prefix_variable*}
${!prefix_variable@}

Match the name of the variable names starting with the highlighted prefix:

zarrelli:~$ firstvariable=1
zarrelli:~$ secondvariable=${!first*}
zarrelli@:~$ echo ${secondvariable}
firstvariable
zarrelli:~$ thirdvariable=${secondvariable}
zarrelli:~$ echo ${thirdvariable}
firstvariable
${variable:position}

We can decide from which position we want to start the variable expansion, so determining what part of its value we want to get back:

zarrelli:~$ picnic="Either I eat an apple or I eat a raspberry"
zarrelli:~$ echo ${picnic:25}
I eat a raspberry

So, we just took a part of the variable, and we decided the starting point, but we can also define for how long cherry-picking is done:

${variable:position:offset}
zarrelli:~$ wheretogo="I start here, I go there, no further"
zarrelli:~$ echo ${wheretogo:14:10}
I go there

So we do not go further, start at a position and stop at the offset; this way, we can extract whatever consecutive characters/digits we want from the value of a variable.

So far, we have seen many different ways to access and modify the content of a variable or, at least, of what we get from a variable. There is a class of very special variables left to look at, and these will be really handy when writing a script.

Special variables

Let's see now some variables which have some spacial uses that we can benefit from:

${1}, ${n}

The first interesting variables we want to explore have a special role in our scripts because they will let us capture more than an argument on our first command-line execution. Have a look at this bunch of lines:

!/bin/bash    
fistvariable=${1}
secondvariable=${2}
thirdvariable=${3}
echo "The value of the first variable is ${1}, the second
is ${2}, the third is ${3}"

Pay attention to $1, $2, $3:

zarrelli:~$ ./positional.sh 
The value of the first variable is , the second is , the third is

First try, no arguments on the command line, we see nothing printed for the variables:

zarrelli:~$ ./positional.sh 1 2 3
The value of the first variable is 1, the second is 2,
the third is 3

Second try, we invoke the script and add three digits separated by spaces and, actually, we can see them printed. The first on the command line corresponds to $1, the second to $2, and the third to $3:

zarrelli:~$ ./positional.sh Green Yellow Red  

The value of the first variable is Green; the second is Yellow; and the third is Red.

Third try, we use words with the same results. But notice here:

zarrelli:~$ ./positional.sh "One sentence" "Another one" 
A third one

The value of the first variable is One sentence, the second
is Another one, the third is A

We used a double quote to prevent the space between one sentence and another being interpreted as a divider for the command-line bits, and in fact, the first and second sentences were added as a complete string to the variables, but the third came up just with an A because the subsequent spaces, not quoted, were considered to be separators and the following bits taken as $4, $5, and $n. Note that we could also mix the order of assignment, as follows:

thirdvariable=${3}
fistvariable=${1}
secondvariable=${2}

The result would be the same. What is important is not the position of the variable we declare, but what positional we associate with it.

As you saw, we used two different methods to represent a positional variable:

${1}
$1

Are they the same? Almost. Look here:

#!/bin/bash    
fistvariable=${1}
secondvariable=${2}
thirdvariable=${3}
eleventhvariable=$11
echo "The value of the first variable is ${fistvariable},
the second is ${secondvriable}, the third is ${thirdvariable},
the eleventh is ${eleventhvariable}"

Now, let's execute the script:

zarrelli:~$ ./positional.sh "One sentence" "Another one" A 
third one

The value of the first variable is One sentence, the second
is Another one, the third is A, the eleventh is One sentence1

Interesting, the eleventhvariable has been interpreted as it were the positional $1 and added a 1. Odd, let's rewrite the echo in the following way:

eleventhvariable=${11}  

And run the script again:

zarrelli$ ./positional.sh "One sentence" "Another one" A third one
The value of the first variable is One sentence, the second is
Another one, the third is A, the eleventh is

Now we are correct. We did not pass an eleventh positional value on the command line, so the eleventhvariable has not been instantiated and we do not see anything printed to the video. Be cautious, always use ${}; it will preserve the value of the variable in your complex scripts when having a grasp of every single detail would be really difficult:

${0}  

This expands to the full path to the script; it gives you a way to handle it in your script. So, let's add the following line at the end of the script and execute it:

echo "The full path to the script is $0"
zarrelli:~$ ./positional.sh 1 2 3
The value of the first variable is 1, the second is 2, the
third is 3, the eleventh is

The full path to the script is ./positional.sh

In our case, the path is local, since we called the script from inside the directory that is holding it:

${#}  

Expands into the number of the arguments passed to the script, showing us the number of arguments that have been passed on the command line to the script. So, let's add the following line to our script and let's see what comes out of it:

echo "We passed ${#} arguments to the script"    
zarrelli:~$ ./positional.sh 1 2 3 4 5 6 7
The value of the first variable is 1, the second is 2, the
third is 3, the eleventh is

The full path to the script is ./positional.sh
We passed 7 arguments to the script
${@}
${*}

Gives us the list of arguments passed on the command line to the script, with one difference: ${@} preserves the spaces, the second doesn't:

#!/bin/bash
fistvariable=${1}
secondvariable=${2}
thirdvariable=${3}
eleventhvariable=${11}
export IFS=*
echo "The value of the first variable is ${fistvariable},
the second is ${secondvariable}, the third is ${thirdvariable},
the eleventh is ${eleventhvariable}"

echo "The full path to the script is $0"
echo "We passed ${#} arguments to the script"
echo "This is the list of the arguments ${@}"
echo "This too is the list of the arguments ${*}"
IFS=
echo "This too is the list of the arguments ${*}"

We changed the characters used by the shell as a delimiter to identify single words. Now, let us execute the script:

zarrelli:~$ ./positional.sh 1 2 3
The value of the first variable is 1, the second is 2,
the third is 3, the eleventh is

The full path to the script is ./positional.sh
We passed 3 arguments to the script
This is the list of the arguments 1 2 3
This too is the list of the arguments 1*2*3
This too is the list of the arguments 123

Here, you can see the difference at play:

  • *: This expands to the positional parameters, starting from the first and when the expansion is within double quotes, it expands to a single word and separates each positional parameter using the first character of IFS. If the latter is null, a space is used, if it is null the words are concatenated without separators.
  • @: This expands to the positional parameter, starting from the first, and if the expansion occurs within a double quote, each positional parameter is expanded to a word on its own:
${?}  

This special variable expands to the exit value of the last command executed, as we have already seen:

zarrelli:~$ /bin/ls disk.sh ; echo ${?} ; tt ; echo ${?}
disk.sh
0
bash: tt: command not found
127

The first command was successful, so the exit code is 0 ; the second gave an error 127command not found, since such a command as tt does not exist.

${$} expands to the process number of the current shell and for a script is the shell in which it is running. Let us add the following line to our positional.sh script:

echo "The process id of this script is ${$}"  

Then let's run it:

zarrelli:~$ ./positional.sh 1 2 3
The value of the first variable is 1, the second is 2, the
third is 3, the eleventh is

The full path to the script is ./positional.sh
We passed 3 arguments to the script
This is the list of the arguments 1 2 3
This too is the list of the arguments 1*2*3
This too is the list of the arguments 123
The process id of this script is 13081

Step by step, our script is telling us more and more:

${!}  

This is tricky; it expands to the process number of the last command executed in the background. Time to add some other lines to our script:

echo "The background process id of this script is ${!}"
echo "Executing a ps in background"
nohup ps &
echo "The background process id of this script is ${!}"

And now execute it:

zarrelli:~$ ./positional.sh 1 2 3
The value of the first variable is 1, the second is 2,
the third is 3, the eleventh is

The full path to the script is ./positional.sh
We passed 3 arguments to the script
This is the list of the arguments 1 2 3
This too is the list of the arguments 1*2*3
This too is the list of the arguments 123
The process id of this script is 13129
The background process id of this script is
Executing a ps in background
The background process id of this script is 13130
nohup: appending output to 'nohup.out'

We used nohup ps & to send the ps in the background (&) and detach it from the current terminal (nohup). We will see later, in more details the use of background commands; it suffices now to see how, before sending the process in to the background, we had no value to print for ${!} ; it was instanced only after we sent ps in to the background.

Do you see that?

nohup: appending output to 'nohup.out'  

Well, for our purposes, it has no value, so how can we redirect this useless output and get rid of it during the execution of our script? You know what? It is a tiny exercise for you to do before you start reading the next chapter, which will deal with the operators and much more fun.

You have been reading a chapter from
Mastering Bash
Published in: Jun 2017
Publisher: Packt
ISBN-13: 9781784396879
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image