Sending output from one command to another

One of the best features of the Unix shells is the ease of combining many commands to produce a report. The output of one command can appear as the input to another, which passes its output to another command, and so on. The output of this sequence can be assigned to a variable. This recipe illustrates how to combine multiple commands and how the output can be read.

Getting ready

The input is usually fed into a command through stdin or arguments. The output is sent to stdout or stderr. When we combine multiple commands, we usually supply input via stdin and generate output to stdout.

In this context, the commands are called filters. We connect each filter using pipes, sympolized by the piping operator (|), like this:

$ cmd1 | cmd2 | cmd3 

Here, we combine three commands. The output of cmd1 goes to cmd2, the output of cmd2 goes to cmd3, and the final output (which comes out of cmd3) will be displayed on the monitor, or directed to a file.

How to do it...

Pipes can be used with the subshell method for combining outputs of multiple commands.

1. Let's start with combining two commands:

        $ ls | cat -n > out.txt

The output of ls (the listing of the current directory) is passed to cat -n, which in turn prepends line numbers to the input received through stdin. The output is redirected to out.txt.

2. Assign the output of a sequence of commands to a variable:

        cmd_output=$(COMMANDS)

This is called the subshell method. Consider this example:

        cmd_output=$(ls | cat -n)
        echo $cmd_output

Another method, called back quotes (some people also refer to it as back tick) can also be used to store the command output:

        cmd_output=`COMMANDS`

Consider this example:

        cmd_output=`ls | cat -n`
        echo $cmd_output

Back quote is different from the single-quote character. It is the character on the ~ button on the keyboard.

There's more...

There are multiple ways of grouping commands.

Spawning a separate process with subshell

Subshells are separate processes. A subshell is defined using the ( ) operators:

• The pwd command prints the path of the working directory
• The cd command changes the current directory to the given directory path:

        $> pwd 
        / 
        $> (cd /bin; ls) 
        awk bash cat... 
        $> pwd 
        /

When commands are executed in a subshell, none of the changes occur in the current shell; changes are restricted to the subshell. For example, when the current directory in a subshell is changed using the cd command, the directory change is not reflected in the main shell environment.

Subshell quoting to preserve spacing and the newline character

Suppose we are assigning the output of a command to a variable using a subshell or the back quotes method, we must use double quotes to preserve the spacing and the newline character (\n). Consider this example:

$ cat text.txt
1
2
3

$ out=$(cat text.txt)
$ echo $out
1 2 3 # Lost \n spacing in 1,2,3

$ out="$(cat text.txt)"
$ echo $out
1
2
3