Redirecting stdin, stdout, and stderr

In this recipe, we will learn how to redirect standard input, standard output, and standard error to and from files. Redirecting data to and from files is one of the basic principles of Linux and other Unix systems.

stdin is the shorthand word for standard input. stdout and stderr are the shorthand words for standard output and standard error, respectively.

Getting ready

It's best if we use the Bash shell for this recipe for compatibility purposes.

How to do it…

To get the hang of redirections, we will be performing a bunch of experiments here. We are really going to twist and turn the redirections and see stdout, stderr, and stdin operate in all kinds of ways. Let's get started:

1. Let's start by saving a list of the files and directories in the top root directory. We can do this by redirecting standard output (stdout) from the ls command into a file:

   $> cd
   $> ls / > root-directory.txt

2. Now, take a look at the file with cat:

   $> cat root-directory.txt

3. Now, let's try the wc command to count lines, words, and characters. Remember to press Ctrl + D when you have finished typing in the message:

   $> wc
   hello,
   how are you?
   Ctrl+D
        2       4      20

4. Now that we know how wc works, we can redirect its input to come from a file instead—the file we created with the file listing:

   $> wc < root-directory.txt
   29  29 177

5. What about standard error? Standard error is its own output stream, separated from standard output. If we redirect standard output and generate an error, we will still see the error message on the screen. Let's try it out:

   $> ls /asdfasdf > non-existent.txt
   ls: cannot access '/asdfasdf': No such file or directory

6. Just like standard output, we can redirect standard error. Notice that we don't get any error message here:

   $> ls /asdfasdf 2> errors.txt

7. The error messages are saved in errors.txt:

   $> cat errors.txt
   ls: cannot access '/asdfasdf': No such file or directory

8. We can even redirect standard output and standard error at the same time, to different files:

   $> ls /asdfasdf > root-directory.txt 2> errors.txt

9. We can also redirect standard output and error into the same file for convenience:

   $> ls /asdfasdf &> all-output.txt

10. We can even redirect all three (stdin, stdout, and stderr) at the same time:

    $> wc < all-output.txt > wc-output.txt 2> \
    > wc-errors.txt

11. We can also write to standard error from the shell to write error messages of our own:

    $> echo hello > /dev/stderr
    hello

12. Another way of printing a message to stderr from Bash is like this:

    $> echo hello 1>&2
    hello

13. However, this doesn't prove that our hello message got printed to standard error. We can prove this by redirecting the standard output to a file. If we still see the error message, then it's printed on standard error. When we do this, we need to wrap the first statement in parenthesis to separate it from the last redirect:

    $> (echo hello > /dev/stderr) > hello.txt
    hello
    $> (echo hello 1>&2) > hello.txt
    hello

14. Stdin, stdout, and stderr are represented by files in the /dev directory. This means we can even redirect stdin from a file. This experiment doesn't do anything useful—we could have just typed wc, but it proves a point:

    $> wc < /dev/stdin
    hello, world!
    Ctrl+D
         1       2      14

15. All of this means that we can even redirect a standard error message back to standard output:

    $> (ls /asdfasdf 2> /dev/stdout) > \ 
    > error-msg-from-stdout.txt
    $> cat error-msg-from-stdout.txt 
    ls: cannot access '/asdfasdf': No such file or directory

How it works…

Standard output, or stdout, is where all the normal output from programs gets printed. Stdout is also referred to as file descriptor 1.

Standard error, or stderr, is where all error messages get printed. Stderr is also referred to as file descriptor 2. That is why we used 2> when we redirected stderr to a file. If we wanted to, for clarity, we could have redirected stdout as 1> instead of just >. But the default redirection with > is stdout, so there is no need to do this.

When we redirected both stdout and stderr in Step 9, we used an & sign. This reads as "stdout and stderr".

Standard input, or stdin, is where all input data is read from. Stdin is also referred to as file descriptor 0. Stdin redirects with a <, but just as with stdout and stderr, we can also write it as 0<.

The reason for separating the two outputs, stdout and stderr, is so that when we redirect the output from a program to a file, we should still be able to see the error message on the screen. We also don't want the file to be cluttered with error messages.

Having separate outputs also makes it possible to have one file for the actual output, and another one as a log file for error messages. This is especially handy in scripts.

You might have heard the phrase "Everything in Linux is either a file or a process". That saying is true. There is no other thing in Linux, except for files or processes. Our experiments with /dev/stdout, /dev/stderr, and /dev/stdin proved this. Files represent even the input and output of programs.

In Step 11, we redirected the output to the /dev/stderr file, which is standard error. The message, therefore, got printed on standard error.

In Step 12, we pretty much did the same thing but without using the actual device file. The funny-looking 1>&2 redirection reads as "send standard output to standard error".

There's more…

Instead of using /dev/stderr, for example, we could have used /dev/fd/2, where fd stands for file descriptor. The same goes for stdout, which is /dev/fd/1, and stdin, which is /dev/fd/0. So, for example, the following will print the list to stderr:

$> ls / > /dev/fd/2

Just like we can send standard output to standard error with 1>&2, we can do the opposite with 2>&1, which means we can send standard error to standard output.