CS360 Lecture notes -- Sh Redirection


This set of lecture notes has some forward references to things that you will learn later in the semester, such as dup2(). For now, you can ignore it when you see it. You may want to come back later in the semster to reinforce your knowledge of this.
There are many different shells that people use under Unix. Two important ones are the Bourne shell (sh) and the C shell (csh).

Most people use the C shell (or Bash or tcsh or zcsh) as an interative interpreter to execute programs because it has the history function and some other junk. The Bourne shell is more primitive, but many people (e.g. me) like it as a scripting language because of its simplicity.

There are lecture notes for the Bourne shell in my ``Scripts and Utilities'' class. You may find them at http://web.eecs.utk.edu/~plank/plank/classes/cs494/494/notes/Sh/lecture.html.

What we are concerned with here are the redirection primitives. Many of these are simple, and are the same in both the Bourne shell and the C shell.

So, for example, suppose the file f1 contains the bytes ``This is f1''. The following redirections should not confuse you. In each case, ask yourself what should the output of the command should be:
UNIX> cat f1
This is f1
UNIX> cat < f1
This is f1
UNIX> < f1 cat
This is f1
UNIX> < cat f1
cat: No such file or directory.
UNIX> cat f1 > f2
UNIX> cat f2
This is f1
UNIX> cat f1 >> f2
UNIX> cat f2
This is f1
This is f1
UNIX> > f2 < f1 cat
UNIX> cat f2
This is f1
UNIX> 
Now, suppose there is no file f3. When we say cat f1 f3, it will print the contents of f1 to standard output and an error message to standard error. Typically, both of these go to the screen:
UNIX> cat f1 f3
This is f1
cat: f3: No such file or directory
UNIX> 
However, if you redirect standard output to a file, then f1 will go to the file, and the error message will go to the screen. Why? because the shell is calling dup2(fd, 1) to the output file, but nothing for file descriptor 2:
UNIX> cat f1 f3 > f2
cat: f3: No such file or directory
UNIX> cat f2
This is f1
UNIX> 
With the C shell, you can redirect both standard output and standard error to the same file by using >& :
UNIX> cat f1 f3 >& f2
UNIX> cat f2
This is f1
cat: f3: No such file or directory
UNIX> 
The Bourne shell has different primitives for dealing with standard output and standard error. Whenever you say x>, it will redirect file desriptor x. For example, another way to redirect standard output to a file under the bourne shell is:
UNIX> sh
$ cat f1 f3 1>f2
cat: cannot open f3
$ cat f2
This is f1
$ 
And we can redirect standard output and standard error to different files very easily:
$ cat f1 f3 1>f2 2>f5
$ cat f2
This is f1
$ cat f5
cat: cannot open f3
$ 
The shell processes these statments left to right, so I can do multiple redirections of standard output, and not only will the shell allow it without complaint, the shell will create all of the files that you specify:
$ rm f2
$ cat f1 f3 1>f2 1>f5
cat: cannot open f3
$ cat f2
$ cat f5
This is f1
$ 
As you can see, f2 was created and is empty. This is because it was opened for writing in the first redirection, and then closed in the second redirection statement.

Can we redirect standard input multiple times? It depends on your shell:

UNIX> echo "This is f1" > f1
UNIX> echo "This is f2" > f2
UNIX> sh -c "cat < f1 < f2"
This is f2
UNIX> sh -c "cat < f2 < f1"
This is f1
UNIX> csh -c "cat < f1 < f2"
Ambiguous input redirect.
UNIX> 

Look at what happens when try to have input and output come from the same file:

$ cat f2
This is f2
$ head f2 > f2
$ cat f2
$ 
The shell does its redirection before executing head. That means that f2 is truncated before head is called, and when head is called, f2 is empty. Therefore, head exits, and f2 remains empty.

The same thing happens if you redirect head's standard input to the standard output:

$ echo "This is f2" > f2
$ head < f2 > f2
$ cat f2
$ 

Now consider x>y again. If you specify y as &n, then it will make sure that file descriptor x in the program is identical to file descriptor y. Another way of saying this is that: "Whatever file descriptor y is currently going, not file descriptor x is going there too, and the two are identical." (You will learn later how this works with the dup2() system call). Thus, look at the following:

$ cat f1 f3 > f2 2>&1
$ cat f2
This is f1
cat: cannot open f3
$
What is going on? First, you redirect standard output to f2. That means file descriptor 1 is going to f2. Then the 2>&1 part says to make file descriptor 2 identical to file descriptor 1. means that standard error will go to f2 as well.

Again, these are processed by the shell from left to right: Suppose you reverse the order of the statements:

$ cat f1 f3 2>&1 > f2
cat: cannot open f3
$ cat f2
This is f1
$ 
Now, the 2>&1 part says to make file descriptor 2 identical to file descriptor 1, which at the time the shell sees this command, is going to the screen. Next, it redirects file descriptor 1 to f2. So, standard error goes to the screen, and standard output goes to f2.

Loot at:

$ cat f1 f3 >f2 2>&1 1>f5
$ cat f2
cat: cannot open f3
$ cat f5
This is f1
$ 
Now, standard output first goes to f2, then the 2>&1 part makes standard error identical to standard output. In other words, both are going to f2. Then the 1>f5 part makes standard output to to f5. Therefore, that line is equivalent to: ``cat f1 f3 >f5 2>f2.''

You can make use of other file descriptors if you want:

$ cat f1 f3 3>f2 1>f5 2>&1 1>&3
$ cat f2
This is f1
$ cat f5
cat: cannot open f3
$ 
Figure that one out for yourself.

You can use this technique to do some pretty ugly stuff. Look at badbadcode.c:

main()
{
  char *s = "Hi!\n";
  int i;
  int fd;

  i = write(3, s, strlen(s));
  printf("%d\n", i);
  if (i < 0) perror("write");
}
And now check the following out. Is this a good thing or a bad thing?
$ badbadcode
-1
write: Bad file number
$ badbadcode 3>f5
4
$ cat f5
Hi!
$ badbadcode 3>&1
Hi!
4
$ 
I'd say it's a good thing, as long as you don't tell anyone about it. In fact, I've done the following to help me debug. Suppose you have a subtle bug in a fairly large piece of code. And you'd like to create some output to help you, but you've already junked up standard output so much that you can't use it. Worse, you know that the bug is nested in a bazillion procedure calls, and you don't want to worry about the flow of control getting there or passing FILE *'s to all of those procedure calls.

Then, you do something like dont_admit_i_taught_you_this.c:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

void v(int i) 
{ 
  char s[100];

  sprintf(s, "Here's my error message.  V was called with %d\n", i);
  write(9, s, strlen(s));
}

void u() { printf("Blah blah blah blah blah blah.\n");  v(4); }
void t() { printf("Blah blah blah blah blah blah.\n");  u(); }
void s() { printf("Blah blah blah blah blah blah.\n");  u(); t(); }
void r() { printf("Blah blah blah blah blah blah.\n");  u(); s(); }
void q() { printf("Blah blah blah blah blah blah.\n");  u(); r(); }
void p() { printf("Blah blah blah blah blah blah.\n");  q(); q(); v(5); }
void o() { printf("Blah blah blah blah blah blah.\n");  p(); }
void n() { printf("Blah blah blah blah blah blah.\n");  o(); }
void m() { printf("Blah blah blah blah blah blah.\n");  n(); }
void l() { printf("Blah blah blah blah blah blah.\n");  m(); }
void k() { printf("Blah blah blah blah blah blah.\n");  l(); m(); v(6); }
void j() { printf("Blah blah blah blah blah blah.\n");  k(); }
void i() { printf("Blah blah blah blah blah blah.\n");  j(); }
void h() { printf("Blah blah blah blah blah blah.\n");  i(); }
void g() { printf("Blah blah blah blah blah blah.\n");  h(); }
void f() { printf("Blah blah blah blah blah blah.\n");  g(); }
void e() { printf("Blah blah blah blah blah blah.\n");  f(); v(7); }
void d() { printf("Blah blah blah blah blah blah.\n");  e(); }
void c() { printf("Blah blah blah blah blah blah.\n");  d(); }
void b() { printf("Blah blah blah blah blah blah.\n");  c(); }
void a() { printf("Blah blah blah blah blah blah.\n");  b(); v(8); }


main()
{
  a();
}

As you can see, v() is getting called a lot. Each time it gets called, I'm writing a string into file descriptor 9. What's file descriptor 9? Well, you can determine that on the shell:

$ dont_admit_i_taught_you_this > /dev/null 9>elog.txt
$ cat elog.txt
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 5
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 4
Here's my error message.  V was called with 5
Here's my error message.  V was called with 6
Here's my error message.  V was called with 7
Here's my error message.  V was called with 8
$ dont_admit_i_taught_you_this > /dev/null
$ 
In that last call, I didn't redirect file descriptor 9. Therefore, the read() statement failed and returned -1. It's like nothing happened.

Remember that trick -- it may come in handy sometime.