CS360 Lecture notes -- The Dow Jones Question

Jim Plank

Directory: /blugreen/homes/plank/cs360/notes/DowJones

Lecture notes -- plain text: /blugreen/homes/plank/cs360/notes/DowJones/lecture

Lecture notes -- html: file:/blugreen/homes/plank/cs360/notes/DowJones/lecture.html

This question appeared on one of my exams a few years ago. In this class, I'll go over the question and its answer in detail. It gives you good practice with fork(), pipe(), signal() and sockets.

The question

Suppose Dow Jones Industrial Inc. decides to support a stock market server. It works as follows. At Dow Jones, there is a person typing in current stock prices. The server serves a socket on port 200 on the machine stocks.dowjones.com. Whenever a client attaches to that socket, the client will be sent whatever stock prices the person is currently typing in. This continues until the client quits.

Obviously, the client should be able to attach to this server by calling telnet:

UNIX> telnet stocks.dowjones.com 200
...

You are a summer intern at Dow Jones Industrial, and your job is to write this server. Fortunately, because you have taken CS360, this will only take you a few hours, and you can spend the rest of the summer getting paid while you pretend to work.

At the heart of this server is the subroutine broadcast_info():

broadcast_info(Dlist clients)
{
  char line[100];
                                                /* Read in stock price  */
  while(fgets(stdin, line, 100) != NULL) {      /* from standard input */

    for (tmp = clients->flink; tmp != clients; tmp = tmp->flink) {
      write(tmp->val, line, strlen(line)) {     /* Write stock prices */
    }                                           /* to every client */
  }
}

Part 1: 7 points

If a client quits, a SIGPIPE signal is generated when the server tries to write to the client's socket. Modify the above subroutine to catch the SIGPIPE signal, and to deal with it correctly. You can modify the above procedure in whatever way you see fit. This includes adding global variables.

Part 2: 8 points

Having clients join is a trickier matter. Let us suppose that our server process has a pipe p open to its parent process, and is serving a socket s. The parent knows whenever a client wants to join (you'll see how in part 3). When a client wants to join, the parent sends a newline to the server over the pipe. The server can then do an accept_connection(s) and know that the accept_connection(s) will return with a connection.

Update broadcast_info() to work with this setup. By ``work'', I mean that it should be doing two things at once -- waiting for lines to be entered on standard input (these lines are broadcast to clients), and waiting for new clients to be announced on the pipe.

It should now look as follows:

broadcast_info(int *p, int s, Dlist clients)
{
 ...
}

Broadcast_info() should still catch SIGPIPE and deal with it correctly.

Part 3: 10 points

Here's the rest of the code for the server (this includes the parent process). Explain how this code works. Specifically, how are things initialized? What happens when clients join (with telnet). What happens when clients quit?

main()
{
  int mainsock, secondsock, i, j;
  int p[2];
  Dlist clients;
  char c;

  pipe(p);

  if (fork() == 0) {
    secondsock = serve_socket("stocks.dowjones.com", 5000);
    clients = make_dl();
    close(p[1]);
    broadcast_info(p, secondsock, clients);
    exit(0);
  } else {
    mainsock = serve_socket("stocks.dowjones.com", 200);
    close(0);
    close(p[0]);
    while(1) {
      i = accept_connection(mainsock);
      if (fork() == 0) {
        close(p[1]);
        j = request_connection("stocks.dowjones.com", 5000);
        while(read(j, &c, 1) != 0) write(i, &c, 1);
        exit(0);
      } else {
        close(i);
        write(p[1], "\n", 1);
      }
    }
  }
}

Part 4: 5 points

This program has a problem with zombie processes. Explain this problem, and then fix it.

The Answer

Part 1

The key here is to catch SIGPIPE and delete the current client file descriptor from the client list. Here's code that will do this:

int Sigpipe_caught;
pipe_handler()
{
  Sigpipe_caught = 1;
}

broadcast_info(Dlist clients)
{
  char line[100];
                                          
  signal(SIGPIPE, pipe_handler);

  while(fgets(stdin, line, 100) != NULL) {

    for (tmp = clients->flink; tmp != clients; tmp = tmp->flink) {
      Sigpipe_caught = 0;
      write(tmp->val, line, strlen(line));
      if (Sigpipe_caught) {
        close(tmp->val);
        tmp = tmp->blink;
        dl_delete_node(tmp->flink);
      }
    }                                    
  }
}

You can also make tmp global and delete it in the handler. Note though that you have take care in deleting the node that tmp points to the right place (the node previous to the deleted one) so that the for() loop will go the right place.

Part 2

Now, you need to call select() to read from two file descriptors at once: standard input, and the pipe. If select() returns that standard input is ready to be read, then you read a line and broadcast the line as before. However, if select() returns that the pipe is ready to be read, then you know two things -- there is a newline on the pipe that needs to be read, and there is a client waiting for a connection, so you should call accept_connection(s), and put the resulting file descriptor on the client list. The code is below:

broadcast_info(int *p, int s, Dlist clients)
{
  Dlist tmp;
  fd_set readset;
  int i;
  char line[100];

  signal(SIGPIPE, pipe_handler);

  while (1) {
    FD_ZERO(&readset);
    FD_SET(0, &readset);
    FD_SET(p[0], &readset);

    select(p[0]+1, &readset, NULL, NULL, NULL);
    if (FD_ISSET(p[0], &readset)) {
      if (read(p[0], line, 1) == 0) exit(0);
      i = accept_connection(s);
      dl_insert_b(clients, i);
    } else {
      if (fgets(line, 100, stdin) == NULL) {
        fprintf(stderr, "Stdin closed.  Bye\n");
        exit(1);
      }
      for (tmp = clients->flink; tmp != clients; tmp = tmp->flink) {
        Sigpipe_caught = 0;
        write(tmp->val, line, strlen(line));
        if (Sigpipe_caught) {
          close(tmp->val);
          tmp = tmp->blink;
          dl_delete_node(tmp->flink);
        }
      }
    }
  }
}

Part 3

This is a very subtle piece of code. First the program sets up a pipe and forks off a child process. The parent serves a socket at port 200, and loops as follows: It waits for a connection on the socket. When it receives a connection, it sends a newline along the pipe to its child. It also forks off another child process, which calls request_connection on port 5000. After this child receives the connection, it sends bytes from the connection at port 5000 to the client process that has called telnet. The initial child process serves a socket at port 5000 and runs broadcast_info().

Thus, when a client joins, the parent returns from accept_connection(), sends a newline to the broadcast_info() process, and forks off a child process which connects to the broadcast_info process via port 5000, and shuttles bytes from that process to the client. When the client quits, the child process that is doing the byte-shuttling will generate a SIGPIPE signal and die. Then the broadcast_info process will generate a SIGPIPE signal and deal with it gracefully.

Why do we need this convoluted process structure? Because the broadcast_info process cannot call select() to wait for either stuff to come in on standard input or for someone to request a connection on port 200. Therefore, we add the extra parent process to wait on port 200. Since the connection is made with the parent, and since we want a connection to be made with the broadcast_info() process, we fork off the extra process which shuttles bytes.

There is probably a better way to do this, using listen(), or perhaps even select(), but I don't know what it is -- the solution given here is comparatively simple, and is the type of thing you often end up doing in Unix. Is it inefficient? Yes. The process that shuttles bytes is something that we shouldn't have to do, but we do because it makes life easy. If we had a good thread-based model of programming supported by the operating system, we could do this much easier.

Part 4

There is one process forked off per client to do byte-shuttling. When this process dies (via SIGPIPE), it becomes a zombie, because its parent process is still alive and never calls wait(). However, the parent really can't know when the child dies, and can't call wait() indiscriminately, since it has to wait for accept_connection() to return. Therefore, to fix this, we can use the hack I described in class -- fork off a child which forks off a grandchild (that does the byte-shuttling) and then returns instantly. Since the child returns instantly, the parent process can wait for it, and the grandchild will then be inherited by init, and will not be a zombie when it dies. The new code, (starting at the while(1) statement) is as follows:

    while(1) {
      i = accept_connection(mainsock);
      if (fork() == 0) {
        if (fork() == 0) {
          close(p[1]);
          j = request_connection("stocks.dowjones.com", 5000);
          while(read(j, &c, 1) != 0) write(i, &c, 1);
          exit(0);
        } else {
          exit(0);
        }
      } else {
        wait(&statusp);
        close(i);
        write(p[1], "\n", 1);
      }