Answer to Question 4 (20 points, 45 minutes)

There are a few decisions that you have to make. First, the main process is going to have the read end of pipe open to each grandchild. How should this be stored? Basically, there were three approaches to this:

Store each one in an array of maxcli elements.
Store each one in a dlist.
Maintain a fd_set for all of these, and use that.

I'll give a description of approach #1, and then give code for other two at the end of this page. My array was called pipes. The read end of grandchild i's pipe is stored in pipes[i]. If there is no grandchild i, then pipes[i] is -1.

So, initialization consists of the following steps:

Initialize maxcli, sock and nclients as before.
Initialize pipes to be an array with maxcli elements, all of which are -1.

The initialization code is below:

main(int argc, char **argv)
{
  int sock, connection, dummy, nclients, maxcli;
  int p[2], *pipes, i, mx;
  fd_set fds;

  maxcli = atoi(argv[1]);
  sock = serve_socket("plank", 5005);
  nclients = 0;
  pipes = (int *) malloc(sizeof(int)*maxcli);
  for (i = 0; i < maxcli; i++) pipes[i] = -1;

The main loop of the program is like before. There are two parts -- what to do if there are less than maxcli clients, and what to do if there are exactly maxcli clients. In the former case, accept_connection() is called as before, but then something different is done with the connection. First pipe() is called to open a new pipe, and then a child is forked. That child closes all file descriptors except connection and the write end of the pipe. This includes all of the open file descriptors in pipes, which are duplicated over the fork() call. Now, that child forks a grandchild and exits. The grandchild calls doserver, and services the client. The original csm process closes the connection and the write end of the pipe, waits for the child to exit, and then finds an empty element of pipes into which it copies the read end of the pipe. This part of the code looks as follows:

  while(1) {
    if (nclients < maxcli) {
      connection = accept_connection(sock);
      nclients++;
      pipe(p);

      if (fork() == 0) {
        /* Close all fds except the connection and p[1] */
        close(sock);
        close(p[0]);
        for (i = 0; i < maxcli; i++) if (pipes[i] != -1) close(pipes[i]);
 
        /* fork the grandchild and exit */
        if (fork() == 0) {
          doserver(connection);
        } else {
          exit(0);
        }
      } else {
        close(connection);
        close(p[1]);
        wait(&dummy);
        for (i = 0; i < maxcli && pipes[i] != -1; i++) ;
        pipes[i] = p[0];
      }

When the number of clients reaches maxcli, the csm process must wait for some grandchildren to finish. It does this by calling select() on all the read ends of the pipes. Thus, it initializes a fd_set to contain all the read ends of the pipes, and calls select(). If a grandchild is done, select() will report that its pipe has something to read (of course, the read() call would return 0, but it would not block, so select() is doing the right thing). The code finds all readable pipes closes them and decrements nclients. This last part of the code follows:

    } else {

      /* set up the fd_set */
      FD_ZERO(&fds);
      mx = 0;
      for (i = 0; i < maxcli; i++) {
        FD_SET(pipes[i], &fds);
        if (pipes[i] > mx) mx = pipes[i];
      }

      select(mx+1, &fds, NULL, NULL, NULL);

      /* Find all closed fd's */
      for (i = 0; i < maxcli; i++) {
        if (FD_ISSET(pipes[i], &fds)) {
          close(pipes[i]);
          pipes[i] = -1;
          nclients--;
        }
      }
    }
  }
}

Grading

Grading went as follows. There were points and deductions, defined as follows:

Initialization

Initializing the pipe-storage data structure: 1 point
Doing all the other initialization stuff as before: 1 point
Overall structure looks ok: 1 point

If there are less than maxcli clients

Calling accept_connection(): 1 point
Calling pipe(): 1 point
Calling fork() twice somewhere: 1 point
Having the child exit: 1 point
Having the child or grandchild close sock and p[0]: 1 point
Having the child or grandchild close the other grandchildrens' pipes: 1 point
Having the grandchild call doserver: 1 point
Having the parent close connection and p[1]: 1 point
Having the parent call wait(): 1 point
Having the parent put the pipe into the pipe data structure: 1 point
Incrementing the number of clients: 1 point

If there are exactly maxcli clients

Setting up the fd_set correctly: 1 point
Calling select(): 1 point
Getting the max fd correct in select(): 1 point
Closing the pipes: 1 point
Deleting the pipes from the data structure: 1 point
Decrementing the number of clients: 1 point

Deductions

Extraneous stuff after doserver (which doesn't return): 1 point.
Extraneous code that would cause bugs: 1 point.
Data structure is a stack rather than an array or list: 1 point
Assuming all pipe fd's are less than maxcli: 1 point

The stack point is in case you did something like put the pipe's file descriptor into an array indexed by nclients -- this assumes that only the most recent grandchildren will finish first, which is not a reasonable assumption (actually, it's just a plain bug).

The last deduction is in case you use the fd_set to keep track of open fd's. This is ok, as long as you assume that the maximum fd value is 64 or something like that. If you traverse the fd_set from 0 to maxcli, you'll miss some. Think about it.

Code for storing the fd's in a dlist:

main(int argc, char **argv)
{
  int sock, connection, dummy, nclients, maxcli;
  int p[2], i, mx;
  fd_set fds;
  Dlist pipes, tmp;

  maxcli = atoi(argv[1]);
  sock = serve_socket("plank", 5005);
  nclients = 0;
  pipes = make_dl();

  while(1) {
    if (nclients < maxcli) {
      connection = accept_connection(sock);
      nclients++;
      pipe(p);

      if (fork() == 0) {
        /* Close all fds except the connection and p[1] */
        close(sock);
        close(p[0]);
        dl_traverse(tmp, pipes) close(tmp->val);
 
        /* fork the grandchild and exit */
        if (fork() == 0) {
          doserver(connection);
        } else {
          exit(0);
        }
      } else {
        close(connection);
        close(p[1]);
        wait(&dummy);
        dl_insert_b(pipes, p[0]);
      }
    } else {

      /* set up the fd_set */
      FD_ZERO(&fds);
      mx = 0;
      dl_traverse(tmp, pipes) {
        FD_SET(tmp->val, &fds);
        if (tmp->val> mx) mx = tmp->val;
      }

      select(mx+1, &fds, NULL, NULL, NULL); 

      /* Find all closed fd's */
      dl_traverse(tmp, pipes) {
        if (FD_ISSET(tmp->val, &fds)) {
          close(tmp->val);
          tmp = tmp->blink;
          dl_delete_node(tmp->flink);
          nclients--;
        }
      }
    }
  }
}

Code for storing the fd's in a master fd_set. This is probably the most efficient implementation.

#define MAXFD 64

main(int argc, char **argv)
{
  int sock, connection, dummy, nclients, maxcli;
  int p[2], i, mx;
  fd_set master, fds;

  maxcli = atoi(argv[1]);
  sock = serve_socket("plank", 5005);
  nclients = 0;
  FD_ZERO(&master);

  while(1) {
    if (nclients < maxcli) {
      connection = accept_connection(sock);
      nclients++;
      pipe(p);

      if (fork() == 0) {
        /* Close all fds except the connection and p[1] */
        close(sock);
        close(p[0]);
        for (i = 0; i < MAXFD; i++) {
          if (FD_ISSET(i, &master)) close(i);
        }
 
        /* fork the grandchild and exit */
        if (fork() == 0) {
          doserver(connection);
        } else {
          exit(0);
        }
      } else {
        close(connection);
        close(p[1]);
        wait(&dummy);
        FD_SET(p[0], &master);
      }
    } else {

      /* set up the fd_set */
      memcpy(&fds, &master, sizeof(fd_set));
      select(MAXFD, &fds, NULL, NULL, NULL); 

      /* Find all closed fd's */
      for (i = 0; i < MAXFD; i++) {
        if (FD_ISSET(i, &fds)) {
          close(tmp->val);
          FD_CLR(i, &master);
          nclients--;
        }
      }
    }
  }
}