CS360 Final Exam - May 15, 2024 - Answers and Grading

Question 3

• A: Round up and add eight: 16.
• B: Round up and add eight: 88.
• C: The first free() hooks a's chunk onto the free list, and the second free() hooks b's chunk onto the free list. So, *(a+1) is the flink pointer for a, which is unknown at this point: Unknown.
• D: This is the blink pointer for a, which points to b's chunk: 0xb000 (remember, the beginning of b's chunk is 8 bytes before the return value of malloc().
• E: This is the flink pointer for b, which points to a's chunk: 0x6000.
• F: This is the blink pointer for b: NULL = 0x0

 Variables   |    A    |    B    |    C    |    D    |    E    |    F   
 ----------- | ------- | ------- | ------- | ------- | ------- | ------ 
 a,b         |   16    |    88   | unknown |  0xb000 |  0x6000 |   0x0  
 c,d         |   16    |   104   | unknown |  0xc000 |  0x7000 |   0x0  
 p,q         |   16    |    96   | unknown |  0xa000 |  0x5000 |   0x0  
 x,y         |   16    |   112   | unknown |  0xd000 |  0x8000 |   0x0  

Grading

Question 4

Grading

Question 5

Grading

free() is going to back up 8 bytes from the address passed to it. It's expecting the size to be there. Because this program modified the size (adding 96 to it), it's going to add a node back onto the freelist that is too large. Some of the bytes it puts back onto the freelist might be in use from other malloc calls. Now subsequent malloc() calls could allocate memory that is currently in use to another pointer and you'll have allocated the same memory twice so you're now stomping over memory.

I labeled a lot of answers as "vague", because they use some good words, but lack the detail to show that they understand the answer. For example:

Because we've messed around with the size of the pointers and changed the values that they're pointing to, mallocing or freeing without freeing the original addresses could cause serious issues.

Answers to the Fill-in-the-Blank Questions:

A: The most likely value printed on Line 3 is undefined. This is because the pointer a is decremented by 2, pointing to memory that is outside the allocated space (before the start of the memory block allocated by malloc). Accessing this location invokes undefined behavior, and the specific value could be anything, such as garbage data present in that memory location.

B: Similar to A, the most likely value printed on Line 4 is also undefined for the same reasons: b is decremented by 2 and points to memory that was not allocated for it, invoking undefined behavior.

C: The value printed on Line 5 is undefined. This operation accesses memory after modifying a pointer in a way that leads to undefined behavior (by decrementing and then attempting to access beyond the original allocated memory block).

D: The value printed on Line 6 is undefined. As with Line 5, this accesses memory after the pointer has been manipulated in an undefined way.

E: The value printed on Line 7 is undefined. Since b is decremented and subsequently, the pointer is used to access memory out of its original bounds, it results in undefined behavior.

F: The value printed on Line 8 is undefined. Again, this is because the memory is accessed out of bounds of the originally allocated block after modifying the pointer b.

Other Possible Values for Question A (and why): Another possible value for Line 3 (as well as Line 4) could be 0 or another specific integer value. This might occur if the memory before the allocated block was previously used and then set to 0 or left with residual values from a prior operation or allocation. The specific integer value printed would depend on whatever value happens to reside in those memory locations at the time of access.

Explanation of Problems with Subsequent malloc() and free() Calls: When procedure a() returns, the potential problems with subsequent malloc() and free() calls stem from the undefined behavior invoked during the function. Here are some specific issues:

Corruption of Memory Management Data Structures: Manipulating pointers that point outside allocated blocks (as a and b are decremented) and then performing operations like *a += *b; could potentially corrupt metadata used by the memory management system (like malloc's internal structures for managing free and used memory). This corruption can cause malloc() to behave unpredictably or incorrectly allocate memory, potentially leading to crashes or data corruption.

Undefined Pointer Arithmetic and Dereferencing: By manipulating pointers to point outside their allocated regions and then using them (e.g., a -= 2;, then dereferencing and modifying *a), the program may overwrite important data or control structures unrelated to the original allocation. This can also lead to memory leaks or double-free errors in subsequent free() calls if the memory management structures are corrupted.

Leaks and Double-Free Errors: If the internal bookkeeping of the memory allocator is corrupted, subsequent calls to free() may not correctly deallocate memory, leading to memory leaks. Alternatively, it might attempt to double-free memory, leading to crashes or further corruption of heap management data.

In general, accessing or manipulating memory out of the bounds of allocated blocks as done in this function can have unpredictable effects and is considered unsafe, leading to instability and security vulnerabilities in software.

Question 6 - 15 points

• pthread_mutex_lock(): This only returns when the calling thread holds the lock. If no thread holds the lock when it is called, then it will return instantly. Otherwise, it will block and wait for another thread to unlock the mutex, and for no other thread to successfully call pthread_mutex_lock().

  pthread_mutex_unlock(): This should only be called by a thread when it holds the lock. It unlocks the mutex, releasing it for other threads to lock.

  The purpose of pthread_mutex_lock()/pthread_mutex_unlock is to protect shared data from race conditions. Proper use ensures that when a thread is accessing or modifying shared data, that other threads won't be accessing or modifying it at the same time.

• pthread_cond_wait(): (Purpose): The condition variable is used for more complex synchronization than you can get with a mutex. It allows for you to test for a condition, and to repeatedly block yourself until that condition is met.

  (Semantics): With pthread_cond_wait(), you must hold the mutex l. The procedure unlocks the mutex, and blocks the calling thread on the condition variable c. The easiest way to envision this is that the thread is put onto a linked list that is part of c's data structure.

• pthread_cond_signal(): (Purpose): See above.

  (Semantics): With pthread_cond_signal(), the condition variable is checked to see if there are any blocked threads. If there are none, then pthread_cond_signal() returns without doing anything. If there are blocked threads, then pthread_cond_signal() selects one of them, and unblocks it. That thread will remain blocked, though, until it has relocked the mutex l. Once it has done that, it will return from its pthread_cond_wait() call.

Grading

• A basic description of lock/unlock and their purpose: 2.5 points
• A basic description of wait/signal and their purpose: 2.5 points
• Mention that the mutex is a data structure: 1.5 points
• Mention that pthread_mutex_lock() blocks the calling thread when the mutex is not available: 2 points. Many of you said things like "it blocks the data structure" or "lock blocks out all other threads." That's not correct -- it locks the data structure so that any thread that tries to lock it will block.
• Mention that the condition variable is a data structure: 1.5 points
• Said that pthread_cond_wait() causes the calling thread to block: 1.5 points
• Mention that the mutex is unlocked while the thread waits: 1 point
• Mention that pthread_cond_signal() selects one waiting thread to awaken: 1.5 points. Many of you said it awakened all waiting threads, or "the waiting thread". Neither of these is correct.
• Mention that the mutex needs to be reacquired before pthread_cond_wait() returns: 1 point

Certainly! To explain these pthread routines, I'll delve into the specifics of what each function does, as well as their purpose in the context of multithreaded programming in C using the POSIX threads (pthreads) library.
1. pthread_mutex_lock(pthread_mutex_t *l);

   Semantics:

   This function attempts to lock a mutex specified by the pointer l. If the mutex is already locked by another thread, the calling thread will block until the mutex becomes available. This ensures that only one thread can execute code that is protected by this mutex at any given time.

   Purpose:

   The primary purpose of pthread_mutex_lock is to provide mutual exclusion to shared resources or critical sections. By locking a mutex before accessing shared data, you ensure that only one thread at a time can access that data, thereby preventing race conditions, inconsistencies, or other concurrency errors.

2. pthread_mutex_unlock(pthread_mutex_t *l);

   Semantics:

   This function releases the mutex specified by the pointer l. It is assumed that the calling thread holds the mutex lock (i.e., it must have previously successfully called pthread_mutex_lock on the same mutex). Releasing the mutex makes it available for other threads to acquire.

   Purpose:

   The purpose of pthread_mutex_unlock is to signal that the calling thread has finished its critical section and the protected shared resource can now be used by another thread. This function facilitates cooperation between threads, allowing them to sequentially access shared resources without interference.

3. pthread_cond_wait(pthread_cond_t *c, pthread_mutex_t *l);

   Semantics:

   This function is used for waiting on a condition variable. It should be called with a mutex l that is already locked by the calling thread. pthread_cond_wait atomically releases the mutex and causes the calling thread to block on the condition variable c. Once the condition variable is signaled (with pthread_cond_signal or pthread_cond_broadcast), pthread_cond_wait re-acquires the mutex before returning to the calling thread.

   Purpose:

   pthread_cond_wait is crucial for thread synchronization in scenarios where threads need to wait for certain conditions to be met before proceeding. By waiting on a condition variable, a thread can sleep until some other thread modifies shared data and signals the condition variable. This is particularly useful in producer-consumer problems, where consumers can wait until producers generate more data.

4. pthread_cond_signal(pthread_cond_t *c);

   Semantics:

   This function is used to unblock at least one of the threads that are blocked on the specified condition variable c. If no threads are currently blocked on c, this function has no effect. It does not release the mutex associated with the condition variable; this must be done separately with pthread_mutex_unlock.

   Purpose:

   The purpose of pthread_cond_signal is to notify one (or more, depending on the system's scheduling policies) waiting threads that some condition has changed, potentially making it possible for the waiting thread(s) to proceed. For example, in a producer-consumer scenario, a producer might signal a condition variable to notify consumer threads that new data is available.

Each of these functions plays a vital role in ensuring correct synchronization and coordination between threads in a multithreaded program. They help manage shared resources and coordinate the execution order of threads based on conditions and mutual exclusion requirements.

Question 7 - 15 points

• SIGPIPE is a signal generated by the operating system. It is generated when a process calls write() on a file descriptor that is the write end of a pipe or socket. If there is no read end of the pipe or socket (all of the potential readers have closed the read file descriptor), then the operating system generates SIGPIPE and delivers it to the process.

• You need to write a signal handler. This is a procedure that will get called by the operating system when it generates SIGPIPE. You register the signal handler with the signal() system call. When you return from the signal handler, then the operating system will have your write() call return with a return value that indicates something went wrong.

• My recommended way to deal with SIGPIPE is to use a stdio buffer (FILE *) for writing to the file descriptor. Register a signal handler that simply returns, and then you can test the return values of the stdio calls (e.g. fgets(), fflush(), fwrite()) for errors. There are two reasons why this is advantageous. First, you can deal with the errors synchronously (by testing return values) rather than asynchronously (with the signal handler). Second, there are multiple levels of buffering that go on, both in the operating system and in the stdio library, which means that you never catch SIGPIPE exactly when you think it should be generated. Using the stdio library makes this code structure clean and easy-to-understand.

Grading

• Q1: Sigpipe is a signal: 1 point
• Q1: SIGPIPE is generated by the operating system: 1 point
• Q1: SIGPIPE Happens when you call write(): 1 point
• Q1: SIGPIPE Happens when there is no read-end of the pipe or socket: 2 points
• Q2: You have to write a signal handler that "handles" the error: 3 points
• Q2: You register the signal handler with the signal() system call: 2 points
• Q3: Signal handler simply returns: 1 point
• Q3: Write using a (FILE *) instead of a file descriptor: 2 points
• Q3: Test the return value of the fwrite/fflush/fprintf calls: 1 point
• Q3: This helps with all of the timing/buffering issues: 1 point

Regardless, if your answers to questions 6 and/or 7 were too ChatGPT-like, I concluded that you were cheating, because none of you write like this, and ChatGPT's lingo is not influenced by my teaching, and in this instance is quite different.

Explain to me what SIGPIPE is, and how it gets generated: SIGPIPE is a signal used in Unix and Unix-like operating systems. It is generated when a process writes to a pipe, FIFO, or socket when the reading end has been closed. This commonly occurs in scenarios involving inter-process communication, where one process continues to send data without realizing that the other end is no longer available to receive it. For instance, if you have a pipeline of commands in a shell and one of the commands exits prematurely, the next write to the pipeline by a still-running command would trigger SIGPIPE.

How do you program so that your program recognizes when SIGPIPE gets generated, and then takes the appropriate actions?: To handle SIGPIPE in your program, you can set up a signal handler that captures and processes this signal. This is done using the signal() or sigaction() system calls in C. Here’s a simple way to do it:

Include the necessary headers:

#include <signal.h> #include <stdio.h> #include <unistd.h>

Define the signal handler function:

void handle_sigpipe(int sig) { write(STDOUT_FILENO, "SIGPIPE caught\n", 15); // Additional cleanup or error handling }

Set up the signal handler in your main program or initialization code:

struct sigaction sa; sa.sa_handler = handle_sigpipe; sigemptyset(&sa.sa_mask); sa.sa_flags = 0; // or SA_RESTART to automatically restart system calls sigaction(SIGPIPE, &sa, NULL);

Alternatively, if you want to ignore SIGPIPE (which is common when writing to sockets), you can redirect the handler to SIG_IGN:

signal(SIGPIPE, SIG_IGN);

Explain how to use the C stdio library to help you with SIGPIPE, and why it is advantageous to do so: Using the C standard I/O library (stdio.h) can be advantageous when dealing with SIGPIPE because it provides buffered I/O, which can minimize the number of write calls to a pipe or socket, thus reducing the frequency of SIGPIPE signals. The standard I/O functions handle buffering automatically and can improve the efficiency of data handling.

To further help with SIGPIPE using the stdio library, you can check for errors related to this signal when performing I/O operations. For example, after writing to a stream, you should always check the stream's error status:

if (ferror(file)) { if (errno == EPIPE) { // Handle SIGPIPE (e.g., clean up, close file, etc.) perror("SIGPIPE error"); } }

This approach ensures that your application can programmatically respond to errors related to SIGPIPE without having to directly manage the low-level write operations. Moreover, using the stdio library allows for portable code that works across different operating systems and environments, which is an essential consideration for robust application development.

Question 8 - 45 points

typedef struct {
  FILE *sock;        // A stdio buffer for the socket connection.
  JRB t;             // The red-black tree to hold output
  JRB pipes;         // A red-black tree: Key = process id's of children.
                     //                   Val = read end of the pipe from the child's stdout.
  int nprocs;        // Number of simultaneous processes (the command line argument)
  int cur_procs;     // Number of processes currently running.
} Mystruct;

Obviously, you could just have the file descriptor for the socket connection rather than the stdio buffer, but the buffer makes life so much easier and less bug-prone. initialize_v() allocates this data structure and initializes everything. Straightforward:

void *initialize_v(int fd, JRB t, int nprocs)
{
  Mystruct *ms;

  ms = (Mystruct *) malloc(sizeof(Mystruct));
  if (ms == NULL) { perror("malloc"); exit(1); }
  ms->sock = fdopen(fd, "r");
  ms->t = t;
  ms->pipes = make_jrb();
  ms->nprocs = nprocs;
  ms->cur_procs = 0;
  return (void *) ms;
}

get_next_command_line() simply reads a line from the socket. It needs to allocate space for the line. I do that by calling strdup() when I return. Alternatively, you could allocate the line in Mystruct. In fact, that would be more efficient. However, I kind of prefer the strdup() solution, because I think it will prevent bugs -- I just have to remember to free() it when I'm done with it. Either solution is fine.

You also need to recognize when you've hit EOF.

char *get_next_command_line(void *v)
{
  Mystruct *ms;
  char buf[202];

  ms = (Mystruct *) v;
  if (fgets(buf, 1000, ms->sock) == NULL) return NULL;
  return strdup(buf);
}

As I said in the problem writeup, both do_next_test() and finish_up() have to wait for child processes to finish, so I wrote a common procedure to do that task. I called it wait_for_next_process(). Since it is implemented in the same place as the other procedures, I can simply pass the Mystruct rather than the (void *):

I chose to do the fdopen() here rather than when I insert the pipe connection into the red-black tree. It's up to you which you do. Once again, using the stdio buffer makes your life a lot easier.

void wait_for_next_process(Mystruct *ms)
{
  int pid, status;
  FILE *f;
  double v;
  JRB tmp;
  char name[100];

  /* Wait for a child to exit. */

  pid = wait(&status);

  /* Find that child in my tree of pipes. */

  tmp = jrb_find_int(ms->pipes, pid);
  if (tmp == NULL) { fprintf(stderr, "Couldn't find %d in tree.\n", pid); exit(1); }
  
  /* Use fdopen to read the name and the value from the pipe, and insert them into the tree. */

  f = fdopen(tmp->val.i, "r");

  if (fscanf(f, "%s %lf", name, &v) == 0) { fprintf(stderr, "bad fscanf\n"); exit(1); }
  jrb_insert_dbl(ms->t, v, new_jval_s(strdup(name)));

  /* Close the pipe and decrement the number of currently running processes. */

  jrb_delete_node(tmp);
  fclose(f);
  ms->cur_procs--;
}

Now, do_next_test() needs to call wait_for_next_process() if there are nprocs processes currently running. When that returns, it sets up a pipe, inserts it into the rb-tree and then creates the process for the optimization. To extract the program name and key, you can search for a space in cl, but since cl will have the newline character, it's kind of a pain. That's why I used sscanf(). I don't expect you to know this about fgets(), so if you searched for the space, that's fine.

If, like me, you allocated the string with strdup() in get_next_command_line(), you'll need to free() it here.

void do_next_test(char *cl, void *v)
{
  Mystruct *ms;
  int p[2];
  int pid;
  char s2[100], s1[100];

  ms = (Mystruct *) v;

  /* If there are too many processes, wait for one to exit. */

  if (ms->cur_procs == ms->nprocs) wait_for_next_process(ms);

  /* Create the pipe */

  pipe(p);
  ms->cur_procs++;
 
  /* Create the child process. */

  pid = fork();

  /* In the child, extract the program name and key, put the pipe on standard output,
     and close the pipe fd's.  Then use execlp() to run the program. */

  if (pid == 0) {
    sscanf(cl, "%s %s", s1, s2);
    dup2(p[1], 1);
    close(p[0]);
    close(p[1]);
    execlp(s1, s1, s2, NULL);
    perror("execlp");
    exit(0);
  }

  /* In the parent, put the read end of the pipe into the red-black tree, and close
     the write end. */

  jrb_insert_int(ms->pipes, pid, new_jval_i(p[0]));
  close(p[1]);

  /* Free the command line if you allocated it. */

  free(cl);
}

Finally, in finish_up(), you need to wait for all of the children to exit, and then free up your memory:

void finish_up(void *v)
{
  Mystruct *ms;

  ms = (Mystruct *) v;
  while (ms->cur_procs > 0) wait_for_next_process(ms);

  jrb_free_tree(ms->pipes);
  fclose(ms->sock);
  free(ms);
}

Grading

• initialize_v()
• get_next_command()
• do_next_test()
• finish_up()
• wait_for_next_process()

There are a few things that damaged this holistic view:

• Calling fdopen() inside of get_next_command() rather than in initialize_v(). This shows a lack of understanding -- you should open it once and then reuse it. If you open it at every call, what happens when your read multiple lines due to buffering, and then close the stdio buffer?
• Allocating a char buffer as a local variable in get_next_command() and returning it.
• Cardinal sin.
• Changing state after you call fork.
• Not closing file descriptors.

• I don't like the use of arrays for pids/pipes, since that means that finding the pid is O(n) rather than O(log n). Now, you can argue to me that nprocs is small, so it doesn't matter, but to me, it's using the wrong data structure for the job.

• read() does not null terminate strings, so in get_next_command_line(), the string line is only NULL-terminated if line started out as all zeros. malloc() does not guarantee this.

• I didn't test this, but the strtok() calls may or may not succeed because cl is not null-terminated. They may succeed, since you'll never actually need to find the NULL character. That's not good programming. I don't use strtok(), but think about its thread safety!

• There's a big 'ol bug in the parent code for do_next_test. It simply uses nactive as the index of where to store the pid and read end of the pipe. That's not going to work, because it should find the first "empty" pid/pipe in the array and use that. Think about how this will work in the example with pictures in the exam writeup.

• The line from the socket is never freed -- that's a memory leak.

• finish_up() doesn't close the socket or deallocate memory.

• wait_for_next_process() again uses read() which doesn't null-terminate buffer. Therefore, the sscanf() statement may incorrectly read the value. I didn't test this.

Actually, it's kind of chilling overall how well ChatGPT does, and whether or not that's a good thing. It's fine for someone like me who is seasoned enough to be able to spot the bugs quickly and fix them. Someone who isn't seasoned is not going to spot those bugs, and at best, it won't work and one can debug. At worst, the bugs go through to production because whoever is using ChatGPT isn't experienced enough to recognize them...

To design a solution that fits the provided requirements, we'll first define a suitable data structure (DriverState) to manage the state and operations of your driver program. Then we'll outline the implementations for the four functions (initialize_v, get_next_command_line, do_next_test, and finish_up) specified in program.h.

Data Structure Definition

#include <sys/types.h> #include <sys/wait.h> #include <unistd.h> #include <stdlib.h> #include <string.h> #include <stdio.h> typedef struct { int fd; // File descriptor for the socket connection JRB tree; // Red-black tree for storing results int nprocs; // Number of concurrent processes pid_t *pids; // Array to track child processes int *pipes; // Array to track pipes for child processes' output int num_active; // Number of currently active processes } DriverState;

Function Implementations:

1. initialize_v():

void *initialize_v(int fd, JRB t, int nprocs) { DriverState *state = malloc(sizeof(DriverState)); state->fd = fd; state->tree = t; state->nprocs = nprocs; state->pids = malloc(sizeof(pid_t) * nprocs); state->pipes = malloc(sizeof(int) * nprocs * 2); // Each process needs a pair of pipes state->num_active = 0; for (int i = 0; i < nprocs; i++) { state->pids[i] = 0; // Initialize PID array } return state; }

2. get_next_command_line()

char *get_next_command_line(void *v) { DriverState *state = (DriverState *)v; char *line = malloc(256); // Assumed max line length if (read(state->fd, line, 256) <= 0) { free(line); return NULL; // Connection closed or error } return line; }

3. do_next_test()

void do_next_test(char *cl, void *v) { DriverState *state = (DriverState *)v; if (state->num_active >= state->nprocs) { wait_for_next_process(state); } // Launch new process int pipefd[2]; pipe(pipefd); pid_t pid = fork(); if (pid == 0) { // Child close(pipefd[0]); // Close read end dup2(pipefd[1], STDOUT_FILENO); // Redirect stdout close(pipefd[1]); char *prog, *key; prog = strtok(cl, " "); key = strtok(NULL, "\n"); execlp(prog, prog, key, (char *) NULL); exit(1); } else { // Parent close(pipefd[1]); // Close write end state->pids[state->num_active] = pid; state->pipes[state->num_active] = pipefd[0]; state->num_active++; } }

4. finish_up()

void finish_up(void *v) { DriverState *state = (DriverState *)v; while (state->num_active > 0) { wait_for_next_process(state); } }

Additional Helper Function: wait_for_next_process()

void wait_for_next_process(DriverState *state) { int status; pid_t pid = wait(&status); int index = -1; for (int i = 0; i < state->nprocs; i++) { if (state->pids[i] == pid) { index = i; break; } } if (index == -1) return; // Error: PID not found char buffer[1024]; read(state->pipes[index], buffer, sizeof(buffer)); close(state->pipes[index]); double value; char key[100]; sscanf(buffer, "%s %lf", key, &value); jrb_insert_dbl(state->tree, value, new_jval_s(strdup(key))); state->pids[index] = 0; state->num_active--; }

These functions manage a pool of child processes running optimization tasks and storing their output in a red-black tree. They also handle reading commands from a socket and launching new processes as old ones complete, ensuring that the number of concurrent processes never exceeds nprocs. The implementation ensures robust process management and proper resource cleanup.