CS360 Lecture notes -- Memory

  • James S. Plank
  • Directory: /home/plank/cs360/notes/Memory
  • Lecture notes: http://web.eecs.utk.edu/~plank/plank/classes/cs360/360/notes/Memory/lecture.html
  • Original Lecture Notes: Mid 1990's.
  • Most recent update: March, 2018

    Before I Start

    (This was written in 2018.) This lecture material can be problematic, because memory layout, and the primitives used to explore them are typically not POSIX standards, and can differ from operating system to operating system. Fortunately, the concepts are easy to map from one machine to another, but I can understand your frustration if things become confusing on a new machine.

    Fortunately, Raspberry Pi's using the Raspbian operating system are pretty clean, 32-bit machines. The output that you see below will be on a Pi, so if you have access to one and want to follow along, then please do so. Most Linux OS's will come close to what you see here, so long as you are ready for 64-bit pointers. If you compile these programs with -32 on a 64-bit Linux machine, you'll get nicer looking pointers.

    (Also, if you compile these with a strict compiler, you'll get warnings. Sorry, but I don't want to junk up my code with the proper typecasts to get rid of the warnings. They don't happen on the Pi).


    Memory

    This lecture is an introduction to memory as we see it in Unix.

    As I have said previously, memory is like a huge array with 232 or 264 elements, depending on whether you are running in 32 or 64-bit mode. A pointer in C is an index to this array. Thus when a C pointer is 0xefffe034, it points to the 0xefffe035th element in the memory array (memory being indexed starting with zero).

    You cannot access all elements of memory. One example that we have seen a lot is element 0. If you try to dereference a pointer with a value of 0, you will get a segmentation violation. This is Unix's way of telling you that that memory location is illegal.

    For example, the following program (segfault.c) will generate a segmentation violation.

    int main()
    {
      char *s;
      char c;
    
      s = (char *) 0;
      c = *s;           /* The segmentation violation happens here. */
      return 0;
    }
    

    UNIX> ./segfault
    Segmentation fault
    UNIX> 
    
    There are many regions of memory that are set up to be legal, when you run your program. They are set up by the operating system, with the help of the hardware. I will talk about four of them here. While there are standard Unix names for them (which I'll tell you), I think that they are confusing, so I use my own:
    1. The code: This memory region holds the instructions of your program. The standard Unix name for this is "text".
    2. The globals: These are your global variables. Standard Unix lingo splits the globals into two parts: "Data", which holds global variables that have been initialized in the program; and "BSS", which holds global variables that have not been initialized.
    3. The heap: This is memory that you get from malloc() (or new in C++).
    4. The stack: This contains your local variables and procedure arguments.
    If we view memory as a big array, the regions (or ``segments'') look as follows:
         |--------------| 0x00000000
         |              |
         |   void       |
         |              |
         |--------------| 
         |              |
         |  code        |
         |              |
         |--------------|
         |  void        |
         |--------------| 
         |              |
         |  globals     |
         |              |
         |--------------|
         |  void        |
         |--------------|
         |              |
         |  heap        |
         |              |  You can make the heap grow with the sbrk() or mmap() system call.
         |--------------|
         |  void        |
         |--------------|
         |              |
         |  stack       |
         |              | 
         |--------------|
         |  void        |
         |--------------| 0xffffffff
    
    Note, the heap grows down as you make more malloc() calls and your program asks the operating system for more memory. As we have seen in the Assembly code lecture notes, the stack grows "upward", by subtracting values from the stack and frame pointers. In reality, the stack is a fixed size (typically 8 MB). You could make it grow by using mmap() properly, but that is not what happens by default.

    Paging

    On all machines, memory is broken up into fixed chunks called pages. Pages are typically 4096 or 8192 bytes long, but other sizes have been supported by other machines. You can find out your machine's page size by calling the getpagesize() system call. On the Pi, they are 4096 bytes (we will see this below).

    The way memory works is as follows: The operating system allocates certain pages of memory for you. Whenever you try to read, write or execute an instruction from an address in memory, the hardware first checks with the operating system to see if that address belongs to a page that has been allocated for you, and if you have permission to do the operation. If so, then it goes ahead and performs the read/write. If not, you'll get a segmentation violation, which is a hardware error. It is caught by the operating system, which in turn "sends" it to your program (we'll see how you can "catch" it later in the semester).

    This is what happens when you do:

      s = (char *) 0;
      c = *s;
    
    When you say "c = *s", the hardware sees that you want to read memory location zero. The hardware checks a table, which has been set up by the operating system, to see if location zero is legal, and discovers that it is not. This results in a segmentation violation.

    The exact mechanics of paging are covered in classes on Operating Systems. I won't go into them further here. What you should understand is that the hardware and operating system manage memory at a page granularity, and that when you want to use memory, to read it, write it, or execute instructions from it, the hardware checks the page on which the memory resides, to make sure you have the permissions to use it. If you don't, you'll get a segmentation violation.


    Looking at Memory Regions of your Program

    The program look_at_memory.c takes a look at the various memory regions of your program, while it is running.

    #include <sys/types.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    
    typedef unsigned long int UI;
    
    /* These are special variables that
       denote the end of the memory segments
       for the code ("text"), initialized
       global variables ("data") and
       uninitialized global variables ("bss").
       */
    
    extern end;
    extern etext;
    extern edata;
    
    /* Here are four global variables.  A
       and B belong to the "data", and X and Y
       belong to the "bss".  */
    
    int A = 4;
    int X;
    int B = 6;
    int Y;
    
    /* Proc_a and main should reside in the
       "text" segment. */
    
    void proc_a()
    {
    }
    
    /* And of course i and buf will belong
       on the stack.  I will call malloc()
       to set buf equal to an address in 
       the heap. */
    
    int main(int argc, char **argv)
    {
      int i;
      char *buf;
    
      buf = (char *) malloc(200);
    
      printf("Page size: %d\n", getpagesize());
      printf("\n");
    
      printf("&etext: 0x%lx\n", (UI) &etext);
      printf("&edata: 0x%lx\n", (UI) &edata);
      printf("&end:   0x%lx\n", (UI) &end);
      printf("\n");
    
      printf("Code Addresses:\n");
      printf("main:   0x%lx\n", (UI) main);
      printf("proc_a: 0x%lx\n", (UI) proc_a);
      printf("\n");
    
      printf("Global Variable Addresses:\n");
      printf("&A: 0x%lx\n", (UI) &A);
      printf("&B: 0x%lx\n", (UI) &B);
      printf("&X: 0x%lx\n", (UI) &X);
      printf("&Y: 0x%lx\n", (UI) &Y);
      printf("\n");
    
      printf("Heap Address:\n");
      printf("buf: 0x%lx\n", (UI) buf);
      printf("\n");
    
      printf("Stack Addresses:\n");
      printf("&i:    0x%lx\n", (UI) &i);
      printf("&buf:  0x%lx\n", (UI) &buf);
      printf("&argc: 0x%lx\n", (UI) &argc);
      printf("\n");
    
      /* Finally, print the addressses as 
         recorded in the directory "/proc". */
    
      sprintf(buf, "cat /proc/%d/maps", getpid());
      system(buf);
      return 0;
    }
    

    Again, not all machines will allow this code to compile. Linux-based machines are a pretty safe bet, though. Go ahead and read the comments to the code above. Basically, we are printing out addresses of variables in each of the memory segments that I describe above. After that, we look at the file /proc/pid/maps, which lets you know about the memory mappings in the process with id pid.

    We run the program on a Raspberry Pi, and we get a lot of information:

    UNIX> ./look_at_memory
    Page size: 4096
    
    &etext: 0x107f4
    &edata: 0x20a74
    &end:   0x20a80
    
    Code Addresses:
    main:   0x1058c
    proc_a: 0x10578
    
    Global Variable Addresses:
    &A: 0x20a6c
    &B: 0x20a70
    &X: 0x20a78
    &Y: 0x20a7c
    
    Heap Address:
    buf: 0x12cb008
    
    Stack Addresses:
    &i:    0x7ebe3b3c
    &buf:  0x7ebe3b38
    &argc: 0x7ebe3b34
    
    00010000-00011000 r-xp 00000000 00:22 126617690  /mnt/nfs/plank/cs360-lecture-notes/Memory/look_at_memory
    00020000-00021000 rw-p 00000000 00:22 126617690  /mnt/nfs/plank/cs360-lecture-notes/Memory/look_at_memory
    012cb000-012ec000 rw-p 00000000 00:00 0          [heap]
    76d9c000-76ec7000 r-xp 00000000 b3:07 524518     /lib/arm-linux-gnueabihf/libc-2.19.so
    76ec7000-76ed7000 ---p 0012b000 b3:07 524518     /lib/arm-linux-gnueabihf/libc-2.19.so
    76ed7000-76ed9000 r--p 0012b000 b3:07 524518     /lib/arm-linux-gnueabihf/libc-2.19.so
    76ed9000-76eda000 rw-p 0012d000 b3:07 524518     /lib/arm-linux-gnueabihf/libc-2.19.so
    76eda000-76edd000 rw-p 00000000 00:00 0 
    76ef6000-76efb000 r-xp 00000000 b3:07 3540510    /usr/lib/arm-linux-gnueabihf/libarmmem.so
    76efb000-76f0a000 ---p 00005000 b3:07 3540510    /usr/lib/arm-linux-gnueabihf/libarmmem.so
    76f0a000-76f0b000 rw-p 00004000 b3:07 3540510    /usr/lib/arm-linux-gnueabihf/libarmmem.so
    76f0b000-76f2b000 r-xp 00000000 b3:07 524429     /lib/arm-linux-gnueabihf/ld-2.19.so
    76f35000-76f3a000 rw-p 00000000 00:00 0 
    76f3a000-76f3b000 r--p 0001f000 b3:07 524429     /lib/arm-linux-gnueabihf/ld-2.19.so
    76f3b000-76f3c000 rw-p 00020000 b3:07 524429     /lib/arm-linux-gnueabihf/ld-2.19.so
    7ebc3000-7ebe4000 rwxp 00000000 00:00 0          [stack]
    7ec2a000-7ec2b000 r-xp 00000000 00:00 0          [sigpage]
    7ec2b000-7ec2c000 r--p 00000000 00:00 0          [vvar]
    7ec2c000-7ec2d000 r-xp 00000000 00:00 0          [vdso]
    ffff0000-ffff1000 r-xp 00000000 00:00 0          [vectors]
    UNIX> 
    
    From this program, we can deduce that our program's address space looks as in the picture below:

    You'll note that every memory address from 0x0 to 0xffffffff is accounted for. You'll also note how the printf() statements from the program match the picture. For example, A and B have values between 0x20000 (the beginning of the globals) and 0x20ffff (the end of the globals). They also have values lower than &edata, which is right, because they are initialized by the program.

    X and Y also have values between 0x20000 and 0x20ffff. They are uninitialized, so they belong in the "bss" segment. Accordingly, their values are between &edata and &end.

    You should also see that main and proc_a are in the code segment, buf is in the heap, and &i, &buf and &argc are all in the stack segment.

    All of the addresses labeled NULL will give you segmentation violations if you try to access them.

    It is slightly unfortunate that the heap addresses and stack addresses will change from run to run:

    UNIX> ./look_at_memory | grep buf
    buf: 0x53b008
    &buf:  0x7e98eb38
    UNIX> ./look_at_memory | grep buf
    buf: 0x1274008
    &buf:  0x7efc4b38
    UNIX> ./look_at_memory | grep buf
    buf: 0x19df008
    &buf:  0x7eec7b38
    UNIX> 
    
    The reason for this is security. Be ready for it.

    Reading, Writing and Executing Memory Regions

    The program mess_with_memory.c changes look_at_memory.c in a few minor ways. First, it defines the procedure proc_a to simply print "Hi":

    void proc_a()
    {
      printf("Hi\n");
    }
    

    Next, it defines three more variables in main(). These are ptr, c and :

      char *ptr;
      char c;
      void (*proc)();
    

    The only subtle one is proc -- it is a pointer to a function, which you can set and execute at runtime.

    At the end of mess_with_memory, we do some dangerous stuff. In fact, if you have ever heard someone say that C is a dangerous or unsafe language, well here's your proof! Read the comments to see what the code is doing.

    
      /* All the previous code from look_at_memory.c is here. */
    
      i = 0xabcd;
    
      while (1) {
        printf("\n");
        printf("Enter an address (with 0x), and R|W|X: ");
        fflush(stdout);
      
        /* From stdin, read a pointer value into ptr, and a character R, W, or X into c. */
    
        if (scanf("0x%x %c", &ptr, &c) != 2 || strchr("RWX", c) == NULL) {
          printf("Exiting\n");
          exit(0);
        }
      
        /* If c is 'R', read the character at the pointer. */
    
        if (c == 'R') {
          printf("Reading 0x%x\n", (UI) ptr);
          fflush(stdout);
          c = *ptr;
          printf("Read 0x%x\n", c); 
      
        /* If c is 'W', write the value 0x67 to the byte at the pointer, 
           and then print out A, i and buf. */
    
        } else if (c == 'W') {
          printf("Writing 0x67 to 0x%x\n", ptr);
          *ptr = 0x67;
          printf("A is now 0x%x\n", A);
          printf("i is now 0x%x\n", i);
          printf("buf is now %s\n", buf);
    
        /* If c is 'X', then treat ptr as a procedure, and call it. */
    
        } else if (c == 'X') {
          memcpy(&proc, &ptr, sizeof(void *));
          proc();
        }
      }
    
      return 0;
    }
    

    So, what the program is allowing the user to do is enter an address, and then either read from, write to, or execute code from that address. Let's give it a try. It is unfortunate that the memory addresses will change from when we called look_at_memory above. I'm only going to include output that is important to understanding this part of the lecture:

    UNIX> ./mess_with_memory
    &etext: 0x10b3c    I'm deleting a bunch of lines here, to make it clearer
    &edata: 0x20e80
    &end:   0x20e94
    
    Code Addresses:
    main:   0x1072c
    proc_a: 0x10714
    
    Global Variable Addresses:
    &A: 0x20e78
    
    Heap Address:
    buf: 0x1685008
    
    Stack Addresses:
    &i:    0x7eadeb3c
    
    00010000-00011000 r-xp 00000000 00:22 126617872  /mnt/nfs/plank/cs360-lecture-notes/Memory/mess_with_memory
    00020000-00021000 rw-p 00000000 00:22 126617872  /mnt/nfs/plank/cs360-lecture-notes/Memory/mess_with_memory
    01685000-016a6000 rw-p 00000000 00:00 0          [heap]
    7eabe000-7eadf000 rwxp 00000000 00:00 0          [stack]
    
    Enter an address (with 0x), and R|W|X: 0x20e78 R
    Reading 0x20e78
    Read 0x4
    
    Enter an address (with 0x), and R|W|X: 0x7eadeb3c R
    Reading 0x7eadeb3c
    Read 0xcd
    
    Enter an address (with 0x), and R|W|X: 0x7eadeb3d R
    Reading 0x7eadeb3d
    Read 0xab
    
    Enter an address (with 0x), and R|W|X: 0x1685008 R
    Reading 0x1685008
    Read 0x30
    
    All of these are straightforward -- we are reading the first byte of A, which, because the Raspberry Pi is little endian, equals 4. We set i to be 0xabcd, so its first byte is 0xcd and its second byte is 0xab. Finally, 0x1685008 is the value of buf, which is a heap address. When we read it, we get 0x30, which is the ASCII character code for '0', which is the first character in the string that we just read from the terminal.

    We can write to these addresses as well -- the 'W' character says to write 0x67 to the address. Below, we'll write it to A, to the third byte of i, and to buf:

    Enter an address (with 0x), and R|W|X: 0x20e78 W
    Writing 0x67 to 0x20e78
    A is now 0x67            We just wrote 0x67 to the first byte of A
    i is now 0xabcd
    buf is now 0x20e78 W
    
    Enter an address (with 0x), and R|W|X: 0x7eadeb3e W
    Writing 0x67 to 0x7eadeb3e
    A is now 0x67
    i is now 0x67abcd        We just wrote 0x67 to the third byte of i
    buf is now 0x7eadeb3e W
    
    Enter an address (with 0x), and R|W|X: 0x1685008 W
    Writing 0x67 to 0x1685008
    A is now 0x67
    i is now 0x67abcd
    buf is now gx1685008 W    We just wrote 0x67 ('g') to the first byte of buf
    
    Recall that memory is partitioned by the processor into pages, and it manages the protection of memory at the page level. So, even though &end is 0x20e94, we see from the /proc/pid/maps file that all memory up to 0x20fff is part of the global variable segment. So, we can read and write 0x20fff without segfaulting, even though that doesn't have any meaning to our program:
    Enter an address (with 0x), and R|W|X: 0x20fff W
    Writing 0x67 to 0x20fff
    Ignore the output
    
    Enter an address (with 0x), and R|W|X: 0x20fff R
    Reading 0x20fff
    Read 0x67
    
    However, if we try to read 0x21000, we will segfault:
    Enter an address (with 0x), and R|W|X: 0x21000 R
    Reading 0x21000
    Segmentation fault
    UNIX> 
    
    We can read an execute memory in the code segment. That seems pretty dangerous, doesn't it? Let's go ahead and read the first bytes of main() and proc_a:
    UNIX> ./mess_with_memory
    &etext: 0x10b3c
    &edata: 0x20e80
    &end:   0x20e94
    
    Code Addresses:
    main:   0x1072c
    proc_a: 0x10714
    
    Global Variable Addresses:
    &A: 0x20e78
    &B: 0x20e7c
    &X: 0x20e8c
    &Y: 0x20e90
    
    Heap Address:
    buf: 0x824008
    
    Stack Addresses:
    &i:    0x7ec4bb3c
    &buf:  0x7ec4bb38
    &argc: 0x7ec4bb24
    
    I'm omiting the maps
    
    Enter an address (with 0x), and R|W|X: 0x1072c R
    Reading 0x1072c
    Read 0x10
    
    Enter an address (with 0x), and R|W|X: 0x10714 R
    Reading 0x10714
    Read 0x0
    
    That's not very exciting. Now, let's set the function pointer proc to 0x10714, which is the first instruction of proc_a, and call proc. You'll see that it calls proc_a:
    Enter an address (with 0x), and R|W|X: 0x10714 X
    Hi
    
    Can we call main()? Why not!
    Enter an address (with 0x), and R|W|X: 0x1072c X
    The first lines, up to the heap addresses are the same as before.
    I'm not including them here.
    Heap Address:
    buf: 0x8240d8
    
    Stack Addresses:
    &i:    0x7ec4bb0c
    &buf:  0x7ec4bb08
    &argc: 0x7ec4baf4
    
    Since we're calling main() recursively, you'll see that buf has a new address, since we called malloc() again, and that the stack addresses are smaller than they were previously.

    How about if we call it on a heap address? We will get a segmentation violation, because the heap does not have execute permissions:

    Enter an address (with 0x), and R|W|X: 0x824008 X
    Segmentation fault
    
    An observant student in class noted that the stack did have execute permissions. That doesn't seem prudent. If we try to an address in our current stack frame, we'll get an illegal instruction, since those addresses have not been set up as instructions:
    UNIX> ./mess_with_memory
    Lines deleted
    Stack Addresses:
    &i:    0x7ef84b3c
    &buf:  0x7ef84b38
    &argc: 0x7ef84b24
    More deleted
    
    Enter an address (with 0x), and R|W|X: 0x7ef84b3c X
    Illegal instruction
    UNIX> 
    
    I tried to then put real code onto the stack and run it, but it failed -- probing, it has to do with ARM branch instructions, and I don't really want to go down that rabbit hole. Too bad.

    Breaking the stack

    It's not hard to break the stack -- simply do infinite recursion. The program break_the_stack.c does this.

    #include <stdio.h>
    
    int main(int argc)
    {
      char eight_K[8192];
    
      printf("Argc = %5d.  &argc = 0x%x.\n", argc, &argc);
      main(argc+1);
      return 0;
    }
    

    Typically, the operating system allocates 8MB for the stack, so this program should segfault in around 1024 iterations (less, because you are pushing more onto the stack than the eight_K array):

    UNIX> ./break_the_stack
    Argc =     1.  &argc = 0x7ec79b44.
    Argc =     2.  &argc = 0x7ec77b34.
    Argc =     3.  &argc = 0x7ec75b24.
    Argc =     4.  &argc = 0x7ec73b14.
    ....
    Argc =  1019.  &argc = 0x7e481ba4.
    Argc =  1020.  &argc = 0x7e47fb94.
    Argc =  1021.  &argc = 0x7e47db84.
    Segmentation fault
    UNIX>
    
    You can also break the stack by simply allocating too much local memory. A declaration like the following will do:
       char big[10000000];
    

    What happens when you run out of heap space?

    When this happens, malloc() returns NULL. The program break_the_heap.c keeps calling malloc() for a megabyte, until malloc() fails. On my Raspberry pi, this happens after 2021 iterations, so it looks like my process gets two gigabytes of heap space before the operating system cries uncle...

    int main()
    {
      int i;
      char *c;
    
      for (i = 0; i < 1000000000; i++) {
        c = (char *) malloc(1024*1024);
        printf("%9d 0x%08lx\n", i, (unsigned long) c);
        if (c == NULL) exit(0);
      }
      return 0;
    }
    

    UNIX> ./a.out
            0 0x76d0c008
            1 0x76c0b008
            2 0x76b0a008
            3 0x76a09008
    ...
         2019 0x7eda0008
         2020 0x7eea1008
         2021 0x00000000
    UNIX> 
    
    On my linux box, this went over 700,000 iterations before I killed it...