CS360 Lecture notes -- Memory

  • Jim Plank
  • Directory: /home/plank/cs360/notes/Memory
  • Lecture notes: http://www.cs.utk.edu/~plank/plank/classes/cs360/360/notes/Memory/lecture.html

    Memory

    The machine layout that I describe what that of the machines in the hydra lab in the early 2000's. When you try to go through the programs in this lecture on current machines, you will get different, but explainable results. For example, on the machines described here, the heap starts at &end. On the current (2010) machines, that is not the case, and the heap starts far later. Using the programs in this lecture, you can figure out exactly how your machine lays out memory.

    Also, you should set up your shell so that you don't generate core files when doing this lecture. I.e., if it is not done in your .cshrc file, do:

    UNIX> limit coredumpsize 0
    
    Finally, in the makefile, I have specified -m32 as a compiler option, which forces the machines to use 32-bit pointers. You can remove this and the programs will still run correctly. The memory addresses of the various layouts will be different, but again explainable.
    This lecture is an introduction to memory as we see it in Unix.

    As I have said previously, memory is like a huge array with (say) 0xffffffff elements. A pointer in C is an index to this array. Thus when a C pointer is 0xefffe034, it points to the 0xefffe035th element in the memory array (memory being indexed starting with zero).

    Unfortunately, you cannot access all elements of memory. One example that we have seen a lot is element 0. If you try to dereference a pointer with a value of 0, you will get a segmentation violation. This is Unix's way of telling you that that memory location is illegal.

    For example, the following code will generate a segmentation violation:

    main()
    {
      char *s;
      char c;
    
      s = (char *) 0;
      c = *s;
    }
    
    As it turns out, there are 4 regions of memory that are legal. They are:
    1. The code (or "text"): These are the instructions of your program
    2. The globals: These are your global variables
    3. The heap: This is memory that you get from malloc().
    4. The stack: This contains your local variables and procedure arguments.
    If we view memory as a big array, the regions (or ``segments'') look as follows:
         |--------------| 0
         |              |
         |   void       |
         |              |
         |--------------| 0x10000
         |              |
         |  code        |
         |              |
         |--------------|
         |  void        |
         |--------------| 0x20000
         |              |
         |  globals     |
         |              |
         |--------------|
         |              |
         |  heap        |
         |              |
         ||||||||||||||||
         |vvvvvvvvvvvvvv|
         |              |
         |              |
         |  void        |
         |              |
         |              |
         |^^^^^^^^^^^^^^|
         ||||||||||||||||
         |              |
         |  stack       |
         |              | 0xefffffff
         |--------------|
    
    Note, the heap grows down as you make more malloc() calls, and the stack goes up as you make nested procedure calls.

    Paging

    On the hydra machines, memory is broken up into 8192-byte chunks. These are called pages. On some machines, pages are 4096 bytes -- this is something set by the hardware.

    The way memory works is as follows: The operating system allocates certain pages of memory for you. Whenever you try to read to or write from an address in memory, the hardware first checks with the operating system to see if that address belongs to a page that has been allocated for you. If so, then it goes ahead and performs the read/write. If not, you'll get a segmentation violation.

    This is what happens when you do:

      s = (char *) 0;
      c = *s;
    
    When you say "c = *s", the hardware sees that you want to read memory location zero. It checks with the operating system, which says "I haven't allocated the page containing location zero for you." This results in a segmentation violation.

    The exact mechanics of paging are covered in classes on Operating Systems. I won't go into it further here.

    As it turns out, the first 8 pages on our hydra machines are void. This means that trying to read to or write from any address from 0 to 0xffff will result in a segmentation violation.

    The next page (starting with address 0x10000) starts the code segment. This segment ends at the variable &etext, which I'll go over in a bit. The globals segment starts at page 0x20000. It goes until the variable &end. The heap starts immediately after &end, and goes up to sbrk(0), which I'll talk about still later. The stack ends with address 0xefffffff. Its beginning changes with the different procedure calls you make. We'll go over this more later in this lecture. Every page between the end of the heap and the beginning of the stack is void, and will generate a segmentation violation upon accessing.


    &etext and &end.

    (For more info on these variables, do man etext).

    These are two external variables that are defined as follows:

    extern etext;
    extern end;
    
    Note that they are typeless. You never use just "etext" and "end." Instead, you use their addresses -- these point to the end of the text and globals segments respectively.

    Look at the program testaddr1.c. This prints out the addresses of etext and end. Then it prints out 6 values:

    When we run testaddr1, we get something like the following:

    UNIX> testaddr1
    &etext = 0x11843
    &end   = 0x21b5c
    
    main   = 0x10b50
    &I     = 0x21b54
    &i     = 0xffbee104
    &argc  = 0xffbee15c
    &ii    = 0xffbee100
    ii     = 0x21b70
    UNIX>
    
    So, what this says is that the code segment goes from 0x10000 to 0x11843. The globals segment goes from 0x20000 to 0x21b5c. The heap goes from 0x21b5c to some address greater than 0x21b70 (since ii allocated 4 bytes starting at 0x21b70). The stack goes from some address less than 0xefffe8f8 to 0xefffffff. All values that are printed by testaddr1 make sense.
    Now, look at testaddr2.c.

    This is the first really gross piece of C code that you'll see. What it does is print out &etext and &end, and then prompt the user for an address in hexadecimal. It puts that address into the pointer variable s. You should never do this unless you are writing code like this which is testing memory. The first thing that it does with s is try to read from that memory location (c = *s). Then it tries to write to the memory location (*s = c). This is a way to see which memory locations are legal.

    So, lets try it out with an illegal memory value of zero:

    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x0
    Reading 0x0:  Segmentation Fault
    UNIX> 
    
    When we tried to read from memory location zero, we got a seg fault. This is because memory location zero is in the void -- the hardware recognized this by asking the operating system, and then generating a segmentation violation.

    Memory locations 0x0 to 0xffff are illegal -- if we try any address in that range, we will get a segmentation violation:

    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0xffff
    Reading 0xffff:  Segmentation Fault
    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x4abc
    Reading 0x4abc:  Segmentation Fault
    UNIX> 
    

    Memory location 0x10000 is in the code segment. This should be a legal address:

    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x10000
    Reading 0x10000:  127
    Writing 127 back to  0x10000:  Segmentation Fault
    UNIX> 
    

    You'll note that we were able to read from 0x10000 -- it gave us the byte 127, which begins some instruction in the program. However, we got a seg fault when we wrote to 0x10000. This is by design: The code segment is read-only. You can read from it, but you can't write to it. This makes sense, because you can't change your program while it's running -- instead you have to recompile it, and rerun it.

    Now, what if we try memory location 0x11fff? This is above &etext, so it should be outside of the code segment:

    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x11fff
    Reading 0x11fff:  -48
    Writing -48 back to  0x11fff:  Segmentation Fault
    UNIX>
    
    You'll note that even though 0x11fff is an address outside the code segment, we're still allowed to read from it. This is because the hardware checks the with operating system to see if an address's page has been allocated. Since page 8 (0x10000 - 0x11fff) has been allocated for the code segment, the hardware treats any address between 0x10000 and 0x11fff as a legal address. You can read from it, but its value is meaningless.

    Now, pages 9 to 15 are undreadable again:

    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x12000
    Reading 0x12000:  Segmentation Fault
    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x1f000
    Reading 0x1f000:  Segmentation Fault
    UNIX> 
    

    The globals starts at 0x20000, so we see that the 16th page is readable and writable:

    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x20000
    Reading 0x20000:  127
    Writing 127 back to  0x20000:  ok
    UNIX>          
    

    We can read from and write to any location (0x20000 to 0x21fff) in this page. The next page (starting at 0x22000) is unreachable:

    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x21dff
    Reading 0x21dff:  0
    Writing 0 back to  0x21dff:  ok
    UNIX> testaddr2
    &etext = 0x1191b
    &end   = 0x21d90
    
    Enter memory location in hex (start with 0x): 0x22000
    Reading 0x22000:  Segmentation Fault
    UNIX> 
    

    What this tells us is that the globals go from 0x20000 to 0x21d90. The heap goes from 0x21d90 up to some higher address in the same page.


    Sbrk(0)

    As with other parts of this lecture, sbrk() has grown to obsolescence. Please see the next set of lecture notes on malloc() for some more current information.

    sbrk() is a system call that we will get into in a few lectures. sbrk(0) returns to the user the current end of the heap. Since we can keep calling malloc(), sbrk(0) can change over time. testaddr3.c shows the value of sbrk(0) -- note it is in page 16 (0x20000 - 0x21fff). Since the hardware performs its check in 8192-byte intervals, we can get at any byte in page 16, even though sbrk(0) returns 0x20c78:

    UNIX> testaddr3
    &etext = 0x11993
    &end   = 0x21e18
    sbrk(0)= 0x21e18
    &c     = 0xffbee103
    
    Enter memory location in hex (start with 0x): 0x21fff
    Reading 0x21fff:  0
    Writing 0 back to  0x21fff:  ok
    UNIX> 
    
    We haven't called malloc() in testaddr3.c. This is the reason why &end and sbrk(0) return the same value. In testaddr3a.c we make a malloc() call in the beginning of the program, and as you see, &end and sbrk(0) return different values:
    UNIX> testaddr3a
    &etext = 0x119a3
    &end   = 0x21e28
    sbrk(0)= 0x23e28
    &c     = 0xffbee103
    
    Enter memory location in hex (start with 0x): 0x23fff
    Reading 0x23fff:  0
    Writing 0 back to  0x23fff:  ok
    UNIX> testaddr3a
    &etext = 0x119a3
    &end   = 0x21e28
    sbrk(0)= 0x23e28
    &c     = 0xffbee103
    
    Enter memory location in hex (start with 0x): 0x24000
    Reading 0x24000:  Segmentation Fault
    UNIX> 
    

    The stack

    So, where's the beginning of the stack? If we try addresses above 0xffbee103 in testaddr3, we see that most of them are legal:
    UNIX> testaddr3
    &etext = 0x11993
    &end   = 0x21e18
    sbrk(0)= 0x21e18
    &c     = 0xffbee103
    
    Enter memory location in hex (start with 0x): 0xffb00000
    Reading 0xffb00000:  0
    Writing 0 back to  0xffb00000:  ok
    UNIX> testaddr3
    &etext = 0x11993
    &end   = 0x21e18
    sbrk(0)= 0x21e18
    &c     = 0xffbee103
    
    Enter memory location in hex (start with 0x): 0xff3f0000
    Reading 0xff3f0000:  0
    Writing 0 back to  0xff3f0000:  ok
    UNIX> testaddr3
    &etext = 0x11993
    &end   = 0x21e18
    sbrk(0)= 0x21e18
    &c     = 0xffbee103
    
    Enter memory location in hex (start with 0x): 0xff3effff
    Reading 0xff3effff:  Segmentation Fault
    UNIX> 
    
    What gives? As it turns out, the operating system allocates all pages from 0xff3f0000 to the bottom of the stack. Where is the bottom of the stack? Let's probe:
    UNIX> testaddr3
    &etext = 0x11993
    &end   = 0x21e18
    sbrk(0)= 0x21e18
    &c     = 0xffbee103
    
    Enter memory location in hex (start with 0x): 0xffbeffff
    Reading 0xffbeffff:  0
    Writing 0 back to  0xffbeffff:  ok
    UNIX> testaddr3
    &etext = 0x11993
    &end   = 0x21e18
    sbrk(0)= 0x21e18
    &c     = 0xffbee103
    
    Enter memory location in hex (start with 0x): 0xffbf0000
    Reading 0xffbf0000:  Segmentation Fault
    UNIX>
    
    So the stack goes from 0xff3f0000 to 0xffbeffff. That is roughly 8 megabytes.

    You can print out the default stack size, and change it using the limit command (read the man page):

    UNIX> limit
    ...
    stacksize       8192 kbytes
    ...
    
    Whenever you call a procedure, it allocates local variables and arguments (plus a few other things) on the stack. Whenever you return from a procedure, those varables are popped off the stack. So, look at testaddr4.c. It has main() call itself recursively as many times as there are arguments. You'll see that at each recursive call, the addresses of argc and argv and the local variable i are smaller addresses -- this is because each time the procedure is called, the stack grows upward to allocate its arguments and local variables. You've seen this already in the assembler lectures.
    UNIX> testaddr4
    argc = 1.  &argc = 0xffbee15c, &argv = 0xffbee160, &i = 0xffbee104
    argc = 0.  &argc = 0xffbee0e4, &argv = 0xffbee0e8, &i = 0xffbee08c
    UNIX> testaddr4 v
    argc = 2.  &argc = 0xffbee154, &argv = 0xffbee158, &i = 0xffbee0fc
    argc = 1.  &argc = 0xffbee0dc, &argv = 0xffbee0e0, &i = 0xffbee084
    argc = 0.  &argc = 0xffbee064, &argv = 0xffbee068, &i = 0xffbee00c
    UNIX> testaddr4 v o l s
    argc = 5.  &argc = 0xffbee144, &argv = 0xffbee148, &i = 0xffbee0ec
    argc = 4.  &argc = 0xffbee0cc, &argv = 0xffbee0d0, &i = 0xffbee074
    argc = 3.  &argc = 0xffbee054, &argv = 0xffbee058, &i = 0xffbedffc
    argc = 2.  &argc = 0xffbedfdc, &argv = 0xffbedfe0, &i = 0xffbedf84
    argc = 1.  &argc = 0xffbedf64, &argv = 0xffbedf68, &i = 0xffbedf0c
    argc = 0.  &argc = 0xffbedeec, &argv = 0xffbedef0, &i = 0xffbede94
    UNIX> 
    
    Now, lets break the stack. This can be done by writing a program that allocates too much stack memory. One such program is in breakstack1.c. It performs infinite recursion, and at each recursive step it allocates 10000 bytes of stack memory in the variable iptr. When you run this, you'll see that you get a segmentation violation when the recursive call is made and the stack is about to dip below 0xff3f0000:
    UNIX> breakstack1
    ...
    &c     = 0xff3fa347, iptr = 0xff3f7c30  ...  ok
    &c     = 0xff3f7bbf, iptr = 0xff3f54a8  ...  ok
    &c     = 0xff3f5437, iptr = 0xff3f2d20  ...  ok
    Segmentation Fault
    UNIX> 
    
    Often when you have infinite recursion and overflow the stack, you get "illegal instruction" instead of Segmenation fault. I can't explain this in detail-- just be ready for it...

    The second way to break the stack is to simply allocate too much local memory. E.g. look at breakstack2.c. It tries to allocate 10M of memory in the stack. It segfaults in a because it tries to reference smaller memory addresses than 0xff3f0000. Exactly where does the seg fault happen? Think about it -- answer below.

    The segfault happens in a when the code attempts to push iptr on the stack for the printf call. This is because the stack pointer is pointing to the void. Had we not referenced anything at the stack pointer, our program should have worked. For example, try breakstack3.c.

    UNIX> breakstack3
    Calling a.  i = 1
    After a is done.  i = 5
    UNIX> 
    
    You should understand, and be able to explain this phenomenon.