Memory Management

1. Memory Management How memory is used to hold data used by processes.

2. Memory Issues • How is memory organized? • How are different parts of the program stored in memory? • Where is data stored in memory? • When is memory allocated to variables? code? • When is memory freed? • How can the programmer specify memory sizes? • How can the programmer examine memory?

3. Memory Hierarchy • registers • cache: L1 cache and L2 cache • RAM - main memory • Secondary storage: hard disk or flash memory • Tertiary storage: • tape - 4mm (DAT), 8mm, Digital Linear Tape • DVD or CD -ROM or -RW

4. Memory for Program Execution To run a program, the operating system allocates at least two memory segments for the process: “text segment” containing the machine instructions for the program. • this segment is usually read-only and can be shared my multiple processes running the same program "data segment" to hold values used by a process. • includes variables, constants, unnamed values, and data required to run process, such as saved registers and references to environments • each process requires its own data segment • the data segment is the area most interesting to the programmer.

5. Three parts of a runtime environment The data segment may be divided into 3 parts: Static area for values that exist during entire duration of the process. • global and static variables, load-time constants. • size is fixed and known before program starts execution Stack area for run-time data whose lifetime obeys last-in first-out discipline. • arguments and local variables in subroutines. Heap area for data allocated dynamically. • lifetime doesn't follow LIFO rule • size may not be not known at compile/load time • objects in Java, C "malloc" storage, C++ "new" allocations

6. Memory Layout of Data Segment STACK(size can grow) The three memory areas have distinct purposes. The size of the Static Area is known after program is linked;it doesn't grow dynamically. The Stack and Heap can grow/shrink dynamically. Therefore, the operating environment usually arranges them to "grow" towards unallocated space. (unallocated memory) HEAP(size can grow) STATIC AREA(size fixed)

7. Where are the variables? • In what memory area are these variables allocated? • When is storage allocated and freed? int count = 0; int sumdata( ) { long sum; int value; do { scanf("%d", &value); sum = sum + value; count++; } while (value != 0); return sum; } int main( ) { long sum; double average; sum = sumdata(); average = sum / count; printf( "average is %f", average ); } Q: Find at least 3 bugs in this code

8. Memory Usage sum average [temporaries] [return addr and value] sum value stack space for "main" stack space for "sumdata" (unallocated memory) HEAP(size can grow) count STATIC AREA

9. Where are the variables? • In what memory area are these variables allocated? • When is storage allocated and freed? #define MAXSIZE 1000 extern double size; double* getarray(int n) { // pointer to array double *a; // allocate storage a = (double *) malloc( n*sizeof(double) ); ... read data into a ... return a; } int main( ) { double *arr; double sum; arr = getarray(size); for(int k=0; k<size; k++) sum += arr[k]; }

10. Memory Usage arr sum k n [return addr and value] a stack space for "main" stack space for "getarray" (unallocated memory) value points to HEAP *a 1000*sizeof(double) size (allocated by another part of the program) STATIC AREA

11. Where's the memory (1)? • What is wrong with this C program? • Where is the storage used by each value? /* print a prompt and get reply as a string */ char *getreply( ) { char *reply; printf("What's your name? "); scanf("%s", reply); printf("Hello, %s\n", name); return reply; } int main( ) { char *name = getreply(); }

12. Where's the memory (2)? • Fix it by allocatinglocal storage for reply string. Q: When is the storage for *reply allocated? when freed? /* print a prompt and get reply as a string */ char *getreply( ) { char reply[20]; // can hold string of 19 chars printf("What's your name? "); scanf("%s", reply); printf("Hello, %s\n", name); return reply; } int main( ) { char *name = getreply(); printf("Goodbye, %s\n", name); }

13. Where's the memory (2)? • Previous example works on some OS. • What if you call getreply more than once? int main( ) { char *name1 = getreply(); printf("Goodbye, %s\n", name1); char *name2 = getreply(); printf("Goodbye, %s\n", name2); printf("Sayonara, %s and %s\n", name1, name2); }

14. Where's the memory (3)? • Fix it by allocatingdynamicstorage for reply string. Q: When is the storage for *reply allocated? when freed? /* print a prompt and get reply as a string */ char *getreply( ) { char reply = (char *) malloc( 20 ); printf("What's your name? "); scanf("%s", reply); printf("Hello, %s\n", name); return reply; } int main( ) { char *name = getreply(); printf("Goodbye, %s\n", name); }

16. Where's the memory (Java)? • Where would you expect the memory for these variables to be allocated? • When is the memory allocated? When is it freed? public class Greeter { String who = "nerd"; public void greet( ) { int n = 100; System.out.printf( "hello, %s", who ); } public static int main( ... ) { Greeter g = new Greeter(); g.greet( ); } } Where is System.out ?

17. Languages and Environments Languages differ in how they allocate storage for data: • Static allocation: older Fortran compilers allocate everything statically -- including subroutine activations records. • Heap-oriented: interpreted languages where the data type and size of data is determined at runtime -- put everything on the heap. Scheme, ML (functional) and Smalltalk (O-O). The interpreter itself may use static and stack storage. • Multiple areas: most common for compiled languages.

18. The Runtime Stack • Used for: • Procedure/function/method calls • temporaries • local variables • Temporaries: intermediate results that cannot be kept in registers. • Procedure calls: Sebesta, Chapter 8. • Local variables: part of calls, but can be considered independently, showing LIFO behavior for nested scopes (next slide).

19. Return address Return address Return address Parameters Parameters Parameters Example of stack allocation in C int sub3( int a ) { do something; } float sub2( int x, int y ) { result = sub3(x) + sub3(y); return result; } int sub1( int u, int v ) { int b = 2; float z = sub2(b,u) + sub2(b,v); } int main( ) { sub1( 10, 20 ); Locals & Temps sub3 Locals & Temps sub2 Locals & Temps sub1 main Local variables

20. Heap Allocation • Used for dynamic allocation of storage. • In statically typed languages like C, C++, and Java the heap is used for objects allocated using "new", "malloc", etc. • in C/C++ this can include any data type, e.g. int *p = new int; • in Java, heap is used for reference types ("objects") only; all objects are allocated on the heap. • this includes arrays in Java (any array is an object). • Dynamically typed languages such as LISP, SmallTalk, Perl, use the heap for almost all data. • the type or size of data stored in a variable can change at run-time. • heap allocation is done automatically (no special syntax such as "new" or "malloc" required)

21. Heap Example 1 Scanner input = new Scanner( System.in ); String result = ""; while( input.hasNext( ) ) { String word = input.next( ); result = result + " " + word; } HEAP: FL free Input: this is why String processing is so slow

22. "" this this is result2 result4 result0 result1 result3 word4 word3 word1 word2 Heap Example 1 Scanner input = new Scanner( System.in ); String result = ""; while( input.hasNext( ) ) { String word = input.next( ); result = result + " " + word; } HEAP: this is why this iswhy Input: this is why String processing is so slow String this iswhy String FL

23. result4 word4 Heap Example 1 Scanner input = new Scanner( System.in ); String result = ""; while( input.hasNext( ) ) { String word = input.next( ); result = result + " " + word; } HEAP: "" this this is this is why After 4 iterations, the first four values of result and word no longer longer referenced. They are "garbage". this iswhy String this iswhy String FL

24. Heap Example 2 Scanner input = new Scanner( in ); String [ ] r = new String[10]; while( input.hasNext( ) && k < 10 ) { String word = input.next( ); r[k] = word; } HEAP: [S r FL free Input: this is why String processing is so slow

25. Heap Example 2 Scanner input = new Scanner( in ); String [ ] r = new String[10]; while( input.hasNext( ) && k < 10 ) { String word = input.next( ); r[k] = word; } HEAP: [S r this r[0] is r[1] why r[2] String r[3] Input: this is why String processing is so slow proces.. r[4] word FL

26. Heap Management Heap management involves: • allocating new blocks to satisfy memory requests • must find a suitable free block • maintain a list of unused ("free") blocks • may be many free blocks spread over the heap • the free blocks themselves may be used to create a linked list. • cope with fragmentation of free heap space

27. result4 word4 Management of Heap Example 1 • When the programmer (or garbage collector) reclaims the unused blocks, they are added to the free list. • the heap manager combines free space into larger blocks. • combining free blocks is not as easy as it looks here -- what if blocks are returned at different times? • the old contents of the blocks usually are not cleared (can be a security issue) HEAP: "" this this is this is why this is why String this iswhy String FL

28. Management of Heap • typically, the heap becomes fragmented after extended use • no good solution to this • some heap managers copy (move) blocks to combine free space • this has a lot of overhead • must also change referencesto moved blocks in running processes! HEAP: [S this this junk is morejunk why xxxxxx String

29. Heap Management in C • Allocate storage: void *malloc( size ) // array of int int *array = (int *)malloc( 20*sizeof(int) ); for(k = 0; k<20; k++) array[k] = ...; // character string of length 80 char *s = (char *)malloc( 80*sizeof(char) ); • Free storage: void free( void* ptr ) • "free" requires a pointer argument free( array ); free( s );

30. Heap Management in C++ • Allocate storage: new // a single double (sort of useless) double *d = new double(3.14159); // array of int int *array = new int[100]; // character string of length 80 char *s = new char[80]; • Free storage: delete reference delete d; // free a scalar variable delete [ ] array;

31. Safe Programming • Test whether memory allocating succeeded before continuing. // C int *array = (int *)malloc( 100*sizeof(int) ); if ( ! array ) fprintf( stderr, "malloc failed\n"); // C++ int *array = new int[100]; if ( array == NULL ) cerr << "new failed\n";

32. Garbage • the above solution can lead to creation of garbage... char *getreply( char *prompt ) { char *reply; reply = (char *)malloc(80); /* allocate storage */ printf("%s", prompt); scanf("%s", reply); return reply; } int main( ) { char *s; s = getreply("Want a million baht?"); if ( strcmp(s,"yes")==0 ) PlayLottery( ); s = getreply("Want to study more?"); // answer to the first question is now garbage // C programs must explicitly "free" dynamic storage

33. Garbage Collection • Automatic recycling of dynamic storage allocations no longer in use by program. • Pioneered by LISP, later by SmallTalk • in LISP, programmer doesn't explicitly request storage, so why require him to free storage? • same idea in SmallTalk: make objects easy • How does heap manager know when a block of allocated storage is no longer in use?

34. Garbage Collection Techniques • Reference counting • in each object, keep a counter of how many objects reference it • doesn't work! (example: circular queue) • expensive: uses space in objects and time to update the counts • Mark and Sweep • OK, but still requires some space and time • Other Techniques • dividing heap into halves, swap halves to clean up • use age of objects ("most objects die young")

35. Avoiding Fragmentation • Maintain free lists of "standard" block sizes • Combine/break blocks as needed • used by algorithm called the "buddy system" Example: • create separate free lists of block sizes 32, 64, 128, ... FL1 -> list of 32B blocks FL2 -> list of 64B blocks FL3 -> list of 128B blocks, etc. • if an application needs a block of size (say) 60 bytes and there is nothing in FL2, then split a 128B block into 2 64B blocks. • if the manager frees two adjacent 64B blocks, it can combine them into one 128B (FL3) block

36. Heap Question • Suppose the heap contains only 3 free blocks like this: Address Block Size 0x10FF0 4KB 0x1BC20 3KB 0x3C000 4KB • Your program requests 10KB, like this: char *s; s = (char *)malloc( 10000 ); Q: what does malloc return?(a) 10000 char (combine all 3 blocks),(b) 4096 char (largest block), (c) null

37. Dangling References and Garbage • A dangling reference is a location that has been deallocated from the environment, but is still referenced by the program. • Garbage is memory that is still allocated to the process but has become inaccessible to the process's program. • Which is the more serious problem? • Garbage may cause a program to exhaust memory and fail; but, the computations performed by program are correct (as far as it completed) • Dangling references can cause program to return incorrect results even though program runs OK.

38. Dangling References • Example in C: invalid reference to local storage /* print a prompt and get a reply from the user */ char *getreply( char *prompt ) { char reply[80]; /* string to hold user's response */ printf("%s", prompt); scanf("%s", reply); return reply; /* reference to a local variable */ } int main( ) { char *s1, *s2; s1 = getreply("Want a million baht?"); s2 = getreply("Want to study more?"); Now s1 and s2 point to local storage on the stack Probably s1 = s2 = response to second question !

39. Dangling References Fixed • To avoid this problem... /* print a prompt and get a reply from the user */ char *getreply( char *prompt ) { char *reply; reply = (char *)malloc(80); /* allocate storage */ printf("%s", prompt); scanf("%s", reply); return reply; } int main( ) { char *s1, *s2; s1 = getreply("Want a million baht?"); s2 = getreply("Want to study more?"); s1 and s2 refer to valid (and distinct) storage locations But, programmer must free s1 and s2 to avoid creating garbage.

40. Dangling References (2) • Alias (pointer) to deallocated storage. Not clear in this small example, but a problem in larger apps. /* create pointers to refer to the strings */ char *s, *result; int k; /* allocate a string of length 255 (bad style) */ s = (char *) malloc( 255*sizeof(char) ); /* read input into the string */ scanf("%s", s); result = s; if ( strcmp(result,"yes") == 0 ) DoSomething( ); /* done with result, so free it! */ free( result ); /* oops! s is a dangling reference */

41. Solution to Dangling References • Don't allow programmer to deallocate storage! • Don't allow pointers, either. • Java and C#: use "reference" instead of pointer. references cannot be manipulated like pointers (no p++). Why Dynamic Allocation? • can we eliminate dynamic allocation (other than stack automatic variables)? • If we don't allow dynamic allocation, many useful programs would not be possible.

42. Memory Management in Java

43. Features • Memory is managed by the Java Virtual Machine (JVM) • this ensures that a Java application never accesses memory outside the JVM. • it also allows the JVM to "catch" memory violations and raise a run-time exception • JVM provides both heap and stack space to Java app. • JVM performs automatic garbage collection.

44. Heap Layout "Most objects die young" -- over 90% of all objects are de-referenced within the first few iterations of garbage collection. • This influences Java's heap management strategy • Divide the heap into 4 sub-areas: Eden Space (newly created objects) Survivor Space (objects that survive first gc) Tenured Generation (objects that survive long enough are promoted to this area) Permanent Generation (tenured objects that survive really long are promoted to here)

45. References • http://www.informit.com/guides/ content.asp?g=java&seqNum=249&rl=1 • Java Memory Profilers: http://www.manageability.org/blog/stuff/open-source-profilers-for-java