CS 3214 Introduction to Computer Systems

1. CS 3214Introduction to Computer Systems Godmar Back Lecture 21

2. Announcements • Read Chapter 10 • Exercise 11 due Nov 12 • Project 4 • Should complete before Thanksgiving, but will accept submissions without using late days until Nov 24 • Don’t procrastinate. A memory allocator is easily something you can sink 100 hours into with no results to show – an option you don’t have – zero room for error on this project • Gave preliminary heads up on whether Project 3 meets minimum requirements • Peter will give definite by this weekend, he said • Please check your submissions that you have tests for all minimum functionality. If not, you have until the end of the semester to submit a version that meets minimum requirements CS 3214 Fall 2009

3. Some of the following slides are taken with permission from Complete Powerpoint Lecture Notes forComputer Systems: A Programmer's Perspective (CS:APP) Randal E. Bryant and David R. O'Hallaron http://csapp.cs.cmu.edu/public/lectures.html Part 2 Memory Management CS 3214 Fall 2009

4. Dynamic Memory Allocation Application • Explicit vs. Implicit Memory Allocator • Explicit: application allocates and frees space • E.g., malloc and free in C • Implicit: application allocates, but does not free space • E.g. garbage collection in Java, ML or Lisp • Allocation • In both cases the memory allocator provides an abstraction of memory as a set of blocks • Doles out free memory blocks to application • Will discuss automatic memory allocation today Dynamic Memory Allocator Heap Memory CS 3214 Fall 2009

5. Implicit Memory Management • Motivation: manually (or explicitly) reclaiming memory is difficult: • Too early: risk access-after-free errors • Too late: memory leaks • Requires principled design • Programmer must reason about ownership of objects • Difficult & error prone, especially in the presence of object sharing • Complicates design of APIs CS 3214 Fall 2009

6. Manual Reference Counting • Idea: keep track of how many references there are to each object in a reference counter stored with each object • Copy a reference to an object globalvar = q • increment count: “addref” • Remove a reference p = NULL • decrement count: “release” • Uses set of rules programmers must follow • E.g., must ‘release’ reference obtained from OUT parameter in function call • Must ‘addref’ when storing into global • May not have to use addref/release for references copied within one function • Programmer must use addref/release correctly • Still somewhat error prone, but rules are such that correctness of the code can be established locally without consulting the API documentation of any functions being called; parameter annotations (IN, INOUT, OUT, return value) imply reference counting rules • Used in Microsoft COM & Netscape XPCOM CS 3214 Fall 2009

7. Automatic Reference Counting • Idea: force automatic reference count updates when pointers are assigned/copied • Most common variant: • C++ “smart pointers” – C++ allows programmer to interpose on assignments and copies via operator overloading/special purpose constructors. • Disadvantage of all reference counting schemes is their inability to handle cycles • But great advantage is immediate reclamation: no “drag” between last access & reclamation CS 3214 Fall 2009

8. Garbage Collection • Determine which objects may be accessed in the future • Don’t know which one’s will, but can determine those who can’t be accessed because there are no pointers to them • Requires that all pointers are identifiable (e.g., not pointer/int conversion • Invented 1960 by McCarthy for LISP CS 3214 Fall 2009

9. Reachability Graph B B C A C A B C B A A C A B A B C C Root set Thread A Thread B Thread C CS 3214 Fall 2009

10. Reachability Graph B B C A C A B C B A A C A B A B C C Root set Thread A Thread B Thread C CS 3214 Fall 2009

11. Reachability Graph B C A C A B C B A A C A B B C C Root set Thread A Thread B Thread C CS 3214 Fall 2009

12. Reachability Graph B C A C A B C B A A C A B B C C Root set Thread A Thread B CS 3214 Fall 2009

13. Reachability Graph B C A C A B C B A A C A B B C Root set Thread A CS 3214 Fall 2009

14. Reachability Graph A A A A Root set Thread A CS 3214 Fall 2009

15. Application Programmer’s Perspective • Tuning garbage collection parameters • Dealing with Memory Leaks • Garbage collection in mixed language environments • C code must coordinate with the garbage collection system in place CS 3214 Fall 2009

16. GC Design Choices • Determining which objects are reachable • “marking” live objects, or • “evacuating”/”scavenging” objects – copying live objects into new area (if objects are movable) • Deallocating unreachable objects • “sweeping” – essentially calling “free()” on all unreachable objects • more efficient if it’s possible to evacuate all life objects from an area cost generally proportional to amount of life objects in area considered cost proportional to amount of dead objects (garbage) in theory, constant cost; in practice, dominated byneed to zero memory CS 3214 Fall 2009

17. Memory Allocation Time-Profile Allocated Memory Amax garbage live Time Start time – ts End time – te CS 3214 Fall 2009

18. Modeling Memory Allocation Allocated Memory Amax allocation rate garbage live Time Start time – ts End time – te CS 3214 Fall 2009

19. Execution Time vs. Memory memory MaxHeap time ts te CS 3214 Fall 2009

20. Execution Time vs. Memory memory Max Heap time ts te CS 3214 Fall 2009

21. Execution Time vs. Memory memory Max Heap time ts te CS 3214 Fall 2009

22. Execution Time vs. Memory memory Max Heap time ts te CS 3214 Fall 2009

23. Execution Time vs. Memory memory Max Heap time ts te CS 3214 Fall 2009

24. Heap Size vs. GC Frequency • All else being equal, smaller maximum heap sizes necessitate more frequent collections • Rule of thumb: need between 1.5x and 2.5x times the size of the live heap to limit collection overhead to 5-15% for applications with reasonable allocation rates • Performance degradation occurs when live heap size approaches maximum heap size CS 3214 Fall 2009

25. Infant Mortality Source: http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html CS 3214 Fall 2009

26. Generational Collection • Observation: “most objects die young” • Allocate objects in separate area (“nursery”, “Eden space”, collect area when run out of space • Will typically have to evacuate few survivors • “minor garbage collection” • But: must treat all pointers into Eden as roots • Typically, requires cooperation of the mutator threads to record assignments: if ‘b’ is young, and ‘a’ is old, a.x = b must add a root for ‘b’. • Aka “write barrier” CS 3214 Fall 2009

27. Example: The Hotspot Collector • See http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html CS 3214 Fall 2009

28. When to collect • “Stop-the-world” • All mutators stop while collection is ongoing • Incremental • Mutators perform small chunks of marking during each allocation • Concurrent/Parallel • Garbage collection happens in concurrently running thread – requires some kind of synchronization between mutator & collector CS 3214 Fall 2009

29. Precise vs. Conservative Collectors • Precise collectors keep only objects alive that are in fact part of reachability graph • Conservative collectors may keep objects alive that aren’t • Reason typically that they do not know where pointers are stored, must conservatively guess • In-between forms: some systems assume precise knowledge of heap objects, but not stack frame layouts • Can be expensive to keep track of where references are stored on the stack, particularly in fully preemptive environments • Conservatism makes GC usable for languages such as C, but prevents moving/compacting of objects CS 3214 Fall 2009

30. Memory Leaks • Objects that remain reachable, but will not be accessed in the future • Due to application semantics • Will ultimately lead to out-of-memory condition • But will degrade performance before that • Common problem, particularly in multi-layer frameworks • Heap profilers can help CS 3214 Fall 2009