SECURE PROGRAMMING Chapter 4 Dynamic Memory Management

1. SECURE PROGRAMMING Chapter 4 Dynamic Memory Management

2. Heap management vulnerabilities Buffer overflows Double-Free Vulnerabilities Writing to freed memory Another Windows vulnerability: Look-Aside Table Mitigation Strategies Vulnerability Hall of Shame Summary Overview • Introduction • C Memory Management • Common C Memory Management errors • C++ Dynamic Memory Management • Common C++ Dynamic Memory Management Errors • Memory Managers • Doug Lea's Memory Allocator • RtlHeap

3. Introduction Memory management is a source of: programming defects security flaws vulnerabilities Main causes: Double freeing Buffer overflows use of freed pointers ….

4. C Memory Management Functions defined: malloc(size_t size) aligned_alloc(size_t alignment,size_t size) realloc(void *p, size_t size) (Do not use size=0) calloc(size_nmemb, size_t size) (initializes to 0) free(void *p)

5. C Memory Management The importance of alignment subobjects complete objects alignment hierarchy (weaker to stronger/stricter) max_align_t alignas(size_t size) (support for SIMD) extended alignment > max_align_t overaligned type Note that realloc() does not preserve alignment!

6. C Memory Management alloca(size_t size) Allocates on stack area Usually inlined Non-standard Dangerous Variable length arrays: similar. Lots of caveats

7. Common C Memory Management Errors Initialization errors Random behavior Security leaks Use memset(void *s, int c, size_t n) or memset_s() Can use calloc, provided the product of the arguments doesn't overflow.

8. Common C Memory Management Errors Failing to check return values AIX and Linux may allow allocation requests to succeed even if there is no space, and then kill the process when it tries to access too much memory. Causes for Heap exhaustion: memory leaks Data structures incorrectly implemented Overall system memory exhausted Other transient conditions An example: indirecting through an offset of the returned pointer (p 154)

9. Common C Memory Management Errors Dereferencing Null or Invalid Pointers (or their offsets) Referencing Freed Memory Freeing Memory Multiple times (example, page 157) Memory Leaks (can facilitate DOS attacks) Zero-Length Allocations (pp 159-160)

10. C++ Dynamic Memory Management Only dynamic functions are: new type [(initialization list)] delete item (or ~x) (Constructors cannot be called explicitly) new return a pointer to the object requested; initialized if given initializer data new delete, plain for “scalar” data, [ ] for arrays Also: new (place) type [(initialization list)]

11. C++ Dynamic Memory Management Allocation Functions Class member or global function May not: Use namespace other than global Declare as static in global scope Returns a void * First parameter: std::size_t size Idea is that new is implemented through malloc/calloc Failure throws an exception of type std::bad_alloc unless called with argument std::nothrow; in that case, it returns null pointer: T * p1 = new T; // can throw bad_alloc T * p2 = new(std::nothrow) T; // returns null pointer on failure

12. C++ Dynamic Memory Management More on allocation failures Standard idiom for allocation and allocation failure: Resource Acquisition Is Initialization (RAII): Attach lifetime of a resource to lifetime of object it refers to Examples, pages 166, 167, 168 (gets handler address)

13. C++ Dynamic Memory Management Deallocation Functions Same restrictions as on allocate functions. Returns void, first parameter is void *, may have second parameter: either a) delete() one parameter b) delete() two parameters, second type std::size_t Both scalar and [ ] versions. If first parameter is void, is a no-op. If first argument is none-of-the-above, we have problems.

14. C++ Dynamic Memory Management Garbage Collection Optional in C++ Boehm-Demers-Weiser conservative GC does not require use of free/delete. Can also be used to detect memory leaks. Weakness: Disguised pointers. Modified pointers Non referenced pointers (examples, pp 169-170)

15. C++ Dynamic Memory Management Garbage Collection Remedy? Inquire rules for pointer safety relaxed: Normal rules preferred (similar to relaxed, but a gc may run to detect leaks or “bad pointers” strict (gc may be running) namespace std { enum class pointer_safety {relaxed, preferred, strict}; pointer_safety get_pointer_safety(); }

16. C++ Dynamic Memory Allocation Garbage Collection In C++11: declare_reachable/undeclare_reachable page 171

17. Common C++ Dynamic Memory Management Errors:Bad Allocation Failure check C++ allows either NULL return or exception throwing: do not mix! (std::nothrow)

18. Common C++ Dynamic Memory Management Errors Improperly pairing C and C++ memory management functions. C++ is a superset of C, so malloc & friends/free are OK to use, However, they may use different memory areas and the do use different algorithms. Do not mix!

19. Common C++ Dynamic Memory Management Errors Use scalar new with scalar delete; use array new with array delete. There are new, member new, operator new and delete, member delete, operator delete. Operator new may allocate raw memory, without calling a constructor; don't call a destructor. Call operator delete instead.

20. Common C++ Dynamic Memory Management Errors

21. Common C++ Dynamic Memory Management Errors Double freeing memory:

22. Common C++ Dynamic Memory Management Errors Standard C++ containers with pointers do not delete their objects, means programmer has to do it (p 177) but (p 178: double-free vulnerability:) Also, not exception-safe. Solutions: 1) Garbage collecting memory management. 2) Smart pointers: overload -> and * to act like pointers ++ (add check for Null, GC, reference counts, etc.) Most common smart pointer: std::shared_ptr from standard library (pp 178/179)

23. Common C++ Dynamic Memory Management Errors Deallocation Function throws an exception NONO (p 180)

24. Memory Managers Manage both allocated and free memory. Runs as part of the user process Three types: OS supplied Compiler supplied User supplied Algorithm due to D. E Knuth, The Art of Computer Programming (several editions) First fit vs best fit. In band linked lists may be a bad idea, but there are none better.

25. Doug Lea's Memory Allocator dlmalloc

26. Doug Lea's Memory Allocator

27. Doug Lea's Memory allocator Free chunks arranged in circular double-linked headed lists called bins; Small sizes have dedicated bins, larger sizes have bins dedicated to a range of sizes, arranged in descending size order. Special bin for recently freed chunks, acts like cache: one chance and they are ent to regular bin. Unlink macro takes a chunk from the bin (code page 184, picture, page 185)

28. RtlHeap by Microsoft

29. RtlHeap by Microsoft: Virtual Memory API Page base: 32 bit linear addressing 4096 byte pages Each region either reserved, committed or free.Must have common protection, type, base allocationPages also protection and pagelock flag status bits

30. RtlHeap by Microsoft:Heap Memory API HeapCreate(maxsize...) → unique handle Default heap, handle obtainable with GetProcessHeap()

31. RtlHeap by MicrosoftLocal, Glabal Memory API Provided local and global memory management for backward compatibility with Windows 3.1

32. RtlHeap by MicrosoftCRT Memory Functions Before Win32, much FUD, with Win32 uses local/global memory management and is safe and portable.

33. RtlHeap by MicrosoftMemory-Mapped File API Virtual address space mapped directly onto a file: → file access becomes dereferencing a pointer.

34. RtlHeap by MicrosoftData Structures Uses virtual memory API, Implements all the others Constantly evolving Programmers need to assume least secure version. Internal data structures: Process environment block free lists look aside lists memory chunk structures

35. RtlHeap by MicrosoftData Structures: Process Env Block Maintains global variables for each process.

36. RtlHeap by MicrosoftData Structures: FreeList[] Array of 128 LIST_ENTRY structs = head of double-linked lists.. Located at 0x178 from address returned by HeapCreate() Keep track of free chunks of a particular size: index*8, except FreeList[0], keeps buffers > 1024, < virtual allocation threshold, sorted from smallest to largest.

37. RtlHeap by MicrosoftData Structures: FreeList[]

38. RtlHeap by MicrosoftData Structures: Look-aside Lists Requires HEAP_NO_SERIALIZE not set and HEAP_GROWABLE set (defaults) Creates 128 singly linked look-aside lists speed up allocation of small ( < 1016 bytes) blocks Start out empty, grow as memory is freed.

39. RtlHeap by MicrosoftData Structures: Memory Chunks This structure precedes address returned by HeapAlloc by 8 bytes. Chunk size field are given in quad words (i.e./8)

40. RtlHeap by MicrosoftData Structures: Memory Chunks Free chunk picture:

41. RtlHeap by Microsoft

42. RtlHeap by Microsoft

43. RtlHeap by Microsoft

44. RtlHeap by Microsoft

45. RtlHeap by Microsoft

46. Doug Lea's Memory allocator

47. Double-Free Vulnerabilities

48. Mitigation Strategies

49. Vulnerability Hall of Shame

50. Summary C Memory Management Common C Memory Management errors C++ Dynamic Memory Management Common C++ Dynamic Memory Management Errors Memory Managers Doug Lea's Memory Allocator Double-Free Vulnerabilities Mitigation Strategies Vulnerability Hall of Shame Summary