Dynamic Memory Management

1. Dynamic Memory Management Secure Coding in C and C++

2. Agenda • Programmer View of Dynamic Memory • Dynamic memory functions • Dynamic memory manager • Common Errors • Common Implementations and Exploits • Doug Lea’s Memory Allocator • RtlHeap • Mitigation Strategies

3. Dynamic Memory Interface • Memory allocation in C: • calloc() • malloc() • realloc() • Deallocated using the free() function. • Memory allocation in C++ using the new operator. • Deallocated using the delete operator.

4. Dynamic Memory Interface • malloc(size_t size); • Allocates size bytes and returns a pointer to the allocated memory. • The memory is not cleared. • free(void * p); • Frees the memory space pointed to by p, which must have been returned by a previous call tomalloc(), calloc(), or realloc(). • If free(p) has already been called before, undefined behavior occurs. • If p is NULL, no operation is performed.

5. Dynamic Memory Interface • realloc(void *p, size_t size); • Changes the size of the memory block pointed to by p to size bytes. • The contents will be unchanged to the minimum of the old and new sizes. • Newly allocated memory will be uninitialized. • If p is NULL, the call is equivalent to malloc(size). • if size is equal to zero, the call is equivalent to free(p). • Unless p is NULL, it must have been returned by an earlier call to malloc(),calloc(), or realloc().

6. Dynamic Memory Interface • calloc(size_t nmemb, size_t size); • Allocates memory for an array of nmemb elements of size bytes each and returns a pointer to the allocated memory. • The memory is set to zero.

7. Memory Managers • Manage both allocated and deallocated memory. • Run as part of the client process. • Use a variant of the dynamic storage allocation algorithm described by Knuth. • Memory allocated for the client process and memory allocated for internal use, is all within the addressable memory space of the client process.

8. Memory Managers • Different Algorithms for Memory Allocation • Sequential Fit Methods • Look for first free area that fits. current pointer circular linked list of blocks

9. Memory Managers • Different Algorithms for Memory Allocation • First Fit • Look for first free area that fits from start of memory. circular linked list of blocks

10. Memory Managers • Different Algorithms for Memory Allocation • Best Fit • Look for the tightest fit. circular linked list of blocks

11. Memory Managers • Different Algorithms for Memory Allocation • Optimal Fit • Sample the ring of free blocks. • Then choose the first block better than the sample. • Goes back to the optimal marriage strategy. • (As developed by Mathematicians, so apply at your own risk)1: • Date n girls. • Date more girls, but marry the one better than the previous n girls. • Stop dating. • Optimal if you can date a total of 2n girls. • Small chance that you end up with the worst. The author, the Department of Computer Engineering, the School of Engineering, Santa Clara University strongly advise: Consult your own personal marriage consultant before following this advise. Negative effects can arise. Participants are not guaranteed to be married to the best girl. Girls may get together and dump on you. …. Ranking possible marriage partners might be considered degrading.

12. Memory Managers • Different Algorithms for Memory Allocation • Worst fit • Pick the biggest free block.

13. Memory Managers • Different Algorithms for Memory Allocation • Buddy System Methods • Previous methods can lead to fragmentation. • Buddy systems only allocate blocks of size 2i. • If request is for block of size m, allocate instead block of size 2[logb m]+1or – if necessary – larger. • When blocks are returned, try to coalesce them with their buddy, an adjacent block of the same size.

14. Memory Managers • Buddy System Example coalesce

15. Memory Managers • Different Algorithms for Memory Allocation • Segragation • Maintain separate lists of blocks of uniform size.

16. Memory Managers • Memory managers • return freed blocks asap into the pool. • coalesce adjoining free blocks into larger blocks. • sometimes use compacting of reserved blocks. • Moves all blocks together.

17. Common Dynamic Memory Errors • Initialization errors, • Failing to check return values, • Writing to already freed memory, • Freeing the same memory multiple times, • Improperly paired memory management functions, • Failure to distinguish scalars and arrays, • Improper use of allocation functions.

18. Common Dynamic Memory Errors • Initialization • Programmer assumes that malloc() zeroes block. • Initializing large blocks of memory can impact performance and is not always necessary. • Programmers have to initialize memory using memset() or by calling calloc(), which zeros the memory.

19. Common Dynamic Memory Errors Initialization /* return y = Ax */ int *matvec(int **A, int *x, int n) { int *y = malloc(n * sizeof(int)); int i, j; for (i = 0; i < n; i++) for (j = 0; j < n; j++) y[i] += A[i][j] * x[j]; return y; } y[i] is initially zero, right?

20. Common Dynamic Memory Errors • Initialization • tar program on Solaris 2.0 included fragments of the /etc/passwd file.

21. Common Dynamic Memory Errors • Failing to Check Return Values • Memory is a limited resource and can be exhausted. • Memory allocation functions report status back to the caller. • VirtualAlloc() returns NULL, • Microsoft Foundation Class Library (MFC) operator new throws CMemoryException *, • HeapAlloc() may return NULL or raise a structured exception. • The application programmer should: • determine when an error has occurred. • handle the error in an appropriate manner.

22. Common Dynamic Memory Errors • Failing to Check Return Values • The standard malloc() function returns a NULL pointer if the requested space cannot be allocated. • When memory cannot be allocated a consistent recovery plan is required.

23. Common Dynamic Memory Errors • Failing to Check Return Values • PhkMalloc • provides an X option that instructs the memory allocator to abort() the program with a diagnostic message on standard error rather than return failure. • This option can be set at compile time by including in the source: • extern char *malloc_options; • malloc_options = "X“.

24. Common Dynamic Memory Errors • Failing to Check Return Values • malloc returns a null pointer if no memory can be allocated. • new operator in C++ throws a bad_alloc exception. • Using new, encapsulate the allocation in a try block.

25. Common Dynamic Memory Errors • Failing to Check Return Values • Checking for return value of malloc int *i_ptr; i_ptr = (int *)malloc(sizeof(int)*nelements_wanted); if (i_ptr != NULL) { i_ptr[i] = i; } else {      /* Couldn't get the memory - recover */ }

26. Common Dynamic Memory Errors • Failing to Check Return Values • Exception handling for new operator try { int *pn = new int; int *pi = new int(5); double *pd = new double(55.9); int *buf = new int[10];    . . . } catch (bad_alloc) {    // handle failure from new }

27. Common Dynamic Memory Errors • Failing to Check Return Values • This does NOT work! int *pn = new int; if (pn) { … } else { … } if condition is always true, regardless of success of memory allocation.

28. Common Dynamic Memory Errors • Failing to Check Return Values • Using the nothrow variant of new works like malloc: int *pn = new(nothrow) int; if (pn) { … } else { … }

29. Common Dynamic Memory Errors • Referencing freed memory • Usually works, since memory is not immediately reused. wrong: accessing freed memory • for (p = head; p != NULL; p = p->next) • free(p); correct: using a temp variable for (p = head; p != NULL; p = q) { q = p->next; free(p); }

30. Common Dynamic Memory Errors • Referring to freed memory • Unlikely to result in a runtime error • because memory is owned by the memory manager of the program. • Freed memory can be allocated before a read. • Read reads incorrect values. • Writes destroy some other variable. • Freed memory can be used by the memory manager. • Writes can destroy memory manager metadata. • Difficult to diagnose run-time errors. • Basis for an exploit

31. Common Dynamic Memory Errors • Freeing memory multiple times • Often result of a cut-paste on code. x = malloc(n * sizeof(int)); /* manipulate x */ free(x); y = malloc(n * sizeof(int)); /* manipulate y */ free(x);

32. Common Dynamic Memory Errors • Freeing memory multiple times • Data structures can contain links to the same item. • Example: (What happens if both lists are freed?)

33. Common Dynamic Memory Errors • Freeing memory multiple times • Error processing • Same memory chunk might be freed by the error handler and by the throwing try block. • In general: • Memory leaks are safer than double frees.

34. Common Dynamic Memory Errors • Improperly failed memory management functions • Always use • new  delete • malloc  free • Improper pairing can work on some platforms sometimes, but code is not portable.

35. Common Dynamic Memory Errors • Failure to distinguish scalars and arrays • C++ has different operators for scalars and arrays • new  delete for scalars • new[]  delete[] for arrays

36. Common Dynamic Memory Errors • Improper use of allocation functions • malloc(0) • Can lead to memory management errors. • A C runtime library can return • a NULL pointer • or return a pseudo-address • The safest and most portable solution is to ensure zero-length allocation requests are not made.

37. Common Dynamic Memory Errors • Improper use of allocation functions • Usingalloca() • Function: • Allocates memory in the stack frame of the caller. • Automatically freed when function calling alloca() returns. • Definition: • Is NOT defined in POSIX, SUSv3, C99. • But available on some BSD, GCC, Linux distributions. • Problems: • Often implemented as an in-line function. • Does not return null error. • Can make allocations larger than stack. • Confused programmers can call free

38. dlmalloc • Doug Lea’s malloc • default for gcc and on most versions of Linux • Description if for dlmalloc 2.7.2, but vulnerabilities are the same for other versions.

39. dlmalloc • dlmalloc manages chunks of memory • Free (aka unallocated) • Allocated

40. dlmalloc The first four bytes of allocated chunks contain the last four bytes of user data of the previous chunk. The first four bytes of free chunks contain the size of the previous chunk in the list.

41. dlmalloc • Free chunks • Organized in double linked lists. • Contain forward and backward pointers to the next and the previous chunk. • Chunk size stored in the last four B. • Allocated and free chunks are distinguished by the PREV_INUSE bit. • Chunk sizes are always even, PREV_INUSE bit is the low order bit of the chunk size.

42. dlmalloc PREV_INUSE PREV_INUSE The first four bytes of allocated chunks contain the last four bytes of user data of the previous chunk. The first four bytes of free chunks contain the size of the previous chunk in the list.

43. dlmalloc • Free chunks are kept in bins. • Addressed by head. • Chunks in bins are of approximately same size. • Additional bin for recently freed memory.

44. dlmalloc • During free() • Memory chunks are consolidated, if possible. • Merged with adjacent free chunk. • Chunk before is free: • Merge with that chunk. • Chunk after is free: • Take that chunk of list. • Merge with current chunk.

45. dlmalloc • Unlink Macro #define unlink(P, BK, FD) { \ FD = P->fd; \ BK = P->bk; \ FD->bk = BK; \ BK->fd = FD; \ }

46. dlmalloc

47. dlmalloc • Unlink Technique • The unlink technique: • Introduced by Solar Designer. • Used against versions of Netscape browsers, traceroute, and slocate that used dlmalloc. • Uses a buffer overflow to manipulate the boundary tags on chunks of memory • To trick the unlink macro into writing four bytes of data to an arbitrary location. • We have seen how dangerous this is.

48. dlmalloc #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } Memory allocation chunk 1 Memory allocation chunk 2 Memory allocation chunk 3

49. dlmalloc #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } The program accepts a single string argument that is copied into first This unbounded strcpy() operation is susceptible to a buffer overflow.

50. dlmalloc #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } the program calls free() to deallocate the first chunk of memory