Debug C Run-Time & Memory Management

1. Debug C Run-Time & Memory Management Chapter 17, Debugging Applications for .NET and Windows by John Robbins March 19th 2004 Divyanand M. Kutagulla, SU, Dept of EECS, dkutagulla@hotmail.com

2. Main Topics • The Debug C Runtime Library (The DCRT) • How to setup use the DCRT • Useful DCRT functions • Memory Management • OS heaps • Useful VS.NET Compiler switches

3. The Debug C Runtime Library (The DCRT) References 1. Robbins’ Book 2. Platform SDK

4. Overview • The Debug C Run-Time Library • DCRT Features • Setting up and Using the DCRT • Useful DCRT functions • DCRT library versions • An Important Note

5. The Debug C Run-Time Library • The Debug version of the C/C++ runtime on the Microsoft platforms • ‘Memory debugging’ for C/C++ • Has been there since Visual C++ 4… • a.k.a. DCRT (say it as: “D-Cert”)

6. DCRT Features • Heap Memory tracking support • Allocations • De-allocations • Memory underwrites, overwrites • Memory Leaks • Reporting Subsystem • Trace of the app’s memory behavior

7. Heap Memory Tracking Support • Memory allocations done through new, malloc, calloc • Memory underwrites, overwrites • Reported through an assertion in the DCRT • Memory Leaks • Reported at app exit though a call to OutputDebugString • MFC reports memory leaks this way…

8. Reporting Subsystem • Reporting Subsystem (trace) • _RPTn & _RPTFn macros • n is a number from 1 to 4, shows how many parameters to report • Can redirect the reports to any destination (a logfile, console etc) • Assertion support through the ASSERT macro • Violates the CROA (Cardinal rule of assertions) by destroying the last error code...

9. Setting up & Using the DCRT • DCRT Source Files • DCRT Setup • DCRT Flags • Useful DCRT functions

10. DCRT Source Files • Optionally installed during VS.NET installation • If CRT source code installation is selected • Install Path: <VS. NET inst dir>\vc7\crt\src\ <VS.NET inst dir>: VS.NET installation directory • E.g. C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\crt\src

11. DCRT Source Files (cont’d)

12. DCRT Source Files (cont’d)

13. DCRT Setup • #define the _CRT_DBG_MAP_ALLOC macro before any #includes in the header. • This will cause all the calls to memory allocation/de-allocation routines to be redirected to special versions of these routines • The special versions record the source file name and the line number in that file where the call occurred • __FILE__ & __LINE__ macros

14. DCRT Setup (cont’d) • After all the header files have been ‘#include’d, ‘#include’ the CRTDBG.H header. • Turn on the DCRT heap code: • Done through calling the _CrtSetDbgFlag() with the appropriate flag arguments to configure the DCRT • Turn on the memory leak reporting • Turn on memory checking on each Memory allocation • Etc…

15. DCRT Flags

16. DCRT Flags (cont’d)

17. DCRT Bug!!! If we allocate memory through the new operator, the memory leak report does not show the correct source file location of the leaked allocation. (sample report below is from the previous code example) Detected memory leaks! Dumping objects -> c:\program files\microsoft visual studio .net2003\vc7\include\crtdbg.h (689) : {57} normal block at 0x00320800, 28 bytes long. Data: < myclass > CD CD CD CD 6D 79 63 6C 61 73 73 00 CD CD CD CD …< OUTPUT TRUNCATED FOR CLARITY >… Object dump complete. Note DCRT BUG!!!: The leak report for the ‘new’ed memory seems to come from crtdbg.h although the error is not in this file..

18. Why the DCRT Bug??? • The new operator is defined as an inline in CRTDBG.H : inline void * __cdecl operator new(size_t s) { return ::operator new(s, _NORMAL_BLOCK, __FILE__, __LINE__); } But it is not expanded inline at point of call when _DEBUG is defined (i.e. when we do a Debug build)!!!

19. Useful DCRT functions References 1. Robbins’ book 2. MSDN

20. Overview • Memory allocation functions • Memory Integrity Check functions • Parameter Validation Functions • Memory State Validation Functions • DCRT Hooks • Memory allocation/deallocation • Memory dumping • DCRT Debug Report Functions

21. Memory allocation functions • _malloc_dbg: C style Memory Allocation • Handles calls to malloc & new in Debug builds • new: C++ style memory allocation • Calls to new are passed to _nh_malloc_dbg defined in DBGHEAP.C • Debugging support offered: (example block layout) • Buffers on either side of user portion of block • File name and line number of the allocations • Allocation Type information : used for leak detection and state reporting

22. Memory Block Allocation Types • _NORMAL_BLOCK: The Default block allocated by call to malloc/new. Only keeps track of the file and line information. Raw memory dump • _CLIENT_BLOCK: Can be customized by the application to keep track of additional information that may be output when memory is dumped

23. Memory Block Allocation Types • _CRT_BLOCK: Used for internal CRT use • _FREE_BLOCK: These blocks are only marked as free and not removed from the allocation list when de-allocated • Filled with 0xDD • _IGNORE_BLOCK When The DCRT heap is turned off in the middle the program, the memory allocated is stored in allocation as these blocks

24. Memory Integrity Check functions • Memory integrity _CrtCheckMemory(): • Assert(_CrtCheckMemory())tracks down underwrites and overwrites as close as they occur, also checks that freed pointers are not used with out being reinitialized. • When the _CRTDBG_CHECK_ALWAYS_DF is set, this Function is called after every memory allocation / de-allocation. Code Example

25. Parameter Validation Functions • Validity of Memory: can be used as debugging parameter validation functions • _CrtIsValidHeapPointer • _CrtIsMemoryBlock • _CrtIsValidPointer

26. Memory State Validation Functions • Memory State Functions: Keep track of Memory state, Can use these to track memory leaks or to track memory consumption • _CrtMemCheckpoint • _CrtMemDifference • _CrtMemDumpStatistics (Code Example)

27. DCRT Hooks • Hook into the memory allocation code stream to see each allocation call (Code Example) • Can’t do the same for deallocations though… • Hook memory dump functions and use them to enumerate client blocks (memory that the app allocates) (Code Example) • Can generate more useful memory dump messages • Can have subtypes of a client block for more dumping granularity

28. DCRT Debug Report Functions • Used to control the DCRT reporting subsystem • Can redirect the Debug heap Report output to a file or a the debug window or a message box • _CrtSetReportMode • _CrtSetReportFile • _CrtSetReportHook • _CrtSetReportHook2 (Code Example)

29. DCRT library versions • Two major categories: • Debug and Release • In each category there are three types: • Single Threaded static library • Multithreaded static library • Multithreaded DLL • In total, six versions of the C runtime are available…

30. An Important Note… • Use the same version of the C runtime library across all the modules of your application • Else … can end up allocating memory using one version of the library and deallocating it using another version • “Different versions of the DCRT running at the same time have multiple versions of the heap memory tracking the same code” - Robbins

31. Memory Management References: 1. Robbins’ Book 2. Platform SDK

32. Overview • Windows Processes • Process Partitions • Process Heaps

33. Windows Processes… • 32 bit processes => Virtual address space (VAS) of 4GB (8TB on 64 bit Windows) • Addressable by all the processes’ threads • Page Map translates all the processes’ Virtual Addresses (VAs) to Physical Addresses (PAs) • Maintained by Windows, accessed each time a thread makes a memory reference

34. 32 bit VAS Partitions • Default 4 GB of process space: • 2 GB application partition in low memory (0x00000000 thro’ 0x7FFFFFFF) • 2GB system partition (0x80000000 thro’ 0xFFFFFFFF) • Can change this allocation to 3GB application partition and 1 GB system partition • Add the /3GB switch to Boot.ini file • For I/O intensive applications (e.g. DBMS) running on server systems

35. Process Heaps • Block of one or more pages in the VAS of the process • Reside in the Application Partition • Created by the processes themselves • Use a different set of functions (windows API functions) to manage this memory

36. Useful VS.NET Compiler Switches References: 1. Robbins’ Book 2. VS.NET Online Help System

37. Overview • Run Time Check Switches (/RTCx) • Buffer Security Switch /GS

38. Run Time Check Switches (/RTCx) • /RTCc: Enable data value truncation checks (e.g. a 16 bit value assigned to a 8 bit value) • Need to select the “Smaller Type Check” option • /RTCs: Enable stack frame runtime checking • All local variables initialized to OxCC (breakpoint instruction in the x86) • Stack Frame corruption: mismatched calling conventions (calling a _stdcall function with a function pointer having a _cdecl convention).

39. Run Time Check Switches • /RTCu: Check for uninitialized variables and their usage. • Can avoid this by compiling this with /W4 (warning level 4) and /Wx treat all warnings as errors • /RTC1: Shorthand for combining /RTCs and /RTCu • Note: The /RTCx switches will not allow any optimizations. These switches are only valid in a debug build.

40. Run Time Check Switches

41. Buffer Security Check Switch /GS • Must be turned on for both Debug and Release builds • Ensures that the function return address is not modified. • A security cookie XORed with the return address is stored before the return address • This value is again recomputed at function exit and checked against the original value. If there is a change an error box pops up… (this behavior can be overridden)

42. Buffer Security Check Switch • Implemented in the Microsoft C runtime files: • SECINIT.C • SECHOOK.C • SECFAIL.C • Provided to allow user to override buffer overrun response…

43. Customizing Buffer Overrun Response • Default behavior of Buffer overrun • A message box is displayed and the program terminates • Can override the above behavior • Hook a user defined processing function to _set_security_error_handler (Code Example)

44. Done at last…WHEW!!! Man… DCRT is really neat !!!

45. The End Questions?