CSE 380 – Computer Game Programming Memory Management

1. CSE 380 – Computer Game ProgrammingMemory Management Pocket WORM, a rip-off of Snake

2. Your first game industry job interview • Employer: “Hello” • You: “Hi” • Employer: “Well your resume looks good. If you have a student project to show I’d like to see it.” • You: “Sure, I have two games I made that you can download from my Web page.” • Employer: “Ok, great. Just a few questions then. What are the memory budgets for those games? How does your memory management system work? Oh, and please on this piece of paper, override the new and delete keywords and write your own definitions. In fact, do the same for malloc and free.”

3. C & C++ • Why are these languages used? • performance • they give you precise control over system resources • Game programmers need to know: • how to implement game systems in these languages • how to employ the performance advantages of these languages • Note: this is a career-long learning process

4. Constrained systems • Fixed resources for developer to work with • memory • disk space • Consoles, cell phones, handheld platforms (i.e. Nintendo DS), etc. • PCs have more resources (they cost more too) • but of course they can do many things

5. An Example of Capabilities • Xbox 360 • 512 MB RAM • PlayStation 2: • 32 MB RAM • PlayStation 3: • 256 MB RAM • 256 MB Video RAM • PSP • 32 MB RAM • Nintendo Wii • 512 MB RAM Source: CNET Reviews & Specs

6. But we have tons of memory now • As memory grows so do the demands on it • designers & artists want more data Computing Power

7. What is a memory budget? • A memory usage limit for the game • Typically divided among game subsystems: • graphics, physics, sound, AI, networking, etc. • Budgets can be hard or soft • soft means systems can request more memory larger than their budgets allow • memory management system may give permission

8. Memory Management Systems • To enforce a limit, you must know your game’s usage • Most games have their own memory management system • allocates memory • deallocates memory • compiles statistics • enforces rules/budgets • 2 Options: • for development only (debug mode) • for live system (release mode as well)

9. Developing on a state of the art PC • Pros: • faster development time • less interference from system errors • Cons: • false negatives • error doesn’t happen to you, but will happen to others • superior system may be good at overcoming flaws • Console rule of thumb: • the game will play completely different on a console than on the PC developing for it

10. The Console Developer’s Obsession • Stay under budget • What is: • the memory footprint of the application? • acceptable before problems will occur? • PC game programmers need to be concerned with memory too • it’s just console programmers take it a few steps further

11. Staying under budget • Approaches we’ll look at: • no memory leaks (or dangling references of course) • minimize data when possible • recycle data when possible • fix data structure sizes when possible • minimize dynamic memory allocation • pre-compute when possible • compress data when appropriate • yell at artists, audio people, & game designers

12. Staying under budget • Building a memory management system that: • optimizes memory usage • minimizes fragmentation • maximizes cache hit ratios • maintains statistics for analysis

13. No Memory Leaks • What’s a memory leak? • a program allocates memory for an object or struct, but when done with it, doesn’t de-allocate it • Why is that bad? • a program’s memory footprint will grow & grow & grow • results: • slower, slower, & slower performance • eventual crash • Good rule of thumb: • if you put something on the heap, you have to take it off

14. new & delete • new creates an object on the heap and returns a pointer to it • allocates a block of memory the size of: • the object’s data • plus object header info • delete frees the memory on the heap referenced by a pointer • invokes object’s deconstructor • C++ rule of thumb: • when you add a statement with new, also add your delete statement at the appropriate place • when you won’t need that object anymore

15. Constructors & Destructors • Constructors • called when an object is created with new • initializes instance variables • Destructors • called when an object is destroyed with delete • destroys what’s at the end of instance variable pointers if necessary. If necessary? • if no one else still needs them

16. malloc & free • C methods for managing blocks of memory • void *malloc(size_t size); • void free(void *pointer); • malloc allocates blocks on the heap • new calls malloc • free releases blocks on the heap • delete calls free

17. * & new goes on the heap • We can create an object/struct/array on the stack • Ex: void myMethod() { RECT rect1; RECT *rect2 = new RECT(); … } • rect1 will be popped off the stack when the method returns • rect2 needs to be deleted unless given to another object to be used as an instance variable

18. Common Sources of Memory Leaks • Construct an object in a method, forget to delete it before the method ends • An object has a pointer instance variable: • when you assign a new object, you forget to delete the old one OR • when you delete the object, you forget to delete the object instance variable • You have an STL container of objects: • when you want to remove or replace the contents, you forget to delete the old contents

19. Constructing * objects in a method • Good practice: • whenever you write new, decide where to put delete

20. Setting a * instance variable • When you set a *, you must: • first delete the old one • then assign the new one • Typically done inside a single set method

21. Deleting *objects from an STL container • Iterate through the container • For each item: • get the object • move the iterator onto the next item • remove the object from the container • delete the object

22. Detecting Memory Leaks • Easiest way, open the Windows Task Manager • while the program is running, is memory increasing? • Better ways exist: • Use VS _Crt libraries and add hook for notification of any memory allocation changes • Override new & delete and count memory allocation/freeing • Manage memory yourself (We’ll look at these approaches too)

23. Beware the Dangling Reference • When memory is freed up but someone still wants to use it • causes a fatal error • Common causes: • you delete a * in an STL before removing it • you construct a * and give it to set method, then delete it, thinking it was a temporary variable

24. Remember this? • Approaches we’ll look at today: • no memory leaks (or dangling references of course) • minimize data when possible • pre-compute when possible • recycle data when possible • fix data structure sizes when possible • minimize dynamic memory allocation • compress data when appropriate • yell at artists, audio people, & game designers

25. Minimize data when possible • Carefully choose data types • sometimes means trading computation for memory • How could we improve our RenderItems? • an int is 4 bytes, max of 2,147,483,647 • a 40X40 world with 64X64 tiles has a max coordinate of 2650 • Do we really need something that big? • How about making x & y shorts? • Console developer might take it a step further • a custom primitive

26. Minimizing data via a custom primitive • A single number to store data for multiple instance variables • this is for memory extremists • 2650 can be fit into 12 bits, so: • x & y can be fit into 24 bits • we could store both in a single int • we then have 8 bits left to store something else: • alpha fits perfectly, 8 bits • We would now need methods for adding and retrieving data

27. A custom primitive for RenderItem’s x,y,a const unsigned int CLEAR_X = 1048575; const unsigned int CLEAR_Y = 4293918975; const unsigned int CLEAR_A = 4294967040; class RenderItem { private: unsigned int x_y_a; … public: RenderItem() { x_y_a = 0; } … // GET & SET METHODS

28. void setX(short initX) { unsigned int xAsInt = initX; xAsInt <<= 20; x_y_a &= CLEAR_X; x_y_a |= xAsInt; } void setY(short initY) { unsigned int yAsInt = initY; yAsInt <<= 8; x_y_a &= CLEAR_Y; x_y_a |= yAsInt; } void setA(byte initA) { x_y_a &= CLEAR_A; x_y_a |= initA; }

29. short getX() { unsigned int x = x_y_a; x >>= 20; return (short)x; } short getY() { unsigned int y = x_y_a; y &= CLEAR_X; y >>= 8; return (short)y; } byte getA() { unsigned int a = x_y_a; a &= CLEAR_X; a &= CLEAR_Y; return (byte)a; } };

30. Pre-compute when possible • Games are approximations • they just need to be fun • Game programmers love constants computed offline: const unsigned int CLEAR_X = 1048575; const unsigned int CLEAR_Y = 4293918975; const unsigned int CLEAR_A = 4294967040; • These are bit strings representing: • 11111111111100000000000011111111 • 00000000000011111111111111111111 • 11111111111111111111111100000000

31. Recycle data when possible • How could we improve our particle system management? • When a particle’s life has expired, rather than delete it from the data structure, reuse it for another particle • How? • give each Particle a bool alive instance variable • only render those in the list that are alive • when time to add a new particle, pick one that’s not alive, fill it with data, and bring it back to life

32. Fix data structure sizes when possible • How could we improve our rendering? • determine the maximum number of render items we will allow • construct a data structure with that many RenderItem objects • don’t let game conditions need more than the max • Each frame: • fill in RenderItems in list at index 0 – (N-1), where N is our counter • Render items in list at indices 0 – (N-1)

33. Unused data structures vs. dynamic memory allocation • Generally speaking, dynamic memory allocation is to be avoided when possible • Why? • computationally expensive for memory management system • increases potential for memory leaks • memory fragmentation

34. Compress data when possible • Use algorithms to reduce footprint • we saw this with the last example • Ex: • Data Compression • RLE – Run Length Encoding • LZW – Lempel Ziv Welch • Fixed Point Math • for systems with no floating-point unit

35. RLE • Lossless compression algorithm • Replace long sequences of identical data with a single piece of data and a count • Ex: Background Tile Repetition • suppose we wanted to store a tiled layer character sequence: • 0,0,0,0,0,0,0.0,0.0,1c,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 • Could be replaced by 10(0),1(1c),16(0) • 52 characters replaced by 17 • Many variations on this theme • commonly used in constrained systems

36. Lempel Ziv Welch • Lossless algorithm • Replaces strings of characters with a code (number) • similar to RLE • Adds new strings to a table of strings • Longer strings are built from smaller string codes

37. http://marknelson.us/1989/10/01/lzw-data-compression/

38. Yell at artists, audio people, & game designers • I’m kidding, I’m kidding • Game programmers should know their system’s true limitations • Game designers & artists: • may not be fully aware of these limits • will try to push these limits • An alternative to employing the size of data is to reduce the amount of data • Common strategy for game engines: • maximum triangle counts per mesh • maximum sizes for art, sound, music • etc.

39. Detecting Memory Leaks • Easiest way, open the Windows Task Manager • while the program is running, is memory increasing? • More accurate ways exist: • Use VS _Crt libraries and add hook for notification of any memory allocation changes • Override new & delete and count memory allocation/freeing • Manage memory yourself

40. Windows Memory Methods • Methods starting with _ are only built (and thus executed) when in debug mode • Windows has a series of _CrtXXX methods for analyzing memory allocation • What for? • detecting memory leaks • Uses callback methods • You tell windows the callback method to call whenever memory is allocated or de-allocated: _CrtSetAllocHook(MyCustomAllocHook); • You then define a response in this method

41. _CrtSetAllocHook • What kind of response might we program? • We could count the number of objects constructed & deconstructed • If the gap between them rises, we likely have a memory leak • Drawbacks to this technique: • size of allocation is available • size of de-allocation is not • Windows-specific methods (not portable)

42. int MyCustomAllocHook( int nAllocType, void *userData, size_t size, int nBlockType, long requestNumber, const unsigned char *filename, int lineNumber) { if( nBlockType == _CRT_BLOCK) return TRUE ; // better to not handle switch(nAllocType) { case _HOOK_ALLOC : objectConstructed++; // add the code for handling // the allocation requests break ; case _HOOK_FREE : objectDestructed++; // add the code for handling // the free requests break ; }… NOTE: These are our counting variables Ref: http://www.codeproject.com/KB/cpp/MLFDef.aspx

43. Overriding new & delete • new & delete are C++ operators • That means they can be overridden • How? • redefine them globally so all classes use your definition • Headers: • void* operator new (size_t size) {… • void* operator new[] (size_t size) {… • void operator delete (void *p) {… • void operator delete[] (void *p) {… • What is void*? • address of the object we are constructing or freeing

44. What do new & delete do? • Very simple stuff • They call malloc & free • C memory allocation & de-allocation methods • Result is system dependant, but for all systems: • malloc: • sets aside memory on the heap equal to the size of the object • returns a pointer to that memory address • free: • releases a memory address • What’s the sizeof an object? • the sum of the sizes of its instance variables

45. Visual Studio’s new & delete void* operator new (size_t size) { void *p=malloc(size); if (p==0) // did malloc succeed? throw std::bad_alloc(); // ANSI/ISO compliant behavior return p; } void operator delete (void *p) { free(p); } Ref:

46. Our new & delete void* operator new (size_t size) { void *p=our_malloc(size); if (p==0) // did malloc succeed? throw std::bad_alloc(); // ANSI/ISO compliant behavior return p; } void operator delete (void *p) { our_free(p); } • NOTE: We would then use the same code for the array versions of new & delete

47. What are our malloc & free going to do? • Same as before. How? • we’ll call C’s malloc and free from them • Problem: • delete doesn’t get size of memory being freed • Solution: • when mallocing, add a little extra memory for some header info • when freeing, extract header info • what kind of info? • size of allocation • checksum for error checking

48. Time to practice pointer arithmetic • What’s that? • Feature of C language • If you have a pointer char *text, it points to memory address storing text • text + 1 points to 1 byte after start of text • might be second character • might not • you need to know what you’re moving your pointer to • Why do we care? • we need to stick our info at the front of our object

49. malloc_info • Our header • we’ll stick in in front of each piece of data we allocate #define MALLOC_CHECKSUM 123456789 struct malloc_info { int checksum; size_t bytes; }; Ref: Grad Student Rick Spillane: http://www.fsl.cs.sunysb.edu/~rick/

50. void* our_malloc(size_t bytes) { void *data; struct malloc_info *info; data = malloc(bytes + sizeof(struct malloc_info)); if (!data) return NULL; else { size_t headerSize = sizeof(struct malloc_info); totalAlloc += bytes + headerSize; dataAlloc += bytes; info = (struct malloc_info *)data; info->checksum = MALLOC_CHECKSUM; info->bytes = bytes; char *data_char = (char*)data; data_char += headerSize; return (void*)data_char; } }