1 / 48

Debugging Programs that use Atomic Blocks and Transactional Memory

Debugging Programs that use Atomic Blocks and Transactional Memory. Ferad Zyulkyarov 1,2 , Tim Harris 3 , Osman S. Unsal 1 , Adrián Cristal 1 , Mateo Valero 1,2. 1 BSC-Microsoft Research Centre 2 Universitat Politècnica de Catalunya 3 Microsoft Research Cambridge.

redford
Download Presentation

Debugging Programs that use Atomic Blocks and Transactional Memory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Debugging Programs that use Atomic Blocks and Transactional Memory Ferad Zyulkyarov1,2, Tim Harris3, Osman S. Unsal1, Adrián Cristal1, Mateo Valero1,2 1BSC-Microsoft Research Centre 2Universitat Politècnica de Catalunya 3Microsoft Research Cambridge 15th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming 09-14 January 2010 – Bangalore

  2. Our Motivation • It is difficult to debug transactional applications with existing debuggers • Existing debuggers are not aware of atomic blocks and transactional memory • Lessons learnt from developing complex TM applications such as Atomic Quake[1] and Quake TM [2]. [1] Zyulkyarov et al. “Atomic Quake: Using Transactional Memory in an Interactive Multiplayer Game Server“, PPoPP'09 [2] Gajinov et al. “QuakeTM: Parallelizing a Complex Serial Application Using Transactional Memory “, ICS'09

  3. Overview • Debugging atomic blocks atomically • Examining the TM state • Managing the TM state at debug-time • Conflict point discovery • Design and implementation

  4. Debugging at the Level of Atomic Blocks • Debugger is extended with the semantics of atomic blocks • Atomicity – atomic blocks are treated as single instruction • Isolation – the user does not observe intermediate results of concurrently running transactions • Hides the low level implementation details of atomic blocks (TM or lock inference)

  5. Atomicity in Debugging • Step over atomic blocks as if single instruction. • Good for debugging sync errors at granularity of atomic blocks vs. individual statements inside the atomic blocks. Current Debugger TM Aware Debugger <statement 1> <statement 2> <statement 3> atomic { <statement 4> <statement 5> <statement 6> <statement 7> } <statement 8> <statement 9> <statement 1> <statement 2> <statement 3> atomic { <statement 4> <statement 5> <statement 6> <statement 7> } <statement 8> <statement 9>

  6. Using Existing Debuggers SV_LinkEdict 1 atomic { 2 InitializeBoundingBox(ent); 3SV_FindTouchedLeafs(ent, sv.worldmodel->nodes); 4 node = FindNewLocation(ent, sv_areanode); 5InsertLinkBefore(&ent->area, &node->trigger_edicts); 6} // end atomic sv_areanode Conflict. Rollback. Re-execute. Compute the bounding box. Want to move from A to B and take the keys. Change the location and take the keys. Find the leafs the bounding box maps to. Another player took the keys before us. Find location B

  7. Using TM Aware Debugger SV_LinkEdict 1 atomic { 2 InitializeBoundingBox(ent); 3SV_FindTouchedLeafs(ent, sv.worldmodel->nodes); 4 node = FindNewLocation(ent, sv_areanode); 5InsertLinkBefore(&ent->area, &node->trigger_edicts); 6} // end atomic sv_areanode Moved from A to B and took the keys.

  8. Isolation in Debugging • What if we want to debug wrong code within atomic block? • Put breakpoint inside atomic block. • Validate the transaction • Step within the transaction. • The user does not observe intermediate results of concurrently running transactions • Switch transaction to irrevocable mode after validation. atomic { <statement 1> <statement 2> <statement 3> <statement 4> }

  9. Debugging Wrong Code Inside Atomic Blocks • Why isolation is important? InsertSorted(object value) { Node n = new Node(value); atomic { Node previous = FindPrevious(value); n.Next = previous.Next; previous.Next = n; } } 1 5 6

  10. Example Importance of Isolation InsertSorted(object value) { Node n = new Node(value); atomic { Node previous = FindPrevious(value); n.Next = previous.Next; previous.Next = n; } } InsertSorted(4) 4 1 5 6

  11. Example Importance of Isolation InsertSorted(object value) { Node n = new Node(value); atomic { Node previous = FindPrevious(value); n.Next = previous.Next; previous.Next = n; } } 4 5 6 1

  12. Example Importance of Isolation InsertSorted(object value) { Node n = new Node(value); atomic { Node previous = FindPrevious(value); n.Next = previous.Next; previous.Next = n; } } Another transaction changed the value to 3. The current TX will be doomed and operate on invalid data. 4 3 1 6

  13. Example Importance of Isolation InsertSorted(object value) { Node n = new Node(value); atomic { Node previous = FindPrevious(value); n.Next = previous.Next; previous.Next = n; } } Confuse the user because of consuming speculative values and not detecting conflict. 4 3 1 6

  14. Why Isolate Speculative State? InsertSorted(object value) { Node n = new Node(value); atomic { Node previous = FindPrevious(value); n.Next = previous.Next; previous.Next = n; } } Confuse the user because of consuming speculative values and not detecting conflict. 4 3 1 6

  15. Why Isolate Speculative State? InsertSorted(object value) { Node n = new Node(value); atomic { Node previous = FindPrevious(value); n.Next = previous.Next; previous.Next = n; } } 4 Conflict. Rollback. Re-execute. 3 1 6

  16. Debugging atomic blocks atomically • Examining the TM state • Managing the TM state at debug-time • Conflict point discovery • Design and implementation

  17. Debugging at the Level of Transactions • Assumes that atomic blocks are implemented with transactional memory. • Examine the internal state of the TM • Read/write set, re-executions, status • TM specific watch points • Break when conflict happens • Filters • Concurrent work with Herlihy and Lev [PACT’ 09].

  18. Querying the TM State Status: Active Re-executions: 1 atomic { <statement 1> <statement 2> <statement 3> <statement 4> } Query TM State Priority: 1 Nesting: 1 Write Set 0x30FE16 0x3AEE0D 0x4A1124 Read Set 0x30FA00 0x30FE16 0x320B0A 0x4A11F8

  19. TM Specific Watchpoints Filter: Break if Address = reservation@04 Thread = T2 Break when conflict happens AND atomic { <statement 1> <statement 2> <statement 3> <statement 4> } Conflict Information Conflicting Threads: T1, T2 Address: 0x84D2F0 Symbol: reservation@04 Readers: T1 Writers: T2

  20. Debugging atomic blocks atomically • Examining the TM state • Managing the TM state at debug-time • Conflict point discovery • Design and implementation

  21. Managing Transactions at Debug-Time • At the level of atomic blocks • Debug time atomic blocks • Splitting atomic blocks • At the level of transactions • Changing the state of TM system (i.e. adding and removing entries from read/write set, change the status, abort) • Analogous to the functionality of existing debuggers to change the CPU state

  22. Debug Time Atomic Blocks • Create and remove atomic blocks while debugging. • Useful to investigate and patch synchronization errors such as data races, atomicity, and order violation. • User marks the places that should execute atomically.

  23. Example Debug Time Atomic Bblocks <statement 1> <statement 2> <statement 3> <statement 4> <statement 5> <statement 6> <statement 7> <statement 8> <statement 9> <statement 10> <statement 11> <statement 12> <statement 13> <statement 14>

  24. Example Debug Time Atomic Bblocks <statement 1> <statement 2> <statement 3> StartDebugAtomic <statement 4> <statement 5> <statement 6> <statement 7> <statement 8> <statement 9> EndDebugAtomic <statement 10> <statement 11> <statement 12> <statement 13> <statement 14> User marks the start and the end of the transactions

  25. Data Race StartDebugAtomic atomic { local = counter; local++; } counter = local; EndDebugAtomic Thread 1 atomic { local = counter; local++; counter = local; } Thread 2 atomic { local = counter; local++; } counter = local;

  26. Split Atomic Block • Useful to find unnecessarily large atomic blocks or searching for data races atomic { s1; } atomic { s2; } atomic { s1; s2; } atomic { s1; <split> s2; }

  27. Debugging atomic blocks atomically • Examining the TM state • Managing the TM state at debug-time • Conflict point discovery • Design and implementation

  28. Conflict Point Discovery • TM applications have unanticipated overheads • Problem raised by Pankratius [talk at ICSE’09] and Rossbach et al. [PPoPP’10] • Tells the users at which source lines how many conflicts happen. • Useful to profile and optimize TM applications. • More comprehensive than Reach Points Gajinov et al. [ICS’ 09]

  29. Example Output

  30. Optimizing Genome

  31. Debugging atomic blocks atomically • Examining the TM state • Managing the TM state at debug-time • Conflict point discovery • Design and implementation

  32. Design Debugger Process Target Process WinDbg Program Implements debugger commands such as setting watchpoints, query and modify the STM. Show Read set Function call on the target process. Wrapper around the STM to hide its implementation details. DbgEng StmLib TmDbgExt TmTargetDbg

  33. Conclusion • New principles and approaches for debugging TM applications • Debugging at the level of atomic blocks • Debugging at the level of transactions • Managing transactions at debug-time • Conflict point discovery • Optimized Genome • Generic and decoupled design

  34. Край Slides available at www.bscmsrc.eu

  35. Backup Slides

  36. Implementing Conflict Point Discovery Log the address of a for conflict detection. Addr: 01 : StartAtomic(); 02 : OpenObjForWrite(a); 03 : a = 5; 04 : CommitAtomic(); Compiler Instrumentation atomic { a = 5; } To know where conflict happens log the return address of OpenObjForRead. Conflict Addr = 0x03 Translate to source line DbgEng Hashtable.cs:51

  37. Internal Breakpoint • Debugger breakpoint used to implement • Atomicity • Watchpoints • Splitting atomic blocks • Debug time atomic blocks

  38. Breakpoint-time Callback • Distinguishes ordinary breakpoints from internal breakpoints • Overrides the debugger behavior of suspending the target process • May execute complementary actions

  39. Example Use of Internal Breakpoint Stepping over atomic block as if a single instruction Internal Breakpoint atomic { <statement 1> <statement 2> <statement 3> <statement 4> <statement 5> <statement 6> <statement 7> <statement 8> } -> StartAtomic() -> Commit()

  40. Atomicity Violations initially a = b = 0; Thread 1 Thread 2 1 atomic{ 2 a++; 3 } atomic { 4 a++; 5 atomic{ } 6 b--; 7 assert(a + b == 0); 8 }

  41. Atomicity Violations • initially a = b = 0; • Thread 1 Thread 2 • StartDebugAomic • 1 atomic{ • 2 a++; • 3 } atomic { • 4 a++; • 5 atomic{ } • 6 b--; • 7 assert(a + b == 0); • } • EndDebugAtomic

  42. Extending the Scope of Atomic Blocks • We can build over debug time transactions to allow users to extend the scope of existing compile time atomic blocks. <statement 1> <statement 2> atomic { <statement 3> <statement 4> } <statement 5> <statement 6> <statement 7> <statement 1> <statement 2> atomic { <statement 3> <statement 4> <statement 5> <statement 6> } <statement 7> <statement 1> <statement 2> StartDebugTx <statement 3> <statement 4> <statement 5> <statement 6> EndDebugTx <statement 7> We want How we do

  43. Shrinking the Scope • The implementation of shrinking the scope of atomic blocks might be tricky for STMs because of the code instrumentation. <statement 1> <statement 2> atomic { <statement 3> <statement 4> <statement 5> <statement 6> } <statement 7> <statement 1> <statement 2> <statement 3> atomic { <statement 4> <statement 5> } <statement 6> <statement 7> <statement 1> <statement 2> StartTx - Ignore <statement 3> StartDebugTx <statement 4> <statement 5> EndDebugTx <statement 6> EndTx - Ignore <statement 7> We want How we do

  44. Debugging Wrong Code Inside Atomic Blocks? SV_LinkEdict 1 atomic { 2 InitializeBoundingBox(ent); 3SV_FindTouchedLeafs(ent, sv.worldmodel->nodes); 4 node = FindNewLocation(ent, sv_areanode); 5InsertLinkBefore(&ent->area, &node->trigger_edicts); 6} // end atomic InsertLinkBefore(object value) { Node n = new Node(value); atomic { AddNode(n); RightRotate(ParentOf(n)); FixColors(); } }

  45. Isolating Speculative State Insert(object value) { Node n = new Node(value); atomic { AddNode(n); RightRotate(ParentOf(n)); FixColors(); } } 5 5 Insert(3) 4 4 3 3

  46. Isolating Speculative State Another transaction changed the value to 2. The current TX will be doomed and operate on invalid data. Insert(object value) { Node n = new Node(value); atomic { AddNode(n); RightRotate(ParentOf(n)); FixColors(); } } 5 5 Insert(3) 4 3 2

  47. Isolating Speculative State Mislead the user because of consuming speculative values and not detecting conflict. Insert(object value) { Node n = new Node(value); atomic { AddNode(n); RightRotate(ParentOf(n)); FixColors(); } } 5 5 2 Rotate Right 3 3 2

  48. Isolating Speculative State Insert(object value) { Node n = new Node(value); atomic { AddNode(n); RightRotate(ParentOf(n)); FixColors(); } } 5 2 2 Fix Colors 3 3 5 Conflict. Rollback. Re-execute.

More Related