Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney

1. The Ordering Requirements of Relativistic andReader-Writer Locking Approaches to Shared Data Access Philip W. Howard with Jonathan Walpole, Josh Triplett, Paul E. McKenney

2. Outline • The Problem • The RWL Solution • The RP Solution • Other problems and their solutions • Multiple Writers • Performance

3. The Problem A B C A C

4. The Problem Reader 1 obtain ref drop ref Writer unlink reclaim obtain ref drop ref Reader 2

5. void init(stuff *node) { node->a = COMPUTE_A; node->b = COMPUTE_B; node->initd = TRUE; } while (!node->initd) {} if (node->a) drop_nuke(); if (node->b) drop_napalm(); The Problem

6. void init(stuff *node) { int value; value = some_computation; node->a = COMPUTE_A; node->b = COMPUTE_B; node->initd = value; } The Problem

7. void init(stuff *node) { node->a = COMPUTE_A; node->b = COMPUTE_B; node->initd = TRUE; } while (!node->initd) {} if (node->a) drop_nuke(); if (node->b) drop_napalm(); The Problem

8. The Problem • Compiler reordering • CPU reordering • Memory System reordering

9. How are these dependencies maintained? Reader 1 obtain ref drop ref Writer unlink reclaim obtain ref drop ref Reader 2

10. Thread 1 A: a=1; Thread 2 B: if (a) A → B else B → A How does → work?

11. Thread 1 A: a=1; mb(); C: c=1; Thread 2 while (!a) B: A B How does  work? Thread 3 if (c) D: A D

12. With Reader-Writer Locks read-lock obtain ref drop ref read-unlock write-lock unlink reclaim write-unlock

13. With Reader-Writer Locks read-lock obtain ref drop ref read-unlock write-lock unlink reclaim write-unlock

14. With Reader-Writer Locks write-lock unlink reclaim write-unlock read-lock obtain ref drop ref read-unlock

15. Locking primitives must • Impose the semantics of the lock • Prevent compiler, CPU, or Memory system from reordering operations across them

16. With Relativistic Programming start-read obtain ref drop ref end-read unlink start-wait end-wait reclaim

17. With Relativistic Programming start-read obtain ref drop ref end-read unlink start-wait end-wait reclaim

18. RP primitives must • Impose the semantics of wait-for-readers • Prevent compiler, CPU, or Memory system from reordering operations across them

19. Outline • The Problem • The RWL Solution • The RP Solution • Other problems and their solutions • Insert • Move Down • Move Up • General Case • Multiple Writers • Performance

20. Insert C E C D E

21. Insert Reader 1 obtain ref deref Writer init link obtain ref deref Reader 2

22. Thread 1 A: a=1; mb(); C: c=1; Thread 2 while (!a) B: A B How does  work? Thread 3 if (c) D: A D

23. Insert Use rp-publish to perform link operation

24. Move Down H H F D D G B F B E A C E G A C

25. Move Down H H H F F D G D D B E B F’ B F’ A C A C E G A C E G • init F’ • link F’ • unlink F

26. Move Down init(F’) link(F’) unlink(F) reclaim(F) deref(H) deref(F) deref(D) deref(E) H H H F F D D D G B B F’ F’ E B C G A A C E A C E G

27. Move Down init(F’) link(F’) unlink(F) reclaim(F) deref(H) deref(F) deref(D) deref(F’) deref(E) H H H F F D D D G B B F’ F’ E B C G A A C E A C E G

28. Move Down init(F’) link(F’) unlink(F) reclaim(F) deref(H) deref(D) deref(F’) deref(E) H H H F F D D D G B B F’ F’ E B C G A A C E A C E G

29. RBTree Delete with Swap (Move Up) F F F B C’ B C’ A E E A E A C C D D null null D

30. Move Up deref(F) deref(C’) init(C’) link(C’) unlink(C) reclaim(C) F F F B C’ B C’ A E E A E A C C D D null null D

31. Move Up init(C’) link(C’) unlink(C) reclaim(C) deref(F) deref(B) deref(E) deref(C) F F F B C’ B C’ A E E A E A C C D D null null D

32. The General Case • Mutable vs. Immutable data

33. The General Case for Writers • Copy nodes to update immutable data or to make a collection of updates appear atomic • Use rp-publish to update mutable data • Use wait-for-readers when updates are in traversal order A B C

34. The General Case for Readers • Use rp-read for reading mutable data • Only dereference mutable data once if (node->next->key == key) { return node->next->value; }

35. Outline • The Problem • The RWL Solution • The RP Solution • Other problems and their solutions • Multiple Writers • Performance

36. Multiple Writers W R2 R1

37. Multiple Writers W1 W2 Reader

38. The phone company Can’t wait to ditch this phone Where’s my phone book? Can’t wait to get my new phone

39. Multiple Writers W1 W2 Reader wait-for-readers W1 W2 Reader

40. W2 W1 RP Reader W2 W1 TORP Reader W2 W1 reader pref Reader W2 W1 writer pref Reader RP vs RWL delays

41. Trigger Events RP RWLR RWLW

42. Outline • The Problem • The RWL Solution • The RP Solution • Other problems and their solutions • Multiple Writers • Performance

43. Thread 1 A: a=1; mb(); C: c=1; Thread 2 while (!a) B: A B How does  work? Thread 3 if (c) D: A D

44. Reader-Writer Locks read-lock read-unlock write-lock write-unlock read-lock read-unlock

45. Reader while (writing) {} reading=1; Writer while (reading) {} writing=1; Reader-Writer Locks

46. Relativistic Programming start-read end-read start-wait end-wait start-read end-read

47. start-read(i) { reader[i]=1; } end-read(i) { reader[i]=0; } start-wait() {} end-wait() { for (i) { while (reader[i]) {} } } Relativistic Programming

48. start-read(i) { reader[i]=Epoch; } end-read(i) { reader[i]=0; } start-wait() { Epoch++; my_epoch = Epoch; } end-wait() { for (i) { while (reader[i] && reader[i] < my_epoch) {} } } Relativistic Programming

49. Performance Benefits to RP • Less expensive read-side primitives • Less serialization / more concurrency • Less waiting

50. Benchmarked Methods nolock No synchronization rp Relativistic Programming torp Totally Ordered RP rwlr Reader-Writer Lock Read preference rwlw Reader-Writer Lock Write preference