1 / 79

Barrier Synchronization

Understand the concept of barrier synchronization in multiprocessor systems. Learn about its implementation, advantages, and common challenges.

pmcminn
Download Presentation

Barrier Synchronization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Barrier Synchronization Nir Shavit Multiprocessor Synchronization Spring 2003

  2. Ideal Parallel Computation 0 0 1 0 1 1 M. Herlihy & N. Shavit (c) 2003

  3. Ideal Parallel Computation 2 2 1 2 1 1 M. Herlihy & N. Shavit (c) 2003

  4. Real-Life Parallel Computation 0 0 1 0 zzz… 1 M. Herlihy & N. Shavit (c) 2003

  5. Real-Life Parallel Computation 2 zzz… 1 1 0 Uh, oh M. Herlihy & N. Shavit (c) 2003

  6. Barrier Synchronization 0 0 barrier 0 M. Herlihy & N. Shavit (c) 2003

  7. Barrier Synchronization barrier 1 1 1 M. Herlihy & N. Shavit (c) 2003

  8. Barrier Synchronization barrier Until every thread has left here No thread enters here M. Herlihy & N. Shavit (c) 2003

  9. Why Do We Care? • Mostly of interest to • Scientific & numeric computation • Elsewhere • Garbage collection • Rare in systems programming M. Herlihy & N. Shavit (c) 2003

  10. Barriers public class Barrier { int count; int size; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} M. Herlihy & N. Shavit (c) 2003

  11. Barriers public class Barrier { int size; int count; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} Number threads participating M. Herlihy & N. Shavit (c) 2003

  12. Barriers public class Barrier { int size; int count; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} Number threads not yet arrived M. Herlihy & N. Shavit (c) 2003

  13. Barriers public class Barrier { int size; int count; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} Initialization M. Herlihy & N. Shavit (c) 2003

  14. Barriers public class Barrier { int size; int count; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} The method M. Herlihy & N. Shavit (c) 2003

  15. Barriers public class Barrier { int size; int count; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} If I’m last, reset everything M. Herlihy & N. Shavit (c) 2003

  16. Barriers public class Barrier { int size; int count; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} Wait for rest to catch up M. Herlihy & N. Shavit (c) 2003

  17. Barriers public class Barrier { int count; int size; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} What’s wrong with this protocol? M. Herlihy & N. Shavit (c) 2003

  18. Reuse Barrier b = new Barrier(n); while ( mumble() ) { work(); b.await() } Do work repeat synchronize M. Herlihy & N. Shavit (c) 2003

  19. Barriers public class Barrier { int count; int size; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} Waiting for Phase 1 to finish M. Herlihy & N. Shavit (c) 2003

  20. Barriers public class Barrier { int count; int size; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} Phase 1 is so over Waiting for Phase 1 to finish M. Herlihy & N. Shavit (c) 2003

  21. Barriers public class Barrier { int count; int size; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchDec()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} Prepare for phase 2 ZZZZZ…. M. Herlihy & N. Shavit (c) 2003

  22. Barriers public class Barrier { int count; int size; public Barrier(int n){ this.size = this.count = n; } public void await() { if (count.FetchInc()==1) { this.count = this.size; } else { while (this.count != 0) {} }}}} Waiting for Phase 2 to finish Waiting for Phase 1 to finish M. Herlihy & N. Shavit (c) 2003

  23. Basic Problem • One thread “wraps around” to start phase 2 • While another thread is still waiting for phase 1 • One solution: • Always use two barriers M. Herlihy & N. Shavit (c) 2003

  24. Sense-Reversing Barriers public class Barrier { int count, size; boolean sense = true; public void await(boolean mySense) { if (count.FetchDec()==1) { this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} M. Herlihy & N. Shavit (c) 2003

  25. Sense-Reversing Barriers public class Barrier { int count, size; boolean sense = true; public void await(boolean mySense) { if (count.FetchDec()==1) { this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Completed odd or even-numbered phase? M. Herlihy & N. Shavit (c) 2003

  26. Sense-Reversing Barriers public class Barrier { int count, size; boolean sense = true; public void await(boolean mySense) { if (count.FetchDec()==1) { this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Thread working on odd or even-numbered phase? M. Herlihy & N. Shavit (c) 2003

  27. Sense-Reversing Barriers public class Barrier { int count, size; boolean sense = true; public void await(boolean mySense) { if (count.FetchDec()==1) { this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} If I’m last, reverse the sense on the way out M. Herlihy & N. Shavit (c) 2003

  28. Sense-Reversing Barriers public class Barrier { int count, size; boolean sense = true; public void await(boolean mySense) { if (count.FetchDec()==1) { this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Otherwise, wait for sense to flip M. Herlihy & N. Shavit (c) 2003

  29. 2-barrier 2-barrier Combining Tree Barriers 2-barrier M. Herlihy & N. Shavit (c) 2003

  30. 2-barrier 2-barrier Combining Tree Barriers 2-barrier M. Herlihy & N. Shavit (c) 2003

  31. Combining Tree Barrier public class CBarrier { int count, size; CBarrier parent; public void await(boolean mySense) { if (this.count.FetchDec()==1){ if (this.parent != null) this.parent.await(); this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} M. Herlihy & N. Shavit (c) 2003

  32. Combining Tree Barrier public class CBarrier { int count, size; CBarrier parent; public void await(boolean mySense) { if (this.count.FetchDec()==1){ if (this.parent != null) this.parent.await(); this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Parent barrier in tree M. Herlihy & N. Shavit (c) 2003

  33. Combining Tree Barrier public class CBarrier { int count, size; CBarrier parent; public void await(boolean mySense) { if (this.count.FetchDec()==1){ if (this.parent != null) this.parent.await(); this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Thread working on odd or even-numbered phase? M. Herlihy & N. Shavit (c) 2003

  34. Combining Tree Barrier public class CBarrier { int count, size; CBarrier parent; public void await(boolean mySense) { if (this.count.FetchDec()==1){ if (this.parent != null) this.parent.await(); this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Am I last? M. Herlihy & N. Shavit (c) 2003

  35. Combining Tree Barrier public class CBarrier { int count, size; CBarrier parent; public void await(boolean mySense){ if (this.count.FetchDec()==1){ if (this.parent != null) this.parent.await(); this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Wait on parent barrier, if any M. Herlihy & N. Shavit (c) 2003

  36. Combining Tree Barrier public class CBarrier { int count, size; CBarrier parent; public void await(boolean mySense){ if (this.count.FetchDec()==1){ if (this.parent != null) this.parent.await(); this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Prepare for next phase M. Herlihy & N. Shavit (c) 2003

  37. Combining Tree Barrier public class CBarrier { int count, size; CBarrier parent; public void await(boolean mySense) { if (this.count.FetchDec()==1){ if (this.parent != null) this.parent.await(); this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} Wake up others at this level M. Herlihy & N. Shavit (c) 2003

  38. Combining Tree Barrier public class CBarrier { int count, size; CBarrier parent; public void await(boolean mySense) { if (this.count.FetchDec()==1){ if (this.parent != null) this.parent.await(); this.count = this.size; this.sense = mySense } else { while (this.sense != mySense) {} }}}} I’m not last, wait for release M. Herlihy & N. Shavit (c) 2003

  39. Combining Tree Barrier • No sequential bottleneck • FetchDec calls proceed in parallel • Low memory contention • Same reason • Cache behavior • Local spinning on bus-based architecture • Not so good for distributed M. Herlihy & N. Shavit (c) 2003

  40. Remarks • Everyone spins on sense field • Local spinning on bus-based (good) • Network hot-spot on distributed architecture (bad) • Sequential bottleneck • Sequence of FetchInc() calls • Not really scalable M. Herlihy & N. Shavit (c) 2003

  41. Tournament Tree Barrier • If tree nodes have fan-in 2 • Don’t need to call FetchDec() • Winner chosen statically • At level i • If i-th bit of id is zero, move up • Otherwise keep back M. Herlihy & N. Shavit (c) 2003

  42. Tournament Tree Barriers winner loser winner loser winner loser M. Herlihy & N. Shavit (c) 2003

  43. Tournament Tree Barriers All flags blue M. Herlihy & N. Shavit (c) 2003

  44. Tournament Tree Barriers Loser thread sets winner’s flag M. Herlihy & N. Shavit (c) 2003

  45. Tournament Tree Barriers Loser spins on own flag M. Herlihy & N. Shavit (c) 2003

  46. Tournament Tree Barriers Winner spins on own flag M. Herlihy & N. Shavit (c) 2003

  47. Tournament Tree Barriers Winner sees own flag, moves up, spins M. Herlihy & N. Shavit (c) 2003

  48. Tournament Tree Barriers Bingo! M. Herlihy & N. Shavit (c) 2003

  49. Tournament Tree Barriers Sense-reversing: next time use blue flags M. Herlihy & N. Shavit (c) 2003

  50. Tournament Barrier class TBarrier { boolean flag; TBarrier partner; TBarrier parent; boolean top; … } M. Herlihy & N. Shavit (c) 2003

More Related