1 / 129

CS1760 Multiprocessor Synchronization

Explore principles and practices of multicore programming, concurrency jargon, parallel primality testing, load balancing, and shared counters in a multiprocessor system. Learn about synchronization, models, algorithms, and real-world programming complexities. Follow a roadmap focusing on correctness and efficiency in parallel computation. Discover the impact of Moore's Law evolution and multicore processors, including the challenges of scaling and exploiting concurrency in software development.

wmh
Download Presentation

CS1760 Multiprocessor Synchronization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CS1760 Multiprocessor Synchronization

  2. Staff Maurice Herlihy (instructor) Daniel Engel (grad TA) Jonathan Lister (HTA) BhrathKayyer (UTA) Art of Multiprocessor Programming

  3. Grading 8 Homeworks (40%) 5 programming assignments (20%) 3 Midterms (40%) Art of Multiprocessor Programming

  4. Collaboration Permitted talking about the homework problems with other students; using other textbooks; using the Internet. NOT Permitted obtaining the answer directly from anyone or anything else in any form. Art of Multiprocessor Programming

  5. Capstone Yes, you can take this course as a capstone course Only one project possible (concurrent packet filter) Requires reading ahead of the course See web page for details Art of Multiprocessor Programming

  6. See Course Web Page for … TA Hours Piazza Other important matters Art of Multiprocessor Programming

  7. Moore’s Law Transistor count still rising Clock speed flattening sharply Art of Multiprocessor Programming 7

  8. Moore’s Law (in practice) Art of Multiprocessor Programming

  9. Extinct: the Uniprocesor cpu memory Art of Multiprocessor Programming 9

  10. Extinct: The Shared Memory Multiprocessor(SMP) cache cache cache Bus Bus shared memory Art of Multiprocessor Programming 10

  11. The New Boss: The Multicore Processor(CMP) Sun T2000 Niagara All on the same chip cache cache cache Bus Bus shared memory Art of Multiprocessor Programming 11

  12. From the 2008 press… …Intel has announced a press conference in San Francisco on November 17th, where it will officially launch the Core i7 Nehalem processor… …Sun’s next generation Enterprise T5140 and T5240 servers, based on the 3rd Generation UltraSPARC T2 Plus processor, were releasedtwo days ago… Art of Multiprocessor Programming 12

  13. Why is Kunle Smiling? Niagara 1 Art of Multiprocessor Programming 13

  14. Why do we care? Time no longer cures software bloat The “free ride” is over When you double your program’s path length You can’t just wait 6 months Your software must somehow exploit twice as much concurrency Art of Multiprocessor Programming 14

  15. Traditional Scaling Process 7x Speedup 3.6x 1.8x User code Traditional Uniprocessor Time: Moore’s law Art of Multiprocessor Programming 15

  16. 7x 3.6x Speedup 1.8x Ideal Scaling Process User code Multicore Unfortunately, not so simple… Art of Multiprocessor Programming 16

  17. Actual Scaling Process Speedup 2.9x 2x 1.8x User code Multicore Parallelization and Synchronization require great care… Art of Multiprocessor Programming 17

  18. Multicore Programming:Course Overview Fundamentals Models, algorithms, impossibility Real-World programming Architectures Techniques Art of Multiprocessor Programming 18

  19. Sequential Computation thread memory object object Art of Multiprocessor Programming 19

  20. Concurrent Computation threads memory object object Art of Multiprocessor Programming 20

  21. Asynchrony Sudden unpredictable delays Cache misses (short) Page faults (long) Scheduling quantum used up (really long) Art of Multiprocessor Programming 21

  22. Model Summary Multiple threads Sometimes called processes Single shared memory Objects live in memory Unpredictable asynchronous delays Art of Multiprocessor Programming 22

  23. Road Map We are going to focus on principles first, then practice Start with idealized models Look at simplistic problems Emphasize correctness over pragmatism “Correctness may be theoretical, but incorrectness has practical impact” Art of Multiprocessor Programming 23

  24. Concurrency Jargon Hardware Processors Software Threads, processes Sometimes OK to confuse them, sometimes not. Art of Multiprocessor Programming 24

  25. Parallel Primality Testing Challenge Print primes from 1 to 1010 Given Ten-processor multiprocessor One thread per processor Goal Get ten-fold speedup (or close) Art of Multiprocessor Programming 25

  26. Load Balancing Split the work evenly Each thread tests range of 109 1 1010 109 2·109 … … P0 P1 P9 Art of Multiprocessor Programming 26

  27. Procedure for Thread i voidprimePrint { inti = ThreadID.get(); // IDs in {0..9} for(j = i*109+1, j<(i+1)*109; j++) { if(isPrime(j)) print(j); } } Art of Multiprocessor Programming 27

  28. Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads Uneven Hard to predict Art of Multiprocessor Programming 28

  29. Issues Higher ranges have fewer primes Yet larger numbers harder to test Thread workloads Uneven Hard to predict Need dynamic load balancing rejected Art of Multiprocessor Programming 29

  30. Shared Counter 19 each thread takes a number 18 17 Art of Multiprocessor Programming 30

  31. Procedure for Thread i int counter = new Counter(1); voidprimePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } } Art of Multiprocessor Programming 31

  32. Procedure for Thread i Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } } Shared counter object Art of Multiprocessor Programming 32

  33. Where Things Reside cache cache cache Bus Bus voidprimePrint { inti = ThreadID.get(); // IDs in {0..9} for(j = i*109+1, j<(i+1)*109; j++) { if(isPrime(j)) print(j); } } Local variables code shared memory 1 shared counter Art of Multiprocessor Programming 33

  34. Procedure for Thread i Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j = counter.getAndIncrement(); if (isPrime(j)) print(j); } } Stop when every value taken Art of Multiprocessor Programming 34

  35. Procedure for Thread i Counter counter = new Counter(1); void primePrint { long j = 0; while (j < 1010) { j =counter.getAndIncrement(); if (isPrime(j)) print(j); } } Increment & return each new value Art of Multiprocessor Programming 35

  36. Counter Implementation public class Counter{ private long value; public long getAndIncrement() { return value++; } } Art of Multiprocessor Programming 36

  37. Counter Implementation public class Counter { private long value; public longgetAndIncrement() { return value++; } } OK for single thread, not for concurrent threads Art of Multiprocessor Programming 37

  38. What It Means public class Counter { private long value; public longgetAndIncrement() { return value++; } } Art of Multiprocessor Programming 38

  39. What It Means public class Counter { private long value; public long getAndIncrement() { return value++; } } temp = value; value = temp + 1; return temp; Art of Multiprocessor Programming 39

  40. Not so good… time Value… 1 2 3 2 read 1 write 2 read 2 write 3 read 1 write 2 Art of Multiprocessor Programming 40

  41. Is this problem inherent? !! !! write read read write If we could only glue reads and writes together… Art of Multiprocessor Programming 41

  42. Challenge public class Counter { private long value; public longgetAndIncrement() { temp = value; value = temp + 1; return temp; } } Art of Multiprocessor Programming 42

  43. Challenge public class Counter { private long value; public long getAndIncrement() { temp = value; value = temp + 1; return temp; } } Make these steps atomic (indivisible) Art of Multiprocessor Programming 43

  44. Hardware Solution public class Counter { private long value; public long getAndIncrement() { temp = value; value = temp + 1; return temp; } } ReadModifyWrite() instruction Art of Multiprocessor Programming 44

  45. An Aside: Java™ public class Counter { private long value; public longgetAndIncrement() { synchronized{ temp = value; value = temp + 1; } return temp; } } Art of Multiprocessor Programming 45

  46. An Aside: Java™ public class Counter { private long value; public long getAndIncrement() { synchronized{ temp = value; value = temp + 1; } return temp; } } Synchronized block Art of Multiprocessor Programming 46

  47. An Aside: Java™ public class Counter { private long value; public long getAndIncrement() { synchronized { temp = value; value = temp + 1; } return temp; } } Mutual Exclusion Art of Multiprocessor Programming 47

  48. Mutual Exclusion,or “Alice & Bob share a pond” A B Art of Multiprocessor Programming 48

  49. Alice has a pet A B Art of Multiprocessor Programming 49

  50. Bob has a pet A B Art of Multiprocessor Programming 50

More Related