140 likes | 266 Views
Bulk Synchronous Parallel Computing. Trevor Schaub Jim Sellers. This presentation was prepared for Professor Stefan Dobrev in partial fulfillment of the requirements for the course CSI 4 14 0. Outline. Mo del description Why it is interesting/important Relationship to real life
E N D
Bulk Synchronous Parallel Computing Trevor Schaub Jim Sellers This presentation was prepared for Professor Stefan Dobrev in partial fulfillment of therequirements forthe course CSI 4140
Outline • Model description • Why it is interesting/important • Relationship to real life • Some basic techniques, simple sample program to demonstrate how the approach to algorithm design differs from PRAM and MPP
Why do we need change? • Diverse parallel computers and models • Shared mem. vs. message passing, no clear winner • Use network locality vs. don’t • Conclusion: parallel code is often computer specific. Need unifying model
Why BSP? • Provides consistent, general framework to develop scaleable software • Similar to OO – a way of thinking and writing programs (paradigm) • Not system specific • Proceed in “supersteps” • Decouples communication and synchronization • The deadlock problem is minimized
What is BSP? • BSP is a process of dividing a task into steps called supersteps • Each process performs the task by doing the supersteps in order • The fundamental concept is the superstep itself, and it’s structure
What’s a superstep? • Processes perform as much as possible using local data • Communication and synchronization • Move on to next superstep superstep …
Dissection of a superstep Threads/Processes T I M E Local computation Global Communication Synchronization
Model description • What are the benefits of a superstep? • Easier debugging • Can look at supersteps in isolation • Removes problem of deadlock • Separate computation from communication • Allows reasoning of the correctness of the code nearly as easy as sequential code. • Easier shift for programmers used to sequential programming
Relationship to real life • Is BSP in use today? • Yes, sometimes without even realizing it. • When should you use BSP? • Becomes more useful with more communication • Benefits vary with the problem
Basic techniques • Identify the sections that can be that need communication or synchronization • Modify program so that sync. happens after communication • Separate ones are grouped where possible • Before each communication group there is a global barrier • The code between the end of each communication group becomes a superstep
Example: • See code example
References • http://approximity.com/papers/ptfopt/node31.html • http://www.cs.ucf.edu/csdept/faculty/goudreau/cop5937_fall96/COP5937.html • http://web.comlab.ox.ac.uk/oucl/research/highlights/bsp_computing.html • http://www.math.uu.nl/people/bisselin/pas1_prev.html • http://www.math.ruu.nl/people/bisselin/software.html • http://web.comlab.ox.ac.uk/oucl/work/bill.mccoll/oparl.html • http://www.cs.hmc.edu/courses/2001/spring/cs156/html12/slides12.pdf • http://www.cag.lcs.mit.edu/bayanihan/papers/javapdc99/html/node2.html • http://www.byte.com/art/9611/sec5/art5.htm