Parallel Computing Using Message-Passing (Our Chapter 6)

1. Parallel Computing Using Message-Passing(Our Chapter 6) • REFERENCE: Chapter 2 – 7 of Parallel Progamming by Barry Wilkinson and Michael Allen (WA for short). • While oriented towards cluster computing, Chapters 3-7 provide a useful collection of different types of parallel problems. • We will look at one or more problems in each chapter. • Provides an good introduction to MIMD computing. • Different models may have other solutions that are better. • Provides some good problem choices for a testbed for proposed models and architectures. • Other problem types need to be added to testbed. • E.g., problems that are considered difficult to parallelize. • The authors’ slides for these chapters will be posted in PDF at the course website. • Currently available at last year’s course website • Chapter 2 in WA on MPI and PVM: • MPI, a part of this chapter, was introduced earlier. • The author’s slides 76, 77, 79, 83, and 85-91 will be covered. These cover parts of Sections 2.2 and 2.3. • Some of Chapter 2 will be handed out and are part of your reading assignment. Chapter 6

2. Some use of the Big-O, , and  notation occurs in Chapters 2-8 and is reviewed in Subsection 2.3.2. • These are standard complexity concepts in basic algorithms that you should understand. • You should review any that you are unfamiliar with. • Chapter 3 in Wilkinson /Allen (or WA) covers Embarassingly/Pleasingly Parallel Computations • The main focus in this chapter is computing the Mandelbrot Set. • Some reference material from chptr 3 will be distributed. • Additional reference material on the Mandelbrot Sets will be provided from the textbook, “Computer Graphics” by F.S. Hill, 2nd Ed. • The slides covered are 99-102, 105-113. • Chapter 4 in WA: Partitioning and Divide-and-Conquer Strategies. • Our main focus is section 4.2.1, “Sorting Using Bucket Sort” • The slides covered are 120, 121, 124-130, 134-139, possibly 140-145. • Some reference material from Chapter 4 will be provided. • The textbook material should be used mainly to clarify material covered on slides. Chapter 6

3. Chapter 5 in WA covers pipeline computations. • Some reference material from Chapter 5 will be provided. • The textbook material can be used mainly as a reference to clarify slides covered. • The slides covered are 90-107. Some slides (e.g., 93-97) are only covered lightly. • Chapter 6 in WA covers Synchronous Computations. • The slides covered are 120 -132. Most slide are only covered lightly (due to time). • The Heat Distribution Problem (slides 141-150) will be discussed briefly – especially a short comparison of block and strip partitioning. • Chapter 7 in WA covers Load Balancing and Termination Detection. • Slides 160-163 were covered lightly. • Slides 163-171, 175-7 may be covered if time permits • The Shortest Path Problem (slides 182-196) is omitted due to time. • May be added in future • Note increased code is needed due to load balancing and termination. Chapter 6