1 / 122

Parallel Algorithms on Networks of Processors

Parallel Algorithms on Networks of Processors. Roy (Hutch) Pargas, PhD Computer Science (UNC Chapel Hill) School of Computing, Clemson University pargas@clemson.edu. Outline. What are parallel algorithms? Why use them? Challenges for parallel algorithm designers Choosing a network

brilliant
Download Presentation

Parallel Algorithms on Networks of Processors

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel Algorithms on Networks of Processors Roy (Hutch) Pargas, PhD Computer Science (UNC Chapel Hill) School of Computing, Clemson University pargas@clemson.edu

  2. Outline • What are parallel algorithms? Why use them? • Challenges for parallel algorithm designers • Choosing a network • Partitioning the data • Designing the algorithm • Example • Recurrences (binary tree) • Analysis (Speedup and Efficiency) • Summary and Conclusions 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  3. Why Parallel Computation? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  4. Why Parallel Computation? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  5. Why Parallel Computation? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  6. Why Parallel Computation? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  7. Why Parallel Computation? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  8. Why Parallel Computation? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  9. New ProcessorsFaster and Cheaper January 2011 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  10. Partition the Data 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  11. Organize the Processors 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  12. Build a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  13. Build a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  14. Choosing a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  15. Choosing a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  16. Choosing a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  17. Choosing a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  18. Choosing a Network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  19. Are There Really Any Multiprocessing Systems in Use Today? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  20. Are There Really Any Multiprocessing Systems in Use Today? Hamburg June 2011 Top 500 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  21. SupercomputersNEC/HP Tsubame (Japan) 1.192 petaflops≈ 1.28 quadrillion floating point operations per sec 73,278 Xeon cores Infiniband grid network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  22. SupercomputersDawning Nebulae (China) 1.27 petaflops≈ 1.36 quadrillion floating point operations per sec 9280 Intel 6-core Xeon processors = 55,680 cores Infinibandgrid network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  23. SupercomputersCray Jaguar (USA) 1.75 petaflops≈ 1.876 quadrillion floating point operations per sec 224,256 AMD cores 3D torus network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  24. SupercomputersNUDT Tianhe-1A (China) 2.566 petaflops≈ 2.75 quadrillion floating point operations per sec 14336CPUs Undisclosed proprietary network 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  25. SupercomputersFujitsu “K” (Japan) K = “kei” = Japanese for 10 quadrillion 8.162 petaflops≈ 9quadrillion floating point operations per sec 68,544 8-core SPARC64 processors = 548,352 cores 3-dimensional torus network called Tofu 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  26. TOP500 Top 500 Computers in the World 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  27. Where Does that Leave Us? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  28. Where Does that Leave Us? • In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  29. Where Does that Leave Us? • In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  30. Where Does that Leave Us? • In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  31. Where Does that Leave Us? • In a wonderful playground of mathematical algorithmic design where imagination and creativity are key! 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  32. Challenges for Parallel Algorithm Designers • Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  33. So Let’s Try It: • Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  34. So Let’s Try It:Elliptic Partial Diff Eqns • Choosing a network • Partitioning the problem • Designing the parallel algorithm 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  35. Elliptic PDEs • Problems involving second-order elliptic partial differential equations are equilibrium problems. Given a region R bounded by a curve C and that the unknown function z satisfies Laplace’s or Poisson’s equation in R, the objective is to approximate the value of z at any point in R. The method of finite differences is an often used numerical method for solving this problem. The basic strategy is to approximate the differential equation by a difference equation and to solve the difference equation. 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  36. Designing the Algorithm • Why Solve Linear Recurrences? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  37. Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  38. Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  39. Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  40. Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  41. Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences many many many times 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  42. Designing the Algorithm • Why Solve Linear Recurrences? • The problem: • Solving PDEs using the Method of Finite Differences leads to • Solving Block Tridiagonal Systems which leads to • Solving Tridiagonal Systems which leads to • Solving Linear Recurrences many many many times • Why Use a Binary Tree? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  43. Linear Recurrences • Key Idea: • Successfully solving the original pde problem depends upon solving recurrences quickly and efficiently. 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  44. Linear Recurrences • Consider the following set of n equations: x0 = a0 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  45. Linear Recurrences • Consider the following set of n equations: x0 = a0 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 Can we solve for xi in parallel? 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  46. Linear Recurrences • For uniformity: x0 = a0+ b0 x-1b0=0, x-1=dummy variable x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  47. Linear Recurrences • For uniformity: x0 = a0+ b0 x-1 x1 = a1 + b1x0 x2= a2+ b2 x1 ... xn-1= an-1+ bn-1 xn-2 • Observe, if xi= a + b xj xj= a’ + b’ xk Then xi = (a + ba’) +bb’ xk = a” + b” xk 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  48. Linear Recurrences • Notation change x0 = a0+ b0 x-1 C0,-1 = (a0,b0) x1 = a1 + b1x0 C1,0 = (a1,b1) x2= a2+ b2 x1 C2,1 = (a2,b2) ... xn-1= an-1+ bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1) • Observe, if xi= a + b xj xj= a’ + b’ xk Then xi = (a + ba’) +bb’ xk = a” + b” xk 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  49. Linear Recurrences • Notation change x0 = a0+ b0 x-1 C0,-1 = (a0,b0) x1 = a1 + b1x0 C1,0 = (a1,b1) x2= a2+ b2 x1 C2,1 = (a2,b2) ... xn-1= an-1+ bn-1 xn-2 Cn-1,n-2 = (an-1,bn-1) • Observe, if xi= a + b xjCi,j = (a,b) xj= a’ + b’ xkCj,k = (a’,b’) Then xi = (a + ba’) +bb’ xkCi,jCj,k = = a” + b” xkCi,k = (a+ba’,bb’) 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

  50. Linear Recurrences • To summarize: xi= a + b xjCi,j = (a,b) xj= a’ + b’ xkCj,k = (a’,b’) Then xi = (a + ba’) +bb’ xkCi,jCj,k = = a” + b” xkCi,k = (a+ba’,bb’) 50 Golden Years Ateneo Mathematics Program Quezon City, Philippines

More Related