Parallel Algorithms & Distributed Computing

1. Parallel Algorithms & Distributed Computing Matt Stimmel Matt White

2. The Goal: 1 Gigapixel

3. How?

4. How?

5. Imhotep, How are we going to make all these bricks? The Plan

6. We’ll use the Hebrew Slaves! The Plan

7. + = Benford 6100 Brick Mold Hebrew Slave Brick The Process

8. Yeah, this is pretty slow. Re-evaluation

9. We’ll use MORE Hebrew Slaves! Re-evaluation

10. + = + = + = + = + = + = The Revised Process Enough bricks for a pyramid! Lots of Benford 6100 Brick Molds Lots of Hebrew Slaves

11. Thus Parallel Algorithms were born

12. Job Process In computers Result

13. Job Sub-task Sub-result Sub-task Sub-result Process Process Process Process Sub-task Sub-result Sub-task Sub-result Faster! Result

14. Job Sub-task Sub-result Sub-task Sub-result Process Process Process Process Sub-task Sub-result Sub-task Sub-result What happens here? Result

15. Job Sub-task Sub-result Sub-task Sub-result Combine Process Process Process Process Split Sub-task Sub-result Sub-task Sub-result Ah Ha! Result

16. Job Sub-task Sub-result Sub-task Sub-result Combine Process Process Process Process Split Sub-task Sub-result Sub-task Sub-result Sequential! Result

17. Job Sub-task Sub-result Sub-task Sub-result Process Sub-result Sub-task Process Combine Split Process Sub-task Sub-result Process Sequential Parallel Sequential So, Really, we have… Result

18. 1 Parallel (1 – Parallel) + Sequential Speedup = Amdahl's Law

19. Or, as a 5th grader would say, “How many times faster it goes” 1 Parallel (1 – Parallel) + Sequential Speedup = Amdahl's Law

20. Amdahl's Law http://en.wikipedia.org/wiki/File:AmdahlsLaw.svg

21. = = - Process Process Process Process = = Why Distributed Computing?

22. Imhotep, we can’t afford all these Hebrew Slaves! The Plan

23. Well the surrounding countries have slaves they’re not using… The Plan

24. + = Slaves from Egypt Egypt’s Benford 6100 Brick Molds + = + = Slaves from Mars Space Brickmakers + = Slaves from ChickenLand Avian Brickmakers + = The Revised Process: Cheap Enough bricks for a pyramid! Robot Slaves Brick Factory

25. Job Internet Internet Sub-task Sub-result Internet Internet Sub-task Sub-result Combine Process Process Process Process Split Internet Internet Sub-task Sub-result Internet Internet Sub-task Sub-result Distributed Computing Result

26. Internet Not My = Internet = - Process Process Process Process Internet = Internet = Why Distributed Computing?

27. Design

28. Server Client Control Data FractalGen ImageCombiner Storage The Big Picture

29. Functions: • Manages Connections to Clients • Allows the user to input task parameters. • Divides job into sub-tasks • Displays information about currently running job Server • Challenges: • Job division • Must divide into a perfect square number of Jobs • Resolution • Problems occur when the resolution isn’t an integer multiple of the number of jobs. • Networking • Treated client connections as a Finite State Machine for file transfers. Didn’t work. • About 1 in 100 transfers failed. Server

30. Functions: • Manages Connection to server • Receives server-generated command line for job • Executes FractalGen on cue • Informs server upon completion Client • Challenges: • Maintaining a graceful disconnect • Aforementioned file transfer Client

31. Functions: • Render the fractal • Specified by the command line • Save the fractal to the disk • Challenges: • How do you draw a Mandelbrot fractal anyway? • Z = Z2 + C • Command Line Parsing • Generating large images: • Limit is approx. 8000x8000 due to graphics hardware • Shader Implementation • Downloaded, then made attempts at optimization. • Iteration count • Shader Model 3 gives a greater instruction cound allowing form more iterations. • Shader Model 2 has greater compatibility, but cannot render as many iterations. • Client computer must have a GPU capable of running at least DirectX 9 or else FractalGen will not work. FractalGen FractalGen

32. Functions: • Merge images back together • Challenges: • Finding .bmp files • Memory Allocation • Estimates demanded efficient heap usage • A lot of pointers, a lot of pointer math • Working with Bitmaps • Multiple color modes – how many bits is a pixel? • “Upside Down” errors • Running over a network ImageCombiner Image Combiner

33. Overview of Technologies Used Server • C# • .NET • Multithreaded • C# • .NET • Multithreaded Client • C# • .NET • XNA • HLSL • Shader Model 3.0 FractalGen • C++ • OOP • Memory Allocation ImageCombiner

34. How is the task divided into separate jobs?

35. Breaking up Jobs: Method A

36. Breaking up Jobs: Method B

37. Overlays

38. Method A: Since B didn’t work

39. Client 1 Client 2 Client 3 Client 4 We now send a square to each client

40. The clients return separate images of the fractal.

41. Run the image combiner.

42. Final Output

43. Server Client Control Data FractalGen ImageCombiner Storage The Big Picture

44. Science

45. Hypothesis: Using a parallel implementation and multiple processors, we will be able to increase performance over the same implementation on a single processor. Science, Step 1

46. Experiment: Run the system, splitting job into varying numbers of sub-tasks and varying the number of processors available. Science, Step 2

47. Control: Run the system using one client. Science, Step 3

48. Major Dramatic Question: Is ≥ 1? Sequential Parallel Playwrighting, Step 1

49. Data

50. Data