1 / 206

Parallel and Distributed Algorithms

Parallel and Distributed Algorithms. Overview. Parallel Algorithm vs Distributed Algorithm PRAM Maximal Independent Set Sorting using PRAM Choice coordination problem Real world applications. Introduction. CPUs with 1000 processors. A massively parallel processing machine.

sol
Download Presentation

Parallel and Distributed Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel and Distributed Algorithms

  2. Overview • Parallel Algorithm vs Distributed Algorithm • PRAM • Maximal Independent Set • Sorting using PRAM • Choice coordination problem • Real world applications

  3. Introduction

  4. CPUs with 1000 processors A massively parallel processing machine Moore’s law coming to an end Need of distributed processing

  5. Parallel Algorithm • A parallel algorithm is an algorithm which can be executed a piece at a time on many different processing devices, and then combined together again at the end to get the correct result.* * Blelloch, Guy E.; Maggs, Bruce M. Parallel Algorithms. USA: School of Computer Science, Carnegie Mellon University.

  6. Distributed Algorithm • A distributed algorithm is an algorithm designed to run on computer hardware constructed from interconnected processors.* *Lynch, Nancy (1996). Distributed Algorithms. San Francisco, CA: Morgan Kaufmann Publishers. ISBN978-1-55860-348-6.

  7. PRAM

  8. Random Access Machine • An abstract machine with unbounded number of local memory cells and with simple set of instruction sets • Time complexity: number of instructions executed • Space complexity: number of memory cells used • All operations take Unit time

  9. PRAM (Parallel Random Access Machine) • PRAM is a parallel version of RAM for designing the algorithms applicable to parallel computers • Why PRAM ? • The number of processor execute per one cycle on P processors is at most P • Any processor can read/write any shared memory cell in unit time • It abstracts from overhead which makes the complexity of PRAM algorithm easier • It is a benchmark

  10. Read A[i-1], Computation A[i]=A[i-1]+1, Write A[i] A[0] Shared Memory (A) P1 A[1]=A[0]+1 A[1] A[1] P2 A[2]=A[1]+1 A[2] A[n-1] Pn A[n]=A[n-1]+1 A[n]

  11. Share Memory Access Conflicts • Exclusive Read(ER) : all processors can simultaneously read from distinct memory locations • Exclusive Write(EW) : all processors can simultaneously write to distinct memory locations • Concurrent Read(CR) : all processors can simultaneously read from any memory location • Concurrent Write(CW) : all processors can write to any memory location • EREW, CREW, CRCW

  12. Complexity • Parallel time complexity: The number of synchronous steps in the algorithm • Space complexity: The number of share memory cell • Parallelism: The number of processors used

  13. LahiruSamarakoon SumanarubanRajadurai Maximal Independent Set

  14. Independent Set (IS): Any set of nodes that are not adjacent

  15. Maximal Independent Set (MIS): An independent set that is no subset of any other independent set

  16. a maximal independent set a maximum independent set Maximal vs. Maximum IS

  17. A Sequential Greedy algorithm Suppose that will hold the final MIS Initially

  18. Phase 1: Pick a node and add it to

  19. Remove and neighbors

  20. Remove and neighbors

  21. Phase 2: Pick a node and add it to

  22. Remove and neighbors

  23. Remove and neighbors

  24. Phases 3,4,5,…: Repeat until all nodes are removed

  25. Phases 3,4,5,…,x: Repeat until all nodes are removed No remaining nodes

  26. At the end, set will be an MIS of

  27. Running time of algorithm: Worst case graph: nodes

  28. Intuition for parallelization At each phase we may select any independent set (instead of a single node), remove S and neighbors of S from the graph.

  29. Example: Suppose that will hold the final MIS Initially

  30. Phase 1: Find any independent set And insert to :

  31. remove and neighbors

  32. Phase 2: On new graph Find any independent set And insert to :

  33. remove and neighbors

  34. remove and neighbors

  35. Phase 3: On new graph Find any independent set And insert to :

  36. remove and neighbors

  37. remove and neighbors No nodes are left

  38. Final MIS

  39. Observation: The number of phases depends on the choice of independent set in each phase The larger the independent set at each phase the faster the algorithm

  40. Randomized Maximal Independent Set ( MIS ) Let be the degree of node 2 1

  41. At each phase : Each node elects itself with probability degree of in 2 1 Elected nodes are candidates for the independent set

  42. If two neighbors are elected simultaneously, then the higher degree node wins Example: if

  43. If both have the same degree, ties are broken arbitrarily Example: if

  44. Using previous rules, problematic nodes are removed Problematic nodes

  45. The remaining elected nodes form independent set

  46. Luby’s algorithm mark lower-degree vertices with higher probability

  47. Using previous rules, problematic nodes are removed Problematic nodes

  48. Luby’s algorithm if both end-points of an edge is marked, unmark the one with the lower degree

  49. The remaining elected nodes form independent set

  50. Luby’s algorithm add all marked vertices to MIS remove marked vertices with their neighbors and corresponding edges

More Related