1 / 98

Joint work with:

Parallel Computation and Algorithms by Raymond Greenlaw Armstrong Atlantic State University. Joint work with:. Jim Hoover Computer Science Department University of Alberta. Pilar de la Torre Department of Computer Science University of New Hampshire. Alex Schaffer

mari-compton
Download Presentation

Joint work with:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel Computationand Algorithms by Raymond Greenlaw Armstrong Atlantic State University Joint work with: Jim HooverComputer Science DepartmentUniversity of Alberta Pilar de la TorreDepartment of Computer Science University of New Hampshire Alex Schaffer National Institute of Health Magnus HalldorssonDecode Genetics Rossella Petreschi Department of Computer ScienceUniversity of Rome La Sapienza Larry Ruzzo Department of Computer ScienceUniversity of Washington Teresa Przytycka Department of Biophysics John Hopkins University Medical School Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  2. Part I: An Introduction toParallel Computationand Ρ-Completeness Theory Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  3. Part I: Outline • Introduction • Parallel Models of Computation • Basic Complexity • Evidence that NC  Ρ • Ρ-Complete Problems • Comparator Circuit Class • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  4. Part I: Outline • Introduction • Parallel Models of Computation • Basic Complexity • Evidence that NC  Ρ • Ρ-Complete Problems • Comparator Circuit Class • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  5. Introduction • Sequential computation: Feasible ~ nO(1) time (polynomial time). • Parallel computation: Feasible ~ nO(1) time and~ nO(1) processors. Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  6. Introduction • Goal of parallel computation: Develop extremely fast ((log n)O(1) ) time algorithms using a reasonable number of processors. • Speedup equation: (best sequential time) (number of processors) Allow a polynomial number of processors.  (parallel time). Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  7. Introduction Roughly, • A problem is feasible if it can be solved by a parallel algorithm with worst case time and processor complexity nO(1). • A problem is feasiblehighly parallelif it can be solved by an algorithm with worst case time complexity (log n)O(1) and processor complexity nO(1). • A problem is inherentlysequentialif it is feasible but has no feasible highly parallel algorithm for its solution. Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  8. Part I: Outline • Introduction • Parallel Models of Computation • Basic Complexity • Evidence that NC  Ρ • Ρ-Complete Problems • Comparator Circuit Class • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  9. Parallel Models of Computation • Parallel Random Access Machine Model • Boolean Circuit Model • Circuits and PRAMs Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  10. P0 C0 P1 C1 P2 C2 Parallel Random Access Machine RAM Processors Global Memory Cells Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  11. 0 1 1 1 0 AND OR OR AND OR NOT AND Boolean Circuit Model Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  12. Circuits and PRAMS Theorem: A function f from {0,1}* to {0,1}* can be computed by a logarithmic space uniform Boolean circuit family {n} with depth(n)є(logn)O(1) and size(n) єnO(1)if and only iff can be computed by a CREW-PRAM M on inputs of length n in time t(n) є (logn)O(1) using p(n) є nO(1). Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  13. Part I: Outline • Introduction • Parallel Models of Computation • Basic Complexity • Evidence that NC  Ρ • Ρ-Complete Problems • Comparator Circuit Class • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  14. Basic Complexity • Decision, Function, and Search Problems • Complexity Classes • Reducibility • Completeness Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  15. Decision, Function, and Search Problems 4 4 Spanning Tree-DGiven: An undirected graph G = (V,E) with weights from N labeling edges in E and a natural number k.Problem: Is there a spanning tree of G with cost less than or equal to k ?Spanning Tree-FGiven: Same (no k ).Problem: Compute the weight of a minimum cost spanning tree.Spanning Tree-SGiven: Same.Problem: Find a minimum cost spanning tree. 3 6 3 6 3 4 4 Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  16. Complexity Classes Definitions:P is the set of all languages L that are decidable in sequential time nO(1).NC is the set of all languages L that are decidable in parallel time (logn)O(1) and processors nO(1). FP is the set of all functions from {0,1}* to {0,1}* that are computable in sequential time nO(1).FNC is the set of all functions from {0,1}* to {0,1}* that are computable in parallel time (logn)O(1) and processors nO(1).NC k, k 1, is the set of all languages L such that L is recognized by a uniform Boolean circuit family {n} with size(n) = nO(1) and depth (n) = O((logn)k). Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  17. Many-One Reducibility Definitions:A language L is many-one reducibleto a language L’, written L PL’ , if there is a function f such that x є L if and only if f(x) є L’. L is P many-one reducibleto L’, written L  PL’ , if the function fis in FP.For k  1, L is NCk many-one reducibleto L’, written L  NC kL’, if the function fis in FNCk. L is NC many-one reducibleto L’, written L NCL’, if the function fis in FNC. m m m m Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  18. Many-One Reducibility Lemma:The reducibilities m,  P, NC k(k 1), and NC are transitive. • If L’ є P and LP L’, LNC kL’ (k >1), or L NCL’ then L є P. • If L’ є NC k (k >1), and LNC kL’ then L є NC k. • If L’ є NC and LNCL’ then L є NC . m m m m m m m m Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  19. Turing Reducibility Definition:L is NC Turing reducibleto L’, written LNCL’, if and only if the L’-oracle PRAM on inputs of length n uses time (logn)O(1) and processors nO(1). Lemma: The reducibility NC is transitive. • If LNCL’then LNCL’. • If LNCL’and L’ єNCthen L є NC. • If LNCL’and L’ єPthen L є P. T T m T T T Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  20. Completeness Definitions: A language L is P-hard under NC reducibility if L’ NCL for every L’ є P. A language L is P-complete under NC reducibility if L є P and L is P-hard. Theorem:If any P-complete problem is in NC then NC equals P. T Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  21. Part I: Outline • Introduction • Parallel Models of Computation • Basic Complexity • Evidence that NC  Ρ • Ρ-Complete Problems • Comparator Circuit Class • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  22. Evidence That NCP • General Simulations Are Not Fast • Fast Simulations Are Not General • Natural Approaches Provably Fail Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  23. Evidence That NC P Gaps in Simulations Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  24. Part I: Outline • Introduction • Parallel Models of Computation • Basic Complexity • Evidence that NC  Ρ • Ρ-Complete Problems • Comparator Circuit Class • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  25. P-Complete Problems There are approximately 175 P-complete problems (500 with variations).Categories: • Circuit complexity • Graph theory • Searching graphs • Combinatorial optimization and flow • Local optimality • Logic • Formal languages • Algebra • Geometry • Real analysis • Games • Miscellaneous Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  26. Circuit Value Problem Given: An encoding  of a Boolean circuit , inputs x1,…,xn, and a designated output y. Problem:Is output y of  TRUE on input x1,…,xn? Theorem: [Ladner 75]The Circuit Value Problem is P-complete under NC1 reductions. m Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  27. P-Complete Variations of CVP • Topologically Ordered [Folklore] • Monotone [Goldschlager 77] • Alternating Monotone Fanin 2, Fanout 2 [Folklore] • NAND [Folklore] • Topologically Ordered NOR [Folklore] • Synchronous Alternating Monotone Fanout 2 CVP [Greenlaw, Hoover, and Ruzzo 87] • Planar [Goldschlager 77] Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  28. NAND Circuit Value Problem Given: An encoding  of a Boolean circuit  that consists solely of NAND gates, inputs x1,…,xn, and a designated output y. Problem:Is output y of  TRUE on input x1,…,xn? Theorem:The NAND Circuit Value Problem is P-complete. Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  29. 1 0 0 1 0 0 1 1 0 1 1 1 1 1 NAND OR NAND NAND AND OR OR NAND 1 0 1 1 NAND Circuit Value Problem Proof:Reduce AM2CVP to NAND CVP. Complement all inputs. Relabel all gates as NAND. Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  30. Graph Theory • Lexicographically First Maximal Independent Set [Cook 85] • Lexicographically First ( + 1)-Vertex Coloring [Luby 84] • High Degree Subgraph [Anderson and Mayr 84] • Nearest Neighbor Traveling Salesman Heuristic [Kindervater, Lenstra, and Shmoys 89] Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  31. 1 1 0 0 0 0 X 1 2 3 4 5 X X NOR NOR 1 2 3 4 5 6 7 0 6 7 X NOR 1 8 8 0 NOR 9 9 0 Lexicographically First Maximal Independent Set Theorem: [Cook 85]LFMIS is P-complete. Proof:Reduce TopNOR CVP to LFMIS. Add new vertex 0. Connect to all false inputs. Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  32. Searching Graphs • Lexicographically First Depth-First Search Ordering [Reif 85] • Stack Breadth-First Search [Greenlaw 92] • Breadth-Depth Search [Greenlaw 93] Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  33. Context-Free Grammar Empty Given: A context-free grammar G = (N,T,P,S). Problem: Is L(G)empty? Theorem: [Jones and Laaser 76], [Goldschlager 81], [Tompa 91]CFGempty is P-complete. Proof: Reduce Monotone CVP to CFGempty. Given  construct G = (N,T,P,S) with N, T, S, and P as follows: Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  34. Context-Free Grammar Empty N = {i | vi is a vertex in } T = {a} S = n, where vn is the output of . P as follows: 1. For input vi, i aif value of vi is 1, 2. i jkif vi vjΛ vk, and 3. i j | k if vi vj V vk . Then the value of vi is 1 if and only if i , where  є {a}+. * Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  35. x1 x3 x2 x4 1 2 3 4 5 6 AND OR AND 7 CFGempty Example x1 = 0, x2 = 0, x3 = 1, and x4 = 1. G = (N, T, S, P ), where N = {1, 2, 3, 4, 5, 6, 7} T = {a} S = 7 P = {3  a, 4  a, 5  1 | 2, 6  34, 7  56} Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  36. Part I: Outline • Introduction • Parallel Models of Computation • Basic Complexity • Evidence that NC  Ρ • Ρ-Complete Problems • Comparator Circuit Class • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  37. Comparator Circuit Class Definition: [Cook 82] Comparator Circuit Value Problem (CCVP) Given: An encoding  of a circuit  composed of comparatorgates, plus inputs x1,…,xn, and a designated output y. Problem: Is output yof  TRUE on input x1,…,xn? Definition: CC is the class oflanguages NC reducible to CCVP. Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  38. CC-Complete Problems • Lexicographically First Maximal Matching [Cook 82] • Telephone Connection [Ramachandran and Wang 91] • Stable Marriage [Mayr and Subramanian 92] Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  39. Part I: Outline • Introduction • Parallel Models of Computation • Basic Complexity • Evidence that NC  Ρ • Ρ-Complete Problems • Comparator Circuit Class • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  40. Open Problems Find an NC algorithm or classify as P-complete. • Edge Ranking • Edge-Weighted Matching • Restricted Lexicographically First Maximal Independent Set • Integer Greatest Common Divisor • Modular Powering Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  41. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Part II:Sequential and Parallel Algorithmsfor Some Problems on Trees Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  42. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Part II: Outline • Introduction • Parallel Preliminaries • Node Ranking • Edge Ranking • Prüfer Codes • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  43. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Part II: Outline • Introduction • Parallel Preliminaries • Node Ranking • Edge Ranking • Prüfer Codes • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  44. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Trees Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  45. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Part II: Outline • Introduction • ParallelPreliminaries • Node Ranking • Edge Ranking • Prüfer Codes • Open Problems Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  46. P0 C0 P1 C1 P2 C2 Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Parallel Random Access Machine • RAM Processors • Global Memory Cells Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  47. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Preliminary Parallel Algorithms • Brent’s Scheduling Principle • Parallel Prefix Computation • Euler Tour • List Ranking • Parallel Tree Contraction Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  48. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Brent’s Scheduling Principle (Brent 1974) If processor allocation is not a problem then a t(n) time parallel algorithm that requires w(n) computational operations can be simulated using w(n)/p(n) + t(n) time and p(n) processors. Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  49. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Example of Brent’s Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

  50. Parallel Computation and Algorithms, Part II: Sequential and Parallel Algorithms for Tree Problems Raymond Greenlaw Parallel Prefix Computation (Ladner & Fisher 1980) The Parallel Prefix Problem can be solved in O(log n) time using n/log n processors on an EREW-PRAM. Parallel Computation and Algorithms, Part I: An Introduction to Parallel Computation and Ρ-Completeness TheoryRaymond Greenlaw

More Related