CS 584

1. CS 584

2. Review • Systems of equations and finite element methods are related.

3. Gauss-Seidel Iteration • Gauss-Seidel parallelism can be increased by using graph coloring.

4. Partitioning P0 P1 P2 P3

5. P0 P1 P2 P3 Communication

6. Finite Element Method • Used for deriving approximate numerical solutions to partial differential equations over a discretized domain. • Two sources of error • numerical (due to approximation) • discretization

7. Finite Element Method • Domain is bounded • Some grid is imposed on the domain • Equations are generated to indicate the relationships between grid points • Jacobi or Gauss-Seidel iteration is used to relax the system and converge to the solution

8. ¶ ¶ 2 2 u u + = 0 ¶ ¶ 2 2 x y The Heat Equation • The steady-state temperature u at any point (x,y) on a metal sheet is governed by: • Discretization allows us to transform this into a system of linear equations which will approximate the partial derivatives.

9. The Finite Element Method • For most applications, the finite element graph is not a regular structure. • The corresponding system of equations is, most often, very large and very sparse. • Efficient parallelization of the system depends heavily on the way the domain is partitioned among processors.

10. Partitioning Techniques • Regular grids • One dimensional striping • Two dimensional blocking • Generalized Graphs • Levelization • Scattered Decomposition • Recursive Bisection

11. Levelization • Begin with a boundary • Number these nodes 1 • All nodes connected to a level 1 node are labeled 2, etc. • Partitioning is performed • determine the number of nodes per processor • count off the nodes of a level until exhausted • proceed to the next level

12. Levelization

13. Levelization • Want to insure nearest neighbor comm. • If p is # processors and n is # nodes. • Let ribe the sum of the number of nodes in contiguous levels i and i + 1 • Let r = max{r1, r2, … , rn} • Nearest neighbor communication is assured if n/p > r

14. Scattered Decomposition • Used for highly irregular grids • Partition load into a large number r of rectangular clusters such that r >> p • Each processor is given a disjoint set of r/p clusters. • Communication overhead can be a problem for highly irregular problems.

15. Recursive Bisection • Recursively divide the domain in two pieces at each step. • 3 Methods • Recursive Coordinate Bisection • Recursive Graph Bisection • Recursive Spectral Bisection

16. Recursive Coordinate Bisection • Divide the domain based on the physical coordinates of the nodes. • Pick a dimension and divide in half. • RCB uses no connectivity information • lots of edges crossing boundaries • partitions may be disconnected • Some new research based on graph separators overcomes some problems.

17. Recursive Graph Bisection • Based on graph distance rather than coordinate distance. • Determine the two furthest separated nodes • Organize and partition nodes according to their distance from extremities. • Computationally expensive • Can use approximation methods.

18. Recursive Spectral Bisection • Uses the discrete Laplacian • Let A be the adjacency matrix • Let D be the diagonal matrix where • D[i,i] is the degree of node I • LG = A - D

19. Recursive Spectral Bisection • LG is negative semidefinite • Its largest eigenvalue is zero and the corresponding eigenvector is all ones. • The magnitude of the second largest eigenvalue gives a measure of the connectivity of the graph. • Its corresponding eigenvector gives a measure of distances between nodes.

20. Recursive Spectral Bisection • The eigenvector corresponding to the second largest eigenvalue is the Fiedler vector. • Calculation of the Fiedler vector is computationally intensive. • RSB yields connected partitions that are very well balanced.

21. RSB 299 edges cut RCB 529 edges cut RGB 618 edges cut