CS 290H Administrivia: June 2, 2008

1. CS 290H Administrivia: June 2, 2008 Course project presentations this Wednesday: • 11:00 – 11:20 : Yen Ting Ng • 11:25 – 11:45 : Ben Adams • 11:50 – 12:10 : Yi Gong • 12:15 – 12:35 : Long Nguyen • 12:40 – 1:00 : Aydin Buluc Final project written reports due Wednesday, June 11.

2. Hierarchy of matrix classes (all real) • General nonsymmetric • Diagonalizable • Normal • Symmetric indefinite • Symmetric positive (semi)definite = Factor width n • Factor width k . . . • Factor width 4 • Factor width 3 • Diagonally dominant SPD = Factor width 2 • Generalized Laplacian = Symm diag dominant M-matrix • Graph Laplacian

3. Definitions • The Laplacian matrix of an n-vertex undirected graph G is the n-by-n symmetric matrix A with • aij = -1 if i ≠ j and (i, j) is an edge of G • aij = 0 if i ≠ j and (i, j) is not an edge of G • aii = the number of edges incident on vertex i • Theorem: The Laplacian matrix of G is symmetric, singular, and positive semidefinite. The multiplicity of 0 as an eigenvalue is equal to the number of connected components of G. • A generalized Laplacian matrix (more accurately, a symmetric weakly diagonally dominant M-matrix) is an n-by-n symmetric matrix A with • aij≤ 0 if i ≠ j • aii≥Σ |aij| where the sum is over j ≠ i

4. Edge-vertex factorization of generalized Laplacians • A generalized Laplacian matrix A can be factored as A = UUTwhere U has: • a row for each vertex • a column for each edge, with two nonzeros of opposite sign • a column for each excess-weight vertex, with one nonzero

5. Support Graph Preconditioning • CFIM: Complete factorization of incomplete matrix • Define a preconditioner B for matrix A • Explicitly compute the factorization B = LU • Choose nonzero structure of B to make factoring cheap (using combinatorial tools from direct methods) • Prove bounds on condition number using both algebraic and combinatorial tools +: New analytic tools, some new preconditioners +: Can use existing direct-methods software -: Current theory and techniques limited

6. Spanning Tree Preconditioner [Vaidya] • A is generalized Laplacian(symmetric positive semidefinite with negative off-diagonal nzs) • B is the gen Laplacian of a maximum-weight spanning tree for A (with diagonal modified to preserve row sums) Form B: costs O(n log n) or less time (graph algorithms for MST) Factorize B = RTR: costs O(n) space and O(n) time (sparse Cholesky) Apply B-1: costs O(n) time per iteration G(A) G(B)

7. Combinatorial analysis: cost of preconditioning • A is generalized Laplacian(symmetric positive semidefinite with negative off-diagonal nzs) • B is the gen Laplacian of a maximum-weight spanning tree for A (with diagonal modified to preserve row sums) • Form B: costsO(n log n) time or less (graph algorithms for MST) • Factorize B = RTR: costs O(n) space and O(n) time (sparse Cholesky) • Apply B-1: costs O(n) time per iteration (two triangular solves) G(A) G(B)

8. Numerical analysis: quality of preconditioner • support each edge of A by a path in B • dilation(A edge) = length of supporting path in B • congestion(B edge) = # of supported A edges • p = max congestion, q = max dilation • condition number κ(B-1A) bounded by p·q (at most O(n2)) G(A) G(B)

9. Spanning Tree Preconditioner [Vaidya] • can improve congestion and dilation by adding a few strategically chosen edges to B • cost of factor+solve is O(n1.75), or O(n1.2) if A is planar • in experiments by Chen & Toledo, often better than drop-tolerance MIC for 2D problems, but not for 3D. G(A) G(B)

10. Numerical analysis: Support numbers Intuition from networks of electrical resistors: • graph = circuit; edge = resistor; weight = 1/resistance = conductance • How much must you amplify B to provide as much conductance as A? • How big does t need to be for tB – A to be positive semidefinite? • What is the largest eigenvalue of B-1A ? The support of B for A is σ(A, B) = min { τ : xT(tB– A)x  0 for all x and all t  τ } • If A and B are SPD then σ(A, B) = max{λ : Ax = λBx} = λmax(A, B) • Theorem: If A and B are SPD then κ(B-1A) = σ(A, B) · σ(B, A)

11. Old analysis, splitting into paths and edges • Split A = A1+ A2 + ··· + Ak and B = B1+ B2 + ··· + Bk • such that Ai and Bi are positive semidefinite • Typically they correspond to pieces of the graphs of A and B (edge, path, small subgraph) • Theorem: σ(A, B)  maxi {σ(Ai , Bi)} • Lemma: σ(edge, path)  (worst weight ratio) · (path length) • In the MST case: • Ai is an edge and Bi is a path, to give σ(A, B)  p·q • Bi is an edge and Ai is the same edge, to give σ(B, A)  1

12. New analysis: Algebraic Embedding Lemma vv[Boman/Hendrickson] Lemma: If V·W=U, then σ(U·UT, V·VT)  ||W||22 (with equality for some choice of W) Proof: • take t  ||W||22 = λmax(W·WT) = max y0 { yTW·WTy / yTy } • then yT (tI - W·WT) y  0 for all y • letting y = VTx gives xT (tV·VT - U·UT) x  0 for all x • recall σ(A, B) = min{τ : xT(tB– A)x  0 for all x, all t  τ} • thus σ(U·UT, V·VT)  ||W||22

13. [ ] a2 +b2-a2 -b2 -a2 a2 +c2 -c2-b2 -c2 b2 +c2 [ ] a2 +b2-a2 -b2 -a2 a2 -b2 b2 B =VVT A =UUT ] [ ] ] [ [ 1-c/a1 c/b/b ab-a c-b ab-a c-b-c = x U V W -a2 -c2 -a2 -b2 -b2 σ(A, B)  ||W||22 ||W||x ||W||1 = (max row sum) x (max col sum) (max congestion) x (max dilation)

14. Open problems I • Other subgraph constructions for better bounds on||W||22? • For example [Boman], ||W||22 ||W||F2= sum(wij2) = sum of (weighted) dilations, and [Alon, Karp, Peleg, West]show there exists a spanning tree with average weighted dilation exp(O((log n loglog n)1/2)) = o(n ); this gives condition number O(n1+) and solution time O(n1.5+), compared to Vaidya O(n1.75) with augmented spanning tree • [Spielman, Teng] recursive partitioning construction gives solution time O(n1+) for all generalized Laplacians! (Uses yet another matrix norm inequality.) • Is there a construction that minimizes ||W||22directly?

15. Open problems II • Make spanning tree methods more effective in 3D? • Vaidya gives O(n1.75) in general, O(n1.2) in 2D • Issue: 2D uses bounded excluded minors, not just separators • Support graph methods for more general matrices? • All SPD matrices? ([Boman, Chen, Hendrickson, Toledo]: different matroid for all diagonally dominant SPD matrices) • Finite element problems? ([Boman]: Element-by-element preconditioner for bilinear quadrilateral elements) • Matrices of bounded factor width?

16. Hierarchy of matrix classes (all real) • General nonsymmetric • Diagonalizable • Normal • Symmetric indefinite • Symmetric positive (semi)definite = Factor width n • Factor width k . . . • Factor width 4 • Factor width 3 • Diagonally dominant SPD = Factor width 2 • Generalized Laplacian = Symm diag dominant M-matrix • Graph Laplacian

17. n1/2 n1/3 Complexity of linear solvers Time to solve model problem (Poisson’s equation) on regular mesh

18. B has positive (dotted) edges that cancel fill B has same row sums as A Strategy: Use the negative edges of B to support both the negative edges of A and the positive edges of B. -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 B A .5 .5 .5 .5 .5 .5 .5 .5 .5 A = 2D model Poisson problem B = MIC preconditioner for A Support-graph analysis of modified incomplete Cholesky

19. Every dotted (positive) edge in B is supported by two paths in B Each solid edge of B supports one or two dotted edges Tune fractions to support each dotted edge exactly 1/(2n – 2) of each solid edge is left over to support an edge of A Supporting positive edges of B

20. Analysis of MIC: Summary • Each edge of A is supported by the leftover 1/(2n – 2) fraction of the same edge of B. • Therefore σ(A, B)  2n – 2 • Easy to show σ(B, A)  1 • For this 2D model problem, condition number is O(n1/2) • Similar argument in 3D gives condition number O(n1/3) or O(n2/3) (depending on boundary conditions)