Approximate Gaussian Elimination for Laplacians

1. Approximate Gaussian Eliminationfor Laplacians

2. Linear Equation Solvers Given matrix and vector find s.t.. Ubiquitous: Engineering Physics Chemistry Biology Statistics Computer Science

3. Linear Equation Solvers Regression

4. Linear Equation Solvers Solving LPs, SDPs, etc. Solved using sequence of systems of linear equations

5. Linear Equation Solvers Heat Diffusion

6. Linear Equation Solvers Given matrix and vector find s.t.. Gaussian Elimination: time for matrix . Faster methods: time, where BUT: Use structure to do better!

7. Linear Equation Solvers Heat Diffusion In practice: Solve in time nearly linear in # non-zeros of matrix (Incomplete Cholesky Factorization, Multi-grid methods)

8. Linear Equation Solvers Given matrix and vector find s.t.. Gaussian Elimination: time for matrix . Faster methods: time, where BUT: Use structure to do better! Incomplete Cholesky, FFM, Multi-grid Spielman-Teng ’04: Laplacian linear equations can provably be solved in nearly-linear time.

9. Linear Equation Solvers Heat Diffusion Laplacian Matrix

10. Linear Equation Solvers Solving LPs, SDPs, etc.

11. Linear Equation Solvers Flow Problem Laplacian

12. Laplacian Solvers in TCS Faster flow algorithms [DS08, CKMST11, KMP12, Mad13, LS14] Interior Point Methods [DS08,Mad13,LS14,KRS15,CMSV16] Learning on graphs [ZGL03, ZS04, ZBLWS04] Graph partitioning [OSV12] Sampling random spanning trees [KM09,MST15,DKPRS17,S18] Graph sparsification[SS08, LKP12,KPPS17]

13. Laplacian Solvers in TCS Many papers improving the Laplacian Solver: [KMP10, KMP11, KOSZ13, LS13, CKM+14, PS14, LPS16] Simpler, faster, parallelizable

14. Beyond Laplacian Solvers Symmetric Diagonally Dominant [G96] Planar 2D Truss Stiffness Matrices [DS07] Simplicial 3D Truss Stiffness Matrices [KPSZ18] Symmetric M-matrices [DS08] Connection Laplacians and BDD matrices [KLPSS16] Directed Laplacians [CKPPSV16, CKPPRSV16, CKKPPRS18] Hardness for many other classes [KZ17]

15. Laplacians Graph Edge weights matrix

16. Laplacian Matrices Graph Edge weights matrix

17. Laplacian Matrices weighted adjacency matrix of the graph diagonal matrix of weighted degrees Graph Edge weights

18. Laplacian Matrices Symmetric matrix All off-diagonals are non-positive and

19. Laplacian of a Graph

20. Positive Semi-Definite Matrices Positive Semi-Definite (PSD) Matrix symmetric and for all Laplacians are PSD

21. Solving a PSD System Gaussian Elimination Find , upper triangular matrix, s.t. Then Easy to apply and

22. Iterative Solvers Approximately solve Is it converging? What is an approximate solution?

23. Approximate Solutions to Linear Systems Define -approximate solution: find s.t. This is the right norm for most applications!

24. Iterative Solvers Approximately solve

25. Iterative Solvers Approximately solve Gradient descent on iterations for -approximation.

26. Preconditioning Iteratively solve Convergence fast if Suppose we can find s.t. and can be applied quickly Iteratively solve Now

27. Approximating PSD Matrices Define iff for all Define iff and

28. Solving a Laplacian System Approximate Gaussian Elimination Find , upper triangular matrix, s.t. is sparse. iterations to get -approximate solution .

29. Approximate Gaussian Elimination Theorem [KS] When is an Laplacian matrix with non-zeros, we can find in time an upper triangular matrix with non-zeros, s.t.w.h.p.

30. Additive View of Gaussian Elimination Standard view of Gaussian Elimination is multiplicative. Why take additive view? Additive view combines well random sampling. Helps show unbiased sampled elimination exists. Combines well with concentration results for sums of randommatrices.

31. Additive View of Gaussian Elimination Find , upper triangular matrix, s.t

32. Additive View of Gaussian Elimination Find the rank-1 matrix that agrees with on the first row and column.

33. Additive View of Gaussian Elimination Subtract the rank 1 matrix. We have eliminated the first variable.

34. Additive View of Gaussian Elimination The remaining matrix is PSD.

35. Additive View of Gaussian Elimination Find rank-1 matrix that agrees with our matrix on the next row and column.

36. Additive View of Gaussian Elimination Subtract the rank 1 matrix. We have eliminated the second variable.

37. Additive View of Gaussian Elimination Repeat until all parts written as rank 1 terms.

38. Additive View of Gaussian Elimination Repeat until all parts written as rank 1 terms.

39. Additive View of Gaussian Elimination Repeat until all parts written as rank 1 terms.

40. Additive View of Gaussian Elimination What is special about Gaussian Elimination on Laplacians? The remaining matrix is always Laplacian.

41. Additive View of Gaussian Elimination What is special about Gaussian Elimination on Laplacians? The remaining matrix is always Laplacian.

42. Additive View of Gaussian Elimination What is special about Gaussian Elimination on Laplacians? The remaining matrix is always Laplacian. A new Laplacian!

43. Why is Gaussian Elimination Slow? Solvingby Gaussian Elimination can take time. The main issue is fill

44. Why is Gaussian Elimination Slow? Solvingby Gaussian Elimination can take time. The main issue is fill

45. Why is Gaussian Elimination Slow? Solvingby Gaussian Elimination can take time. The main issue is fill New Laplacian Elimination creates a clique on the neighbors of

46. Why is Gaussian Elimination Slow? Solvingby Gaussian Elimination can take time. The main issue is fill New Laplacian Laplacian cliques can be sparsified!

47. Gaussian Elimination Pick a vertex to eliminate Add the clique created by eliminating Repeat until done

48. Approximate Gaussian Elimination Pick a vertex to eliminate Add the clique created by eliminating Repeat until done

49. Approximate Gaussian Elimination Pick a random vertex to eliminate Add the clique created by eliminating Repeat until done

50. Approximate Gaussian Elimination Pick a random vertex to eliminate Sample the clique created by eliminating Repeat until done Resembles randomized Incomplete Cholesky