390 likes | 587 Views
Numerical Simulation 219A Spring 2008. Solution of Sparse Linear Systems. Thanks to: Kin Sou, Deepak Ramaswamy, Michal Rewienski, Jacob White, Shihhsien Kuo and Karen Veroy and especially Luca Daniel. Outline of today’s lecture. Solution of Sparse Linear Systems
E N D
Numerical Simulation 219A Spring 2008 Solution of Sparse Linear Systems Thanks to: Kin Sou, Deepak Ramaswamy, Michal Rewienski, Jacob White, Shihhsien Kuo and Karen Veroy and especially Luca Daniel
Outline of today’s lecture • Solution of Sparse Linear Systems • Examples of Problems with Sparse Matrices • Struts and joints, resistor grids, 3-D heat flow • Tridiagonal Matrix Factorization • General Sparse Factorization • Fill-in and Reordering • Graph Based Approach • Sparse Matrix Data Structures • Scattering
5 7 9 3 4 6 8 2 1 Applications Sparse Matrices Space Frame Nodal Matrix Space Frame X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X Unknowns : Joint positions Equations : forces = 0
Applications Sparse Matrices Resistor Grid Unknowns : Node Voltages Equations : currents = 0
Applications Sparse Matrices Nodal Formulation Resistor Grid Matrix non-zero locations for 100 x 10 Resistor Grid
Applications Sparse Matrices Nodal Formulation Temperature in a cube Temperature known on surface, determine interior temperature Circuit Model
Outline • Solution of Dense Linear Systems • Solution of Sparse Linear Systems • LU Factorization Reminder. • Example of Problems with Sparse Matrices • Struts and joints, resistor grids, 3-d heat flow • Tridiagonal Matrix Factorization • General Sparse Factorization • Fill-in and Reordering • Graph Based Approach • Sparse Matrix Data Structures • Scattering
Tridiagonal Example Sparse Matrices Nodal Formulation Matrix Form How many operations to do LU factorization? m
Pivot Multiplier Tridiagonal Example Sparse Matrices GE Algorithm For i = 1 to n-1 {“For each Row” For j = i+1 to n {“For each target Row below the source” For k = i+1 to n {“For each Row element beyond Pivot” } } } Order n Operations!
Outline • Solution of Dense Linear Systems • Solution of Sparse Linear Systems • LU Factorization Reminder. • Example of Problems with Sparse Matrices • Struts and joints, resistor grids, 3-d heat flow • Tridiagonal Matrix Factorization • General Sparse Factorization • Fill-in and Reordering • Graph Based Approach • Sparse Matrix Data Structures • Scattering
Sparse Matrix Fill-In Sparse Matrices Example Resistor Example Nodal Matrix 0 Symmetric Diagonally Dominant
X X X = fill in where 0 became non-zero Fill-In Sparse Matrices Example Matrix Non zero structure Fill Ins X X X X X= Non zero
X X X X X Fill-In Sparse Matrices Second Example Fill-ins Propagate X X X X X Fill-ins from Step 1 result in Fill-ins in step 2
0 Fill-In Sparse Matrices Reordering Fill-ins 0 No Fill-ins Node Reordering - Can reduce fill-in - Preserves properties (Symmetry, Diagonal Dominance) - Equivalent to swapping rows and columns
Possible Fill-in Locations Fill-in Estimate = (Non zeros in unfactored part of Row -i) (Non zeros in unfactored part of Col -i) Markowitz product Fill-In Sparse Matrices Reordering Where can fill-in occur ? Already Factored Multipliers
Fill-In Sparse Matrices Reordering Markowitz Reordering Greedy Algorithm !
1 2 3 1 0 0 3 2 Fill-In Sparse Matrices Reordering Why only try diagonals ? • Corresponds to node reordering in Nodal formulation • Reduces search cost • Preserves Matrix Properties - Diagonal Dominance - Symmetry
Fill-In Sparse Matrices Pattern of a Filled-in Matrix Very Sparse Very Sparse Dense
Fill-In Sparse Matrices Unfactored Random Matrix Symmetric
Fill-In Sparse Matrices Factored Random Matrix Symmetric
Outline • Solution of Dense Linear Systems • Solution of Sparse Linear Systems • LU Factorization Reminder. • Example of Problems with Sparse Matrices • Struts and joints, resistor grids, 3-d heat flow • Tridiagonal Matrix Factorization • General Sparse Factorization • Fill-in and Reordering • Graph Based Approach • Sparse Matrix Data Structures • Scattering
Matrix Graphs Sparse Matrices Construction Structurally Symmetric Matrices and Graphs 1 X X X X X X X 2 X X X X 3 X X X 4 X X X 5 • One Node Per Matrix Row • One Edge Per Off-diagonal Pair
1 2 X X X X 3 X X X 4 X X X X 5 X X X X X X Matrix Graphs Sparse Matrices Markowitz Products Markowitz Products = (Node Degree)2
Matrix Graphs Sparse Matrices Factorization One Step of LU Factorization 1 X X X X X X X X 2 X X X X X 3 X X X 4 X X X X X 5 • Delete the node associated with pivot row • “Tie together” the graph edges
1 2 3 4 5 1 2 3 4 5 Matrix Graphs Sparse Matrices Example Graph Markowitz products ( = Node degree)
5 5 4 3 2 3 4 Graph Matrix Graphs Sparse Matrices Example Swap 2 with 1 X 1 2 2 1 X Eliminate 1
Graphs Sparse Matrices Resistor Grid Example Unknowns : Node Voltages Equations : currents = 0
Matrix Graphs Sparse Matrices Grid Example
What should you pivot for? Sparse Matrices For growth control? Or to avoid fill-ins? A) LU factorization applied to a strictly diagonally dominant matrix will never produce a zero pivot B) The matrix entries produced by LU factorization applied to a strictly diagonally dominant matrix will never increase by more than a factor 2(n-1) [which is anyway the best you can do by pivoting for growth control] Bottom line: if your matrix is strictly diagonally dominant no need for numerical pivot for growth control!
Sparse Factorization Approach • Assume matrix requires NO numerical pivoting. Diagonally dominant or symmetric positive definite. • Pre-Process: • Use Graphs to Determine Matrix Ordering • Many graph manipulation tricks used. • Form Data Structure for Storing Filled-in Matrix • Lots of additional non-zero added • Put numerical values in Data Structure and factor Computation must be organized carefully!
Summary of Sparse Systems • LU Factorization and Diagonal Dominance. • Factor without numerical pivoting • Sparse Matrices • Struts, resistor grids, 3-d heat flow -> O(N) non-zeros • Tridiagonal Matrix Factorization • Factor in O(N) operations • General Sparse Factorization • Markowitz Reordering to minimize fill • Graph Based Approach • Factorization and Fill-in • Useful for estimating Sparse GE complexity
Outline • Solution of Sparse Linear Systems • Example of Problems with Sparse Matrices • Tridiagonal Matrix Factorization • General Sparse Factorization • Fill-in and Reordering • Graph Based Approach • Summary of the Algorithm • Sparse Matrix Data Structures • Scattering and symbolic fill in
Sparse Data Structure Sparse Matrices Vector of row pointers Arrays of Data in a Row Matrix entries Val 11 Val 12 Val 1K 1 Column index Col 11 Col 12 Col 1K Val 21 Val 22 Val 2L Col 21 Col 22 Col 2L Row pointers Column Indices Goal: Never store a 0 Never multiply by 0 Val N1 Val N2 Val Nj N Col N1 Col N2 Col Nj
Sparse Data Structure Sparse Matrices Problem of Misses Eliminating Source Row i from Target row j Row i Row j Must read all the row j entries to find the 3 that match row i
Sparse Data Structure Sparse Matrices Data on Misses More misses than ops! Every Miss is an unneeded Memory Reference!
Sparse Data Structure Sparse Matrices Scattering for Miss Avoidance Row j Use target row approach – Row j is the target. 1) Read all the elements in Row j, and scatter them in an n-length vector 2) Access only the needed elements using array indexing!
Sparse Matrices – Another Data Structure Orthogonal linked list But if fill in occurs, pointer structures change. What do we do? Pre-compute pivoting order and sparse fill in structure symbolically
Summary of Sparse Systems • LU Factorization and Diagonal Dominance. • Factor without numerical pivoting • hard problems (ill-conditioned) • Sparse Matrices • Struts, resistor grids, 3-d heat flow -> O(N) nonzeros • Tridiagonal Matrix Factorization • Factor in O(N) operations • General Sparse Factorization • Markowitz Reordering to minimize fill • Graph Based Approach • Factorization and Fill-in • Useful for estimating Sparse GE complexity • Sparse Data Structures • Scattering and symbolic fill in