Linear Systems of Equations Ax = b

1. Linear Systems of EquationsAx = b Marco Lattuada Swiss Federal Institute of Technology - ETH Institut für Chemie und Bioingenieurwissenschaften ETH Hönggerberg/ HCI F135 – Zürich (Switzerland) E-mail: lattuada@chem.ethz.ch http://www.morbidelli-group.ethz.ch/education/index

2. Definition of the Problem We want to solve: where x is the vector of the unknowns, while A and b are given. Hypotheses: • The number of equations is equal to the number of unknowns (that is, A is a square matrix) • The coefficients of A, b and x are real • The solution of the system exists and it is unique • A-1 exists • A is not singular • A's columns are linearly independent • A's lines are linearly independent • det(A) is non-zero • rank(A) is equal to n • Ax = 0 only if x is a null vector Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 2

3. Analytical Approach b replaces the ith column Cramer’s rule (1750): The solution of a system of equations: Is given by: where Ai is defined as follows: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 3

4. Calculation of the determinant No jth column No ith row How to compute the determinant of a square matrix? Laplace formula (1772): where Ci,j is the cofactor of element ai,j. The cofactor Ci,j is the determinant of the submatrix obtained by removing the ith row and the jth column of the matrix, multiplied by (-1)i+j: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 4

5. Numerical approach: Gauss Elimination Method Let us consider the system: Let us consider the following operations: • I multiply one line by a constant • I substitute one line with a linear combination of the others • I operate a permutation of the lines The result does not change Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 5

6. Multiply by -3 • Sum it to 1st line • Multiply by -3 • Sum it to 1st line • Multiply by -4 • Sum it to 2nd line Triangular System Gauss Elimination Method Numerical Example Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 6

7. Gauss Elimination Method General Case I want to replace a21 with a zero I define the multiplier l21: Note that: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 7

8. Gauss Elimination Method Gauss Transformation Matrix where: Solution: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 8

9. Gauss Elimination Method Numerical Example Total number of operations required (n-1)(n-2) operations (flops) n(n-1) operations (flops) Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 9

10. Gauss Transformation Method • Gauss Elimination Method • Changes the matrix A • Needs the coefficient vector b • Must re-run the method if b is changed Let us change our point of view! can be used to transform A Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 10

11. Gauss Transformation Method Properties • The final matrix A is a right triangular matrix • The matrix M is a left triangular matrix • The inverse of M is also a left triangular matrix • The matrix L = M-1 has the simple form: Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 11

12. LR (LU) Factorization Consider the following expression: Let us multiply by L = M-1 both sides: Right triangular Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 12

13. LR Factorization Starting matrix A is transformed (factorized) as: Let us solve a linear system with a generic vector b: • For every vector b, two simple triangular systems must be solved without factorizing again • The matrices LR can be stored using the elements of A • If A is modified, it is often possible to modify L and R accordingly without factorizing Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 13

14. Pivot value must be ≠ 0 Problems of Gaussian Elimination and LR Starting matrix: Consider the following system: Consider the following similar system: a11 = 0  I switch the lines  x1 = 1 and x2 = 1 Manual LR Factorization without pivoting Marco Lattuada – Statistical and Numerical Methods for Chemical Engineers Solution of Linear Systems of Equations – Page # 14