Unit #1 Linear Systems

1. Unit #1Linear Systems Spring 2009-2010 Dr. Jehad Al Dallal

2. Linear Equations • One equation 7x=21 • … • A system of n simultaneous linear equations can be represented in matrix notation as: Ax = b where A is an nXn matrix, and x and b are vectors of length n. • Can write solution as x = A-1b whereA-1 is the inverse of A.

3. Examples • If A is a 2X2 matrix, for example then 2x1 – x2 = 3 and 3x1 + 4x2 = -1. Each represents a straight line and the solution of the above is given by their intersection. If A is a 3X3 matrix each of the three equations represents a plane, and the solution is the point lying at the intersection of the three planes.

4. Solution methods • Exact solution methods • Example: Gaussian Elimination method • Approximation methods • Example: Gauss-Seidel method

5. Gaussian Elimination • Eliminate x1 from all the equations after the first. • Then eliminate x2 from all the equations after the second. • Then eliminate x3 from all the equations after the third. • And so on, until after n-1 steps we have eliminated xj from all the equations after the jth, for j = 1, 2, …, n-1. • These steps are referred to as the forward elimination stage of Gaussian elimination.

6. Example Subtract -3/10 times equation 1 from equation 2, and 5/10 times equation 1 from equation 3. Next we swap equations 2 and 3. This is called partial pivoting. It is done to get the largest absolute value on or below the diagonal in column 2 onto the diagonal. This makes the algorithm more stable with respect to round-off errors (see later).

7. Example (continued) Now subtract -0.1/2.5 times equation 2 from equation 3. This completes the forward elimination stage of the Gaussian elimination algorithm.

8. Back Substitution • After the forward elimination phase, the matrix has been transformed into upper triangular form. • Equation n just involves xn and so can now be solved immediately. • Equation n-1 just involves xn-1 and xn, and since we already know xn we can find xn-1. • Working our way backwards through the equations we can find xn, xn-1,…, x1. • This is called the back substitution phase of the Gaussian elimination algorithm.

9. The Example Again Equation 3 is 6.2x3 = 6.2, so x3 = 1. This value is substituted into equation 2: 2.5x2 + (5)(1) = 2.5 so x2 = -1. Substituting for x2 and x3 in equation 1: 10x1 + (-7)(-1) = 7 so x1 = 0.

10. LU Factorisation • The Gaussian elimination process can be expressed in terms of three matrices. • The first matrix has 1’s on the main diagonal and the multipliers used in the forward elimination below the diagonal. This is a lower triangularmatrix with unit diagonal, and is usually denoted by L.

11. LU Factorisation • The second matrix, denoted by U, is the upper triangular matrix obtained at the end of the forward elimination. • The third matrix, denoted by P, is a permutation matrix representing the row interchanges performed in pivoting.

12. L, U, and P • The original matrix can be expressed as: • LU = PA

13. Norms and condition numbers • Norm of a vector : single number that measures the general size of the elements of the vector. • Most common norms use p = 1, 2, and ¥

14. Example • X=[2 4 6 -8] • ||x||1=2+4+6+8=20 • ||x||2 =(22+42+62+82)0.5 • ||x||¥=|-8|=8

15. Condition Number • The condition number measures the amplification of the relative error.

16. Example

17. Uses of Condition Number • As a measure of the amplification of relative error due to changes in matrix A. • As a measure of how close a matrix is to being singular. If K(A) is large then A is close to singular.