ENGG2012B Lecture 7 Matrix algebra

1. ENGG2012BLecture 7Matrix algebra Kenneth Shum ENGG2012B

2. Last week • The rank of a matrix is • The maximal number of linearly independent rows. • The maximal number of linearly independent columns. • The rank of a matrix can be calculated by • first reducing the matrix to RREF • then counting the number of non-zero rows. ENGG2012B

3. Last week • In the proof of the correctness of the method of calculating rank by RREF, we needto show that the rank of a matrix does not change if we • exchange two rows, • multiplying a row by a non-zero constant • add a scalar multiple of a row to another row. ENGG2012B

4. Last week multiply by B u multiply by A w v n x p matrix m x n matrix A is m x n, B is n x p Multiply by u w A B m x p matrix ENGG2012B

5. How to define the product of two matrices The i-th component is ENGG2012B

6. The product of two matrices • The (i,j)-entry of of the product AB is defined as Row i Column j ENGG2012B

7. ADDITION AND MULTIPLICATION OF MATRICES ENGG2012B

8. Matrix equality • Two matrices are said to be equal if • They have the same number of rows and the same number of columns (i.e. same size). • The corresponding entry are identical. ENGG2012B

9. Matrix addition and scalar multiplication • We can add two matrices if they have the same size • To multiply a matrix by a real number, we just multiply all entries in the matrix by that number. ENGG2012B

10. Matrix multiplication • Given an mn matrix A and a pq matrix B, their product AB is defined if n=p. • If n = p, we define their product, say C = AB, by computing the (i,j)-entry in C as the dot product of the i-th row of A and the j-th row of B. pq m q mn ENGG2012B

11. Examples is undefined. is undefined. but ENGG2012B

12. Square matrix • A matrix with equal number of columns and rows is called a square matrix. • For square matrices of the same size, we can freely multiply them without worrying whether the product is well-defined or not. • Because multiplication is always well-defined in this case. • The entries with the same column and row index are called the diagonal entries. • For example: ENGG2012B

13. Compatibility with function composition Multiplied by Multiplied by Multiplied by ENGG2012B

14. Order does matter in multiplication Rotate 90 degrees Reflection around x-axis Multiplied by Multiplied by not the same Reflection around x-axis Rotate 90 degrees Multiplied by Multiplied by ENGG2012B

15. Matrix product is not commutative • For real numbers, we have 35 = 53. • Multiplication of real numbers is commutative. • For matrices, in general AB BA. • Multiplication of matrices is non-commutative. • For example ENGG2012B

16. Associativity • For real numbers, we have (34)5 = 3(45). • Multiplication of real numbers is associative. • For any three matrices A, B, C, it is always true that (AB)C= A(BC), provided that the multiplications are well-defined. • Multiplication of matrices is associative. ENGG2012B

17. INVERTIBLE MATRIX ENGG2012B

18. Identity matrix • A square matrix whose diagonal entries are all one, and off-diagonal entries are all zero, is called an identity matrix. • We usually use capital letter I for identity matrix, or add a subscript and write In if we want to stress that the size is nn. ENGG2012B

19. Multiplication by identity matrix is trivial • The output and input are identical • IA = A for any A. • BI = B for any B. Multiplied by ENGG2012B

20. Invertible matrix • Given an nn matrix A, if we can find a matrix A’, such that then A is said to be invertible, or non-singular. • The matrix A’ is called an inverse of A. Multiplied byA Multiplied by A’ Multiplied byIn ENGG2012B

21. Example implies is invertible. Rotate 90 CCW Rotate 90 CW Multiplied by Multiplied by ENGG2012B

22. Matrix inverse may not exist • If matrix A induces a many-to-one mapping, then we cannot hope for any inverse. For example has no inverse There are more thanone vectors which maps tothe same image. because ENGG2012B

23. Naïve method for computing matrix inverse • Consider • Want to find A’ such that AA’= I • Solve for p, q, r, s in ENGG2012B

24. Uniqueness of matrix inverse • Before we discuss how to compute matrix inverse, we first show there is at most one A’ such that AA’ = A’ A = I. • Suppose on the contrary that there is another matrix A’’ such that AA’’ = A’’ A = I. • We want to prove that A’ = A’’. ENGG2012B

25. Proof of uniqueness Defining property of A’’ Multiply by A’ from the left I times anything is the same thing Matrix multiplication is associative Defining property of A’ I times anything is the same thing ENGG2012B

26. Notation • Since the matrix inverse (if exists) is unique, we use the symbol A-1 to represent the unique matrix which satisfies • From now on, we shall say that A-1 is the inverse of A. ENGG2012B

27. A convenient fact • To check that a matrix B is the inverse of A, it is sufficient to check either • BA = I, or • AB = I. • It can be proved that (1) implies (2), and (2) implies (1). • The detail is left as an exercise. ENGG2012B

28. GAUSS-JORDAN ELIMINATION ENGG2012B

29. Row operation using matrix • Recall that there are three kind of elementary row operations • Row exchange • Multiply a row by a non-zero constant • Replace a row by the sum of itself and a constant multiple of another row. • We can perform elementary row operation by matrix multiplication (from the left). • All three kinds of operation are invertible. ENGG2012B

30. Row exchange • Example: exchange row 2 and row 3 Multiply the same matrix from the left again, we get back the original matrix. ENGG2012B

31. Multiply a row by a constant • Multiply the first row by -1. Multiply the same matrix from the left again, we get back the original matrix. ENGG2012B

32. Row replacement • Add the first row to the second row We can multiply by another matrix from the left to undo ENGG2012B

33. Elementary matrix (I) • Three types of elementary matrices • Exchange row i and row j Col. i Col. j Row i Row j ENGG2012B

34. Elementary matrix (II) • Multiply row i by m Col. i Row i ENGG2012B

35. Elementary matrix (III) • Add s times row i to row j Col. i Col. j Row i Row j ENGG2012B

36. Row reduction • A series of row reductions is the same as multiplying from the left a series of elementary matrices. … E1, E2, E3, … are elementary matrices. ENGG2012B

37. If we can row reduce to identity then A is invertible. (Matrixmultiplication isassociative) ENGG2012B

38. Gauss-Jordan elimination • Append an identity matrix to the right • Then, we can interpret it as If we can row reduce A to the identity by a series of row operationsthen we can apply the same series of row operations to I and obtain the inverse of A. ENGG2012B

39. Algorithm http://en.wikipedia.org/wiki/Gauss%E2%80%93Jordan_elimination • Input: an nn matrix A. • Create an n  2n matrix M • The left half is A • The right half is the n by n indentity matrix In • Try to reduce the expanded matrix Msuch that the left half is equal to In. • If succeed, the right half of M is the inverse of A. • If you cannot reduce the left half of M to In, then A is not invertible. ENGG2012B

40. Example • Find the inverse of • Create a 36 matrix • After some row reductionswe get • Answer: ENGG2012B