Lecture 10 Dimensions, Independence, Basis and Complete Solution of Linear Systems

1. Lecture 10Dimensions, Independence, Basis and Complete Solution of Linear Systems Shang-Hua Teng

2. Linear Independence Linear Combination Linear Independence is linearly independent if only if none of them can be expressed as a linear combination of the others

3. Examples

4. Linear Independence and Null Space Theorem/Definition is linearly independent if and only a1v1+a2v2+…+anvn=0 only happens when alla’s are zero The columns of a matrix A are linearly independent when only solution to Ax=0 is x = 0

5. 2D and 3D v w u v

6. How do we determine a set of vectors are independent? Make them the columns of a matrix Elimination Computing their null space

7. Permute Rows and Continuing Elimination (permute columns)

8. Theorem If Ax = 0 has more have more unknown than equations (m > n: more columns than rows), then it has nonzero solutions. There must be free variables.

9. Echelon Matrices Free variables

10. Reduced Row Echelon Matrix R Free variables

11. Computing the Reduced Row Echelon Matrix • Elimination to Echelon Matrix E1PA = U • Divide the row of pivots by the pivots • Upward Elimination E2E1PA = R

12. Example: Gauss-Jordan Method for Matrix Inverse • [A I] • E1[A I] = [U, E1] • In its reduced Echelon Matrix • A-1 [A I] = [IA-1]

13. A Close Look at Reduced Echelon Matrix • The last equation of R x = 0 is redundant 0 = 0 • Rank of A is the number of pivots rank(A).

14. What is the Rank of Outer Product

15. Free variables Rank and Reduced Row Echelon Matrix • Theorem/Definition • Rank(A) = number of independent rows • Rank(A) = number of independent columns

16. Dimension of the Column Space and Null Space • The dimensions of the column space of A is equal to Rank(A). • The dimension of the null space of A is equal to the number of free variables which is n – Rank(A) • A is an m by n matrix

17. Rank and Reduced Row Echelon Matrix The Pivot columns are not combinations of earlier columns Pivot columns Free variables Free Columns

18. Reduced Echelon and Null Space Matrix • Nullspace Matrix • Special Solutions

19. Null Space Matrix • Ax=0 has n-Rank(A) free variables and special solutions • The Nullspace matrix has n-Rank(A) columns • The columns of the nullspace matrix are independent • The dimension of the Null space is n – rank(A)

20. Complete Solution of Ax = 0 • After column permutation, we can write r pivot columns n-r free columns • Nullspace matrix Pivot variables Free variables • Moreover: RN = [0]

21. Complete Solution to Ax = b • A is an m by n matrix, and b is an n-place vector • Unique solution • Infinitely many solution • No solution Suppose Ax = b has more then one solution, say x1, x2 then A x1 = b A x2 = b So A (x1 - x2 ) = 0 (x1 - x2 ) is in nullspace(A)

22. Complete Solution to Ax = b Suppose we found a particular solution xp to Ax = b i.e, A xp = b Let F be the indexes of free variables of Ax = 0 Let xF be the column vector of free variables Let N be the nullspace matrix of A Then defines the complete set of solutions to Ax = b xp

23. Example: Complete Solution to Ax = b

24. Augmented matrix [Ab] Elimination to obtain [Rd]

25. Set free variables to 0 to find a particular solution (1,0,6,0)T Compute the nullspace matrix Complete solution is

26. Full Rank Matrix • Suppose A is an m by n matrix. Then • A is full column if rank(A) = n • columns of A are independent • A is full row rank if rank(A) = m • Rows of A are independent

27. Full Column Rank Matrix • Columns are independent • All columns of A are pivot columns • There are non free variables or special solutions • The nullspace N(A) contains only the zero vector • If Ax=b has a solution (it might not) then it has only one solution n by n m-n rows of zeros

28. Full Row Rank Matrix • Rows are independent • All rows of A have pivots, R has no zero rows • Ax=b has a solution for every right hand side b • The column space is the whole space Rm • There are n-m special solutions in the null space of A

29. The Whole Picture • Rank(A) = m = n Ax=b has unique solution • Rank(A) = m < n Ax=b has n-m dimensional solution • Rank(A) = n < m Ax=b has 0 or 1 solution • Rank(A) < n, Rank(A) < m Ax=b has 0 or n-rank(A) dimensions

30. Basis and Dimension of a Vector Space • A basis for a vector space is a sequence of vectors that • The vectors are linearly independent • The vectors span the space: every vector in the vector can be expressed as a linear combination of these vectors

31. Basis for 2D and n-D • (1,0), (0,1) • (1 1), (-1 –2) • The vectors v1,v2,…vn are basis for Rn if and only if they are columns of an n by n invertible matrix

32. Column and Row Subspace • C(A): the space spanned by columns of A • Subspace in m dimensions • The pivot columns of A are a basis for its column space • Row space: the space spanned by rows of A • Subspace in n dimensions • The row space of A is the same as the column space of AT, C(AT) • The pivot rows of A are a basis for its row space • The pivot rows of its Echolon matrix R are a basis for its row space

33. Important Property I: Uniqueness of Combination • The vectors v1,v2,…vn are basis for a vector space V, then for every vector v in V, there is a unique way to write v as a combination of v1,v2,…vn . v = a1v1+ a2 v2+…+ an vn v = b1v1+ b2 v2+…+ bn vn • So: 0=(a1 - b1) v1 + (a2 -b2)v2+…+ (an -bn)vn

34. Important Property II: Dimension and Size of Basis • If a vector space V has two set of bases • v1,v2,…vm . V = [v1,v2,…vm ] • w1,w2,…wn . W= [w1,w2,…wn ]. • then m = n • Proof: assume n > m, write W = VA • A is m by n, so Ax = 0 has a non-zero solution • So VAx = 0 and Wx = 0 • The dimension of a vector space is the number of vectors in every basis • Dimension of a vector space is well defined

35. Dimensions of the Four SubspacesFundamental Theorem of Linear Algebra, Part I • Row space: C(AT) – dimension = rank(A) • Column space: C(A)– dimension = rank(A) • Nullspace: N(A) – dimension = n-rank(A) • Left Nullspace: N(AT) – dimension = m –rank(A)