1 / 25

Lecture 11 Fundamental Theorems of Linear Algebra Orthogonalily and Projection

Lecture 11 Fundamental Theorems of Linear Algebra Orthogonalily and Projection. Shang-Hua Teng. The Whole Picture. Rank(A) = m = n A x = b has unique solution. Rank(A) = m < n A x = b has n-m dimensional solution. Rank(A) = n < m A x = b has 0 or 1 solution.

tevy
Download Presentation

Lecture 11 Fundamental Theorems of Linear Algebra Orthogonalily and Projection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 11Fundamental Theorems of Linear AlgebraOrthogonalily and Projection Shang-Hua Teng

  2. The Whole Picture • Rank(A) = m = n Ax=b has unique solution • Rank(A) = m < n Ax=b has n-m dimensional solution • Rank(A) = n < m Ax=b has 0 or 1 solution • Rank(A) < n, Rank(A) < m Ax=b has 0 or n-rank(A) dimensions

  3. Basis and Dimension of a Vector Space • A basis for a vector space is a sequence of vectors that • The vectors are linearly independent • The vectors span the space: every vector in the vector can be expressed as a linear combination of these vectors

  4. Basis for 2D and n-D • (1,0), (0,1) • (1 1), (-1 –2) • The vectors v1,v2,…vn are basis for Rn if and only if they are columns of an n by n invertible matrix

  5. Column and Row Subspace • C(A): the space spanned by columns of A • Subspace in m dimensions • The pivot columns of A are a basis for its column space • Row space: the space spanned by rows of A • Subspace in n dimensions • The row space of A is the same as the column space of AT, C(AT) • The pivot rows of A are a basis for its row space • The pivot rows of its Echolon matrix R are a basis for its row space

  6. Important Property I: Uniqueness of Combination • The vectors v1,v2,…vn are basis for a vector space V, then for every vector v in V, there is a unique way to write v as a combination of v1,v2,…vn . v = a1v1+ a2 v2+…+ an vn v = b1v1+ b2 v2+…+ bn vn • So: 0=(a1 - b1) v1 + (a2 -b2)v2+…+ (an -bn)vn

  7. Important Property II: Dimension and Size of Basis • If a vector space V has two set of bases • v1,v2,…vm . V = [v1,v2,…vm ] • w1,w2,…wn . W= [w1,w2,…wn ]. • then m = n • Proof: assume n > m, write W = VA • A is m by n, so Ax = 0 has a non-zero solution • So VAx = 0 and Wx = 0 • The dimension of a vector space is the number of vectors in every basis • Dimension of a vector space is well defined

  8. Dimensions of the Four SubspacesFundamental Theorem of Linear Algebra, Part I • Row space: C(AT) – dimension = rank(A) • Column space: C(A)– dimension = rank(A) • Nullspace: N(A) – dimension = n-rank(A) • Left Nullspace: N(AT) – dimension = m –rank(A)

  9. Orthogonality and Orthogonal Subspaces • Two vectors v and w are orthogonal if • Two vector subspaces V and W are orthogonal if

  10. Example: Orthogonal Subspace in 5 Dimensions The union of these two subspaces is R5

  11. Orthogonal Complement • Suppose V is a vector subspace a vector space W • The orthogonal complement of V is • Orthogonal complement is itself a vector subspace

  12. Dimensions of the Four SubspacesFundamental Theorem of Linear Algebra, Part I • Row space: C(AT) – dimension = rank(A) • Column space: C(A)– dimension = rank(A) • Nullspace: N(A) – dimension = n-rank(A) • Left Nullspace: N(AT) – dimension = m –rank(A)

  13. Orthogonality of the Four SubspacesFundamental Theorem of Linear Algebra, Part II • The nullspace is the orthogonal complement of the row space in Rn • The left Nullspace is the orthogonal complement of the column space in Rm

  14. Proof • The nullspace is the orthogonal complement of the row space in Rn

  15. The Whole Picture dim r dim r A xr= b C(A) xr C(AT) b A x= b Rn Rm xn dim n- r A xn= 0 N(A) N(AT) dim m- r

  16. Uniqueness of The Typical Solution • Every vector in the column space comes from one and only one vector xr from the row space • Proof: suppose there are two xr ,yr from the row space such that Axr =Ayr =b, then Axr -Ayr = A(xr -yr ) = 0 (xr -yr ) is in row space and nullspace hence must be 0 • The matching of dim in row and column spaces

  17. Deep Secret of Linear AlgebraPseudo-inverse • Throw away the two null spaces, there is an r by r invertible matrix hiding insider A. • In some sense, from the row space to the column space, A is invertible • It maps an r-space in n space to an r-space in m-space

  18. Invertible Matrices • Any n linearly independent vector in Rn must span Rn . They are basis. • So Ax = b is always uniquely solvable • A is invertible

  19. Projection • Projection onto an axis (a,b) x axis is a vector subspace

  20. Projection onto an Arbitrary Line Passing through 0 (a,b)

  21. Projection on to a Plane

  22. Projection onto a Subspace • Input: • Given a vector subspace V in Rm • A vector b in Rm… • Desirable Output: • A vector in x in V that is closest to b • The projection x of b in V • A vector x in V such that (b-x) is orthogonal to V

  23. How to Describe a Vector Subspace V in Rm • If dim(V) = n, then V has n basis vectors • a1, a2, …, an • They are independent • V = C(A) where A = [a1, a2, …, an]

  24. Projection onto a Subspace • Input: • Given n independent vectors a1, a2, …, an in Rm • A vector b in Rm… • Desirable Output: • A vector in x in C([a1, a2, …, an]) that is closest to b • The projection x of b in C([a1, a2, …, an]) • A vector x in V such that (b-x) is orthogonal to C([a1, a2, …, an])

  25. Think about this Picture dim r dim r A xr= b C(A) xr C(AT) b A x= b Rn Rm xn dim n- r A xn= 0 N(A) N(AT) dim m- r

More Related