1 / 20

Linear Spaces

Linear Spaces. Row and Columns Spaces From: D.A. Harville , Matrix Algebra from a Statistician’s Perspective , Springer. Chapter 4. Definitions - I. Column Space of m x n matrix A ≡ set of all m -dimensional column vectors that can be expressed as linear combinations of columns of A

bailey
Download Presentation

Linear Spaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Linear Spaces Row and Columns Spaces From: D.A. Harville, Matrix Algebra from a Statistician’s Perspective, Springer. Chapter 4

  2. Definitions - I • Column Space of mxn matrix A ≡ set of all m-dimensional column vectors that can be expressed as linear combinations of columns of A • Row Space of mxn matrix A ≡ set of all n-dimensional row vectors that can be expressed as linear combinations of rows of A

  3. Definitions - II • Linear Spaces: A non-empty set V of matrices of the same dimensions is a linear space if: • For every A in V, and every B in V, A+B is in V • For every A in V, and every scalar k, kA is in V • A1,…,Ak in V and scalars x1,…,xkx1A1+…+xkAk is in V • Examples • Column space of any mxn matrix is a linear space of mx1 vectors • Set containing all mxn matrices is a linear space • Set of all nxn symmetric matrices is a linear space (sums and scalar multiples) • All linear spaces contain the null matrix of correct dimension • {0} is a linear space with only the null matrix included

  4. Notation • C(A) ≡ Column space of the matrix A • R(A) ≡ Row space of the matrix A • Rmn≡ Linear space of all mxn matrices • Rn ≡ Linear space of all nx1 column vectors (1xn row vectors) • R(In) = Rn ≡ Linear space of all 1xn row vectors • C(In) = Rn ≡ Linear space of all nx1 column vectors • x S x is an element of S • xS x is not an element of S

  5. 2 Lemmas Involving Row/Column Spaces and Linear Spaces Lemma 4.1.1. For any matrix A, y C(A) if and only if y’  R(A’) y C(A)  y = Axfor some x  y’ = (Ax)’ = x’A’  y’  R(A’) y’  R(A’)  y’ = x’A’for some x  (y’)’ = (x’A’)’ = Ax  y = Ax  y  C(A) Lemma 4.1.2. B ≡ mxnV ≡ Linear space of mxn matrices Then for any A  V, A+B  ViffB  V If A  V and B  V, then A+B  V by definition of a linear space If A  V and A+B  V then k1A + k2(A+B)  V by definition of a linear space. Let k1 = -1, k2 = 1  k1A + k2(A+B) = B and thus B  V

  6. Subspaces • Subset U of a linear space V is termed a subspace of V if it is a linear space. • Trivial Cases: (1) The null set {0} and (2) the entire set V • Column space C(A) of mxn matrix A is a subspace of Rm (set of all m-dimensional vectors) • Row space R(A) of mxn matrix A is a subspace of Rn (set of all n-dimensional vectors) • For any 2 subsets, S and T of a given set (say 2 subspaces of Rmxn), S is contained in T if all elements of S are elements of T (S  T) • If S  T and T  S, then S = T

  7. Results Involving Subspaces • Lemma 4.2.1. A ≡ mxn Then for any subspaces U of Rm and V of Rn : • C (A)  Uiff every column of A is in U, R(A)  Viff every row of A is in Vwhere A = [a1 … an] • If C (A)  U then all ai  U (Let xi = (0,…,0,1,0,…,0)’) • If ai  U i=1,…,n then y  C (A)  y = Ax  y = x1a1 + … + xnan and since U is linear space, y  U • Lemma 4.2.2. A ≡ mxnB ≡ mxpC ≡ qxn: • C (B)  C (A) iff there exists nxpF such that B = AF R(C)  R(A) iff there exists qxmL such that C = LA • Corollary 4.2.3. For any A ≡ mxnF ≡ nxpL ≡ qxm : C (AF)  C (A) and R(LA)  R(A) • Corollary 4.2.4. A ≡ mxnE ≡ nxkF ≡ nxpL ≡ qxmT ≡ sxm: • If C (E)  C (F) then C (AE)  C (AF) and if C (E) = C (F) then C (AE) = C (AF) • If R(L)  R(T) then R(LA)  R(TA) and if R(L) = R(T) then R(LA) = R(TA) • Lemma 4.2.5. A ≡ mxnB ≡ mxp: (1) C (A)  C (B) iffR(A’)  R(B’) (2) C (A) = C (B) iffR(A’) = R(B’) • (1)  A = BF  A’ = (BF)’ = F’B’

  8. Bases • Span of a finite set of matrices of common dimensions: • Finite nonempty set {A1,…,Ak}: set of all matrices being linear combinations of {A1,…,Ak} • Empty set: {0} (unique basis – see below for definition of basis) • Span of a finite set S of matrices written as sp(S) which is a linear space • sp({A1,…,Ak}) ≡ sp(A1,…,Ak) ≡ span of the set {A1,…,Ak} • Finite set S of matrices in linear space Vspans V if sp(S) = V • Basis for V is a finite linearly independent set of matrices in V that spans V • C(A) for A ≡ mxnis spanned by the set of its n columns. If lin. indep.  basis • R(A) for A ≡ mxnis spanned by the set of its m rows. If lin. indep.  basis • If the columns or rows of A are not linearly independent, they are not a basis

  9. Natural Bases for Rmnand Linear Space of nxn Symmetric Matrices

  10. Results Involving a Basis - I Lemma 4.3.1. A1,…,Ap and B1,…,Bq in linear space V If {A1,…,Ap} spans V, then so does {A1,…,Ap,B1,…,Bq} If {A1,…,Ap,B1,…,Bq} spans V and if B1,…,Bq are linear functions of A1,…,Ap then {A1,…,Ap} spans V Theorem 4.3.2. V ≡ linear space spanned by a set of r matrices. Let S ≡ set of k lin. indep. Matrices in V Then k≤ r and if k = r, then S is a basis for V Corollary 4.3.3. The number of matrices in a linearly independent set of mnmatrices cannot exceed mn. Number of matrices in a linearly independent set of nnsymmetric matrices cannot exceed n+n(n-1)/2 = n(n+1)/2 Theorem 4.3.4. Every linear space of mnmatrices has a basis Lemma 4.3.5. Matrix A in linear space V can be written as a unique linear combination of matrices in any particular basis {A1,…,Ak} with unique coefficients x1,…,xk and A = x1A1+…+xkAk

  11. Results Involving a Basis - II Theorem 4.3.6. Any two bases of the same linear space V have the same number of matrices. The number of matrices in the basis is its dimension: dim(V) Theorem 4.3.7. If linear space V is spanned by set of r matrices, then dim(V) ≤ r. If there is set of k linearly independent matrices in V, dim(V) ≥ k Theorem 4.3.8. If U is a subspace of linear space V, dim(U) ≤ dim(V) Theorem 4.3.9. Any set of r linearly independent matrices in r-dimensional linear space V is a basis for V Theorem 4.3.10. U, V≡ linear spaces of mnmatrices with U V and dim(U) = dim(V), then U = V. Aside: dim(Rmn) = mn  dim(V) ≤ mn Theorem 4.3.11. Any set S that spans a linear space V of mnmatrices contains a subset that is basis for V. The number of matrices in the subset is dim(V) Theorem 4.3.12. For any set S of r lin. indep. matrices in k-dimensional V, there exists a basis for V containing the r matrices in S and k-r additional linearly independent matrices

  12. Rank of a Matrix • Row Rank – Dimension of the row space of A (number of lin. indep. Rows) • Column Rank – Dimension of the column space of A Theorem 4.4.1. For any matrix A, row rank = column rank Theorem 4.4.2. NonnullA≡ mxn with row rank = r, column rank = c. Then  B ≡ mcand L ≡ cnsuch that A = BL. Similarly,  K ≡ mrand T ≡ rnsuch that A = KT. Theorem 4.4.2. Basis for C(A) has c vectors: {b1,…,bc} B=[b1…bc] . C(B) = sp(b1,…,bc) = C(A)   L ≡ cnsuch that A = BL Basis for R(A) has r vectors: {t1’,…,tr’} T=[t1…tr]’ . R(T) = sp(t1’,…,tr’) = R(A)   K ≡ mrsuch that A = KT Theorem 4.4.1. Assume A≡ nonnull (A = 0 r = c = 0) A = BLA = KT  R(A)  R(L) and C(A)  C(K) R(L) spanned by the c rows of LC(K) spanned by the r columns of K  r ≤ dim(R(L)) ≤ c and c ≤ dim(C(K)) ≤ r r = c

  13. Results on Dimensions/Ranks Lemma 4.4.3. For any A≡ mn: rank(A) ≤ m, rank(A) ≤ n Theorem 4.4.4. A≡ mn,B≡ mp, C≡ qn: [Theorem 4.3.8.] If C(B)  C(A) then rank(B) ≤ rank(A). If R(C)  R(A) then rank(C) ≤ rank(A) Corollary 4.4.5. A≡ mn,F≡ np: rank(AF) ≤ rank(A) rank(AF) ≤ rank(F) [Corollary 4.2.3.] Theorem 4.4.6. A≡ mn,B≡ mp, C≡ qn: [Theorem 4.3.10.] If C(B)  C(A) and rank(B) = rank(A) then C(B) = C(A). If R(C)  R(A) and rank(C) = rank(A) then R(C) = R(A) Corollary 4.4.7. A≡ mn,F≡ np: If rank(AF) = rank(A) thenC(AF) = C(A). If rank(AF) = rank(F) thenR(AF) = R(F)

  14. Nonsingular, Full Row Rank, and Full Column Rank Matrices • Among mnmatrices, maximum rank = min(m,n) • Mininimum rank is 0, only for the null matrix 0 • A≡ mnis said to be full row rank if rank(A) = m • A≡ mnis said to be full column rank if rank(A) = n • A≡ nnis said to be nonsingular if rank(A) = n (full row and column rank) Theorem 4.4.8. A≡ mnnonnull matrix of rank r. Then  B≡ mrand T≡ rnsuch that A = BT For any B≡ mrand T≡ rns.t.A = BT, rank(B) = rank(T) = r B ≡ full column rank and T ≡ full row rank By Theorem 4.4.2.:  B≡ mrand T≡ rnsuch that A = BT By their dimensions: rank(B) ≤ r and rank(T) ≤ r (Lemma 4.4.3). By definition: rank(A) = rank(BT) = r By Corollary 4.4.5.: r = rank(BT) ≤ rank(B) ≤ r, r = rank(BT) ≤ rank(T) ≤ r rank(B) = rank(T) = r

  15. Decomposition of Rectangular Matrix

  16. Definitions and Results Involving Ranks Theorem 4.4.10. A≡ mnwith rank(A) = r A has r linearly independent rows and columns The rrsubmatrix Ar made up of the r lin. Indep. rows and the r lin. indep. columns is nonsingular. Any of the remaining rows or columns are linearly dependent of those of Ar. There is no submatrix of A with rank > r Corollary 4.4.11. Any nnsymmetric matrix of rank r has an rrprincipal submatrix that is nonsingular Useful Equalities rank(A’) = rank(A) (r = max # of linearly independent rows and columns of matrix) For any nonzero scalar k, rank(kA) = rank(A) (No linear dependencies changed) rank(-A) =rank(A) (special case of 2))

  17. Partitioned Matrices and Sums of Matrices - I

  18. Partitioned Matrices and Sums of Matrices - II

  19. Partitioned Matrices and Sums of Matrices - III

  20. Partitioned Matrices and Sums of Matrices - IV

More Related