170 likes | 198 Views
Learn about solutions, rank conditions, and SVD methods for systems of linear equations in computer vision. Discover how to solve, characterize, and optimize homogeneous and over/under-determined systems.
E N D
Systems of Linear Equations(see Appendix A.6, Trucco & Verri) CS485/685 Computer Vision Prof. George Bebis
Systems of linear equations • An arbitrary system of m linear equations in n unknowns can be written as: where: or Example (m=n=2):
Solutions of Ax=b (m = n) • Characterize the solutions of Ax=b using conditions on the rank of A and A|b (i.e., augmented matrix).
Solutions of Ax=b (m=n) • The system has one solution if : rank(A|b) = rank(A) = n Solution: i.e., b be expressed as a linear combination of the columns of A:
Solutions of Ax=b (m=n) The following statements are equivalent: (a)rank(A|b) = rank(A) = n (b) A is invertible (c) (d) b has a unique expansion in the column space of A
Solving Ax=b using SVD • Assuming that A=UDVT, then UDVTx=b or DVTx=UTb • Setting VTx=z and UTb=d, we have Dz=d (1) Compute z=D-1d (i.e., assume no zeroes in the diagonal) (2) Compute solution x =Vz Ax=b
Solutions of Ax=b (m=n) (2) The system has no solutionif rank(A|b) > rank(A) b cannot be expressed as a linear combination of the columns of A e.g., using substitution leads to the contradiction 16=9
Solutions of Ax=b (m=n) (3) The system has infinitely many solutionsif rank(A|b) = rank(A) < n - Less equations than unknowns (i.e, free variables). - b can be expressed as a linear combination of the columns of A in more than one ways.
Homogeneous system: Ax=0 (m = n) • If b=0, then Ax=0 is called homogeneous. (1) Has the trivial solution x=0 iff rank(A) = n (i.e., A is invertible) (2) Has a non-trivial solution iff rank(A) < n (i.e., A is singular)
m>n m<n Over/Under determined Systems
Solving Ax=b (m > n) • Consider the over-determined system of linear equations: • Let r be the residual vector for some x: • The vector x* which yields the smallest possible residual is called a least-squaressolution:
Solving Ax=b (m > n) (cont’d) • Although a least-squares solution always exist, it might not be unique! • The least-squares solution x with the smallest norm ||x|| is unique and it is given by:
Computing A+ using SVD • If ATA is ill-conditioned (or singular), we can use SVD to obtain a least squares solution as follows: where: (where t is a small threshold)
Homogeneous systems • The minimum-norm solution is x=0; need to modify the meaning of a least-squares solution by imposing the constraint: • This is a "constrained" optimization problem:
Homogeneous systems (cont’d) • The solution for homogeneous systems is not always unique. • Special case: Solution: (vnis the last column of V; the one corresponding to the smallest σ)
Homogeneous systems (cont’d) • General case: Solution: with (vn-k+1, …,vnare the last columns of V ; correspond to the smallest σ’s)