1 / 32

Rank-Sparsity Incoherence for Matrix Decomposition

Reading Group. Rank-Sparsity Incoherence for Matrix Decomposition. ( MIT EECS: Venkat Chandrasekaran , Sujay Sanghavi , Pablo A. Parrilo , and Alan S. Willsky , in 2009). Presenter: Zhe Chen ECE / CMR Tennessee Technological University February 18, 2011. Outline. Overview Introduction

duard
Download Presentation

Rank-Sparsity Incoherence for Matrix Decomposition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reading Group Rank-Sparsity Incoherence for Matrix Decomposition (MIT EECS: VenkatChandrasekaran, SujaySanghavi, Pablo A. Parrilo, and Alan S. Willsky, in 2009) Presenter:Zhe Chen ECE / CMR Tennessee Technological University February 18, 2011

  2. Outline Overview Introduction Rank-Sparsity Incoherence Exact Decomposition Using Semidefinite Programming Simulation Results One-Sentence Summary

  3. Overview Given a matrix formed by adding an unknown sparse matrix to an unknown low-rank matrix. The problem is to decompose the given matrix into its sparse and low-rank components. A notion of rank-sparsity incoherence is developed. Sufficient conditions for exact recovery are given. When the sparse and low-rank matrices are drawn from certain natural random ensembles, the sufficient conditions for exact recovery are satisfied with high probability.

  4. Outline Overview Introduction Rank-Sparsity Incoherence Exact Decomposition Using Semidefinite Programming Simulation Results One-Sentence Summary

  5. The Problem Indeed, there are a number of scenarios in which a unique splitting of C into “low-rank” and “sparse” parts may not exist. Conditions should be met.

  6. Identifiability Problems (1) • being small implies that M cannot be very sparse • 1. If the low-rank matrix is very sparse, impose certain conditions on the row/column spaces of the low-rank matrix: • For a matrix M let T(M) be the tangent space at M with respect to the variety of all matrices withrank less than or equal to rank(M) • Here is the spectral norm

  7. Identifiability Problems (2) • being small implies singular values are not too large • 2. If the sparse matrix has all its support concentrated in one row/column, impose conditions on the sparsity pattern of the sparse matrix: • For a matrix M let be the tangent space at M with respect to the variety of all matrices with number of non-zero entries less than or equal to |support(M)|

  8. Rank-sparsity incoherence • However, for a given matrix M, it is impossible for both quantities and to be small simultaneously. • The authors develop a notion of rank-sparsity incoherence, an uncertainty principle between the sparsity pattern of a matrix and its row/column spaces.

  9. Optimization Formulation • In general solving the decomposition problem is NP-hard • Convex relaxation is employed here • The following optimization formulation is proposed to recover A* and B* given C = A* + B*: • is a trade-off parameter • is the decomposed

  10. Conditions This paper provides a simple deterministic condition for exact recovery. The conditions only depend on the row/column spaces of the low-rank matrix B* and the support of the sparse matrix A*.

  11. Outline Overview Introduction Rank-Sparsity Incoherence Exact Decomposition Using Semidefinite Programming Simulation Results One-Sentence Summary

  12. Tangent-Space Identifiability (1) The algebraic variety of rank-constrained matrices is defined as: For any matrix , the tangent space T(M) with respect to P(rank(M)) at M is the span of all matrices with either the same row-space as M or the same column-space as M. Let Then:

  13. Tangent-Space Identifiability (2) The variety of support-constrained matrix is defined as For any matrix , the tangent space with respect to S(|support(M)|) at M is given by

  14. Tangent-Space Identifiability (3) (Proof is omitted here.) A necessary and sufficient condition for unique recovery is: That is, the subspaces and have a trivial intersection.

  15. Rank-Sparsity Uncertainty Principle (Proof is omitted here.) For any matrix both and cannot be simultaneously small. Note that Proposition 1 is for different matrices, while Theorem 1 is for the same matrix.

  16. Outline Overview Introduction Rank-Sparsity Incoherence Exact Decomposition Using Semidefinite Programming Simulation Results One-Sentence Summary

  17. Optimality Condition (1) • Notations • The orthogonal projection onto the space is denoted , which simply sets to zero those entries with support not inside support(A*). • The subspace orthogonal to is denoted . The projection onto is denoted . • The orthogonal projection onto the space is denoted • The space orthogonal to is denoted , and the corresponding projection is denoted .

  18. Optimality Condition (2) (Proof is omitted here.)

  19. Sufficient Conditions Based on and (Proof is omitted here.) Given matrices A* and B* with , and from Proposition 1, condition (1) of Proposition 2 is satisfied. If a slightly stronger condition holds, there exists a dual Q that satisfies the requirements of condition (2) of Proposition 2.

  20. Sparse and Low-Rank Matrices with (Proof is omitted here.) (Proof is omitted here.)

  21. Sparse and Low-Rank Matrices with (Proof is omitted here.) This is a result with deterministic sufficient conditions on exact decomposability.

  22. Decomposing Random Sparse and Low-Rank Matrices (1) (Proof is omitted here.) Sparse and low-rank matrices drawn from certain natural random ensembles satisfy the sufficient conditions of Corollary 3 with high probability. Random sparsity model

  23. Decomposing Random Sparse and Low-Rank Matrices (2) (Proof is omitted here.) • Random orthogonal model • Consider low-rank matrices in which the singular vectors are chosen uniformly at random from the set of all partial isometries • Low-rank matrices drawn from such a model have incoherent row/column spaces.

  24. Decomposing Random Sparse and Low-Rank Matrices (3) (Proof is omitted here.) Applying these two results in conjunction with Corollary 3, we have that sparse and low-rank matrices drawn from the random sparsity model and the random orthogonal model can be uniquely decomposed with high probability.

  25. Outline Overview Introduction Rank-Sparsity Incoherence Exact Decomposition Using Semidefinite Programming Simulation Results One-Sentence Summary

  26. Simulation Results (1)

  27. Simulation Results (2) Define: Declare success in recovering (A*, B*) if . Exact recovery is possible for a range of Modify (1.3) with

  28. Simulation Results (3) Compute the difference between solutions for some t and as follows: Generate a random that is 25-sparse and a random with rank = 2 If a reasonable guess for t (or ) is not available, one could solve (5.2) for a range of t and choose a solution corresponding to the “middle” range in which is stable and near zero.

  29. Simulation Results (4)

  30. Outline Overview Introduction Rank-Sparsity Incoherence Exact Decomposition Using Semidefinite Programming Simulation Results One-Sentence Summary

  31. One-Sentence Summary Sufficient conditions on sparse and low-rank matrices are provided so that the SDP exactly recovers such matrices.

  32. Thank you!

More Related