The power of Convex Relaxation: Near-optimal matrix completion Emmanuel J. Candes and Terence Tao March, 2009

1. The power of Convex Relaxation:Near-optimal matrix completionEmmanuel J. Candes and Terence TaoMarch, 2009 Presenter: ShujieHou February, 04th,2011 Department of Electrical and Computer Engineering Cognitive Radio Institute Tennessee Technological university

2. The notations and basic knowledge on matrix completion are assumed to be known in this set of slides!

3. contribution • Concerning with the theoretical underpinnings of matrix completion, specifically, quantifying the minimum number of entries needed to recover a low-rank matrix exactly. • Developing a simple hypotheses about the matrix which makes the matrix recoverable by semidefinite programming from nearly minimally sampled entries.

4. Information theoretic limit • Assume , it is singular value decomposing (SVD) • The degrees of freedom of this matrix by counting the number of the parameters in SVD • However, the minimum number of samples should be on the order of rather than on the order of based on coupon collector’s effect. Theoretic limit

5. Model review • This paper develops three useful matrix models for which nuclear norm minimization is guaranteed to succeed from nearly minimally sampled entries. Relax

6. Strong incoherence property Basis vector in Euclidean space Strong incoherence property is that there exists parameter Implies that the eigenvector is spread out

7. The first Theorem • Matrices obeying strong incoherence property with a small value of the parameter can be recovered from few entries Practical problem: Do many matrices of interest obey the strong incoherence property?

8. First model: uniformly bounded model(1) • In all, the only assumption about this model is that the singular vectors has small components.

9. First model: uniformly bounded model(2) This theorem shows that the uniformly bounded model satisfies the strong coherence property with large probability. This means that theorem (1.1) suits this model with large probability

10. Second model: Low-rank low-coherence model This model is a special case of the first model. Any theorems and lemma of course suites for this model. Namely, it obeys strong incoherence property, also, theorem 1.1 can be specialized for this model.

11. Third model: Random orthogonal model This model is also a special case of the first model. Based on the above three models, it can be shown all of them obey strong incoherence property It proves that the matrices with large probability obey the strong coherence property

12. Proof of theorem Revisit Instead, a more generalized theorem The proof of this theorem

13. A necessary condition (1)

14. A necessary condition (2) • Give a lower bound of sampling number for matrix completion

15. Proof of theorem (1.2)(1) Y is a dual certificate of M Note: In order to prove theorem(1.2), what needs to do is to prove dual certicateY of M obey this lemma.

16. Proof of theorem (1.2)(2) Introducing a new operator The crux of the proof is to bound the spectrum norm of

17. The proof of theorem (1.2)(3) • “Exact matrix recovery via convex optimization” makes use of the tools in asymptotic geometric analysis to prove the key result. • This paper employs moment methods in random matrix theory to estimate the spectral norm of . • The details of proof for every lemma and theorem is from page13-page45.

18. discussions • Many algorithms have been proposed for matrix completion with reasonable time cost which implies its practical usefulness. • Can the matrix completion problem be robust to noise?

19. Matrix completion with noiseEmmanuel J. Candes and Yaniv Plan Presenter: ShujieHou February, 04th,2011 Department of Electrical and Computer Engineering Cognitive Radio Institute Tennessee Technological university

20. Contributions • Mainly focuses on the review of the previous theoretical results (the two previously presented papers). • Proves the results that matrix completion is robust under the condition that observed entries are corrupted with a small amount of noise.

21. Noise model If the observations are or The optimization model to recover noisy elements Relax from equality to inequality to tolerate small noise

22. Key theorem • Roughly speaking, the theorem states that when perfect noiseless recovery occurs, then matrix completion is stable vis a vis perturbations.

23. Oracle model Assume the eigenvector space is known by oracle. The solution of this model achieves The solution of the above model The solution of the noiseless model

24. Numerical experiments(1) Low-rank matrix is Noisy component is

25. Numerical Experiments(2) • For the large size matrix, for example, 1000 by 1000, the RMS is 0.24. • Big improvement than doing nothing at all!

26. Numerical Experiments(3) • Regularized model for noisy observations • is set to be the smallest possible value such that if the solution of the above model is zero.

27. Experimental results(4) Regularized model RMS error achievable for oracle model Theoretic estimated error of oracle model

28. Experimental results(5)

29. Experimental results(6)

30. Discussions • Expecting the tremendous growth for matrix completion in the next few years. • Whether one can recover low-rank matrices from a few general linear functional. • Computational algorithms.

31. Questions? • Thank you!