1 / 15

Online Learning for Matrix Factorization and Sparse Coding

Online Learning for Matrix Factorization and Sparse Coding. Julien Mairal, Francis Bach, Jean Ponce and Guillermo Sapiro Journal of Machine Learning Research 2010. Introduction. This paper focuses on the large scale matrix factorization problem, including Dictionary learning for sparse coding

jennessa-cherry
Download Presentation

Online Learning for Matrix Factorization and Sparse Coding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Online Learning for Matrix Factorization and Sparse Coding Julien Mairal, Francis Bach, Jean Ponce and Guillermo Sapiro Journal of Machine Learning Research 2010

  2. Introduction • This paper focuses on the large scale matrix factorization problem, including • Dictionary learning for sparse coding • Non-negative matrix factorization (NMF) • Sparse principal component analysis (SPCA) • Contributions of this paper: • An iterative online algorithm is proposed for large scale matrix factorization • This algorithm is proved to converge almost surely to a stationary point of the objective function • This algorithm is shown to be much faster than previous methods in the experiment.

  3. Problem Statement • Classical dictionary learning problem Given a finite training set , the objective is to optimize the following function where • Online Learning This algorithm process one sample (or a mini-batch) at a time and sequentially minimize the following function:

  4. Basic Algorithm

  5. Dictionary Update

  6. Optimizing the Algorithm • Handling fixed-sized data sets • Scaling the “past” data • Mini-batch extension

  7. Proof of Convergence • Assumptions: • Main results

  8. Extensions to Matrix Factorization • Non-negative matrix factorization (NMF) • Non-negative sparse coding (NNSC) • Sparse principal component analysis (SPCA)

  9. Data for Experiment • 1.25 million patches from Pascal VOC’06 image database

  10. Online VS. Batch • Training data size: 1 million • OL1: • OL2: • OL3:

  11. Comparison with NMF and NNSC • NMF • NNSC

  12. Face Results

  13. Image Patches Results

  14. Inpainting Results • Image size: 12-Megapixel • Dictionary with 256 elements • Training data: 7 million 12 by 12 color patches

  15. Conclusion • A new online algorithm for learning dictionaries adapted to sparse coding tasks, and proven its convergence. • Experiments demonstrate that this algorithm is significantly faster than existing batch methods. • This algorithm can be extended to other matrix factorization problems such as non-negative matrix factorization and sparse principal component analysis.

More Related