Initialization enhancer for non-negative matrix factorization

1. Initialization enhancer for non-negative matrix factorization Zhonglong Zheng, Jie Yang, YitanZhu Engineering Applications of Artificial Intelligence 20 (2007) 101–110 Presenter Chia-Cheng Chen

2. Outline • Introduction • Non-negative matrix factorization algorithm • Initializing NMF with different techniques • Experimental results • Conclusion

3. Background(1/2)

4. Background(2/2)

5. Introduction • NMF has been applied to many areas such as dimensionality reduction, image classification, image compression. • However, particular emphasis has to be placed on the initialization of NMF because of its local convergence, although it is usually ignored in many documents.

6. Non-negative matrix factorization algorithm(1/4) • Non-negative matrix factorization (NMF) algorithm where Dimensionality reduction is achieved when r < N

7. Non-negative matrix factorization algorithm(2/4) • Euclidean distance • Update rule

8. Non-negative matrix factorization algorithm(3/4) • KL divergence • Update rule

9. Non-negative matrix factorization algorithm(4/4) • SJTU-face-database • 400 images • Size: 64x64

10. Initializing NMF with different techniques(1/5) • Three techniques • PCA-based initialization • Clustering-based initialization • Gabor-based initialization

11. Initializing NMF with different techniques(2/4) • PCA-based initialization m x N matrix X Use SVD compute the eigenvectors and eigenvalues

12. Initializing NMF with different techniques(3/5) • PCA-based initialization

13. Initializing NMF with different techniques(4/5) • Clustering-based initialization (Fuzzy c-means) • Membership matrix • Objective function • Update rule

14. Initializing NMF with different techniques(5/5) • Gabor-based initialization • Gabor kernals where • Gabor feature

15. Experimental results

16. Experimental results

17. Conclusion • Non-negative matrix factorization is a useful tool in the analysis of a diverse range of data. • Researchers often take random initialization into account when utilizing NMF. • In fact, random initialization may make the experiments unrepeatable because of its local minima property, although neural networks are not.