290 likes | 604 Views
Structured Sparse Principal Component Analysis. Authors: Rodolphe Jenatton , Guillaume Obozinski , Francis Bach. Reading Group Presenter: Peng Zhang Cognitive Radio Institute Friday, October 01, 2010. Outline. Introduction (in Imaging Sense) Principal Component Analysis (PCA)
E N D
Structured Sparse Principal Component Analysis Authors: RodolpheJenatton, Guillaume Obozinski, Francis Bach Reading Group Presenter: Peng Zhang Cognitive Radio Institute Friday, October 01, 2010
Outline • Introduction (in Imaging Sense) • Principal Component Analysis (PCA) • Sparse PCA (SPCA) • Structured Sparse PCA (SSPCA) • Problem Statement • The SSPCA Algorithm • Experiments • Conclusion and Other Thoughts
Introduction (Imaging Sense) • The face recognition problem • A database includes a huge amount of faces • How to let computer to recognize different faces with database • The challenge • Huge amount of data • Computation complexity • The trick • Represent the face using a weighted “face dictionary” • Similar to code book in data compression • Example: An 200 X 200 pixel face can be represented by 100 coefficients using the “face dictionary” • The solution • Principal component analysis (PCA)
PCA • PCA • A compression method • Given a large amount of sample vectors {x} • 2nd moment statistics of the sample vectors • Eigen-decomposition finds the “dictionary” and “energy” of the dictionary codes • Eigen-vectors {v} form the “dictionary” • Eigen-values {d} give the “energy” of “dictionary” elements
PCA • Original signal can be represented using only part of the dictionary • Data is compressed with fewer elements • Meaning of “dictionary” v: • It is the weights of each elements in x • The problem for PCA for face recognition: No physical meaning for “dictionary”
PCA The Face Samples The “dictionary”, eigen-faces PCA These eigen-faces can reconstruct original faces perfectly, but make no sense in real life Face recognition
Structured SPCA Non-sparse Eigen-faces from PCA Sparse Eigen-faces from SPCA But the eigen-faces are still meaningless most of time • The SPCA goal: • Make dictionary more interpretable • The “sparse” solution: Limit the number of nonzeros
Structured SPCA Eigen-faces from SSPCA • The new idea, SSPCA • Eigen-faces will be meaningful when some structured constraints are set • Meaningful areas in faces are constrained in “grids”
Structured SPCA • This paper’s contribution • Add the “structure” constraint to make the dictionary more meaningful • How the constraint works • Meaningful dictionary is more close to “true” dictionary • Meaningful dictionary is more robust against noise • Meaningful dictionary is more accurate in face recognition
Outline • Introduction • Principal Component Analysis (PCA) • Sparse PCA (SPCA) • Structured Sparse PCA (SSPCA) • Problem Statement • The SSPCA Algorithm • Experiments • Conclusion and Other Thoughts
Problem Statement • From SPCA to SSPCA • The optimization problem • X is sample matrix, U is coefficient matrix, V is dictionary • ||.|| and are different types of norms • The trick in SPCA • L1 norm force the dictionary to be a sparse solution
Problem Statement Structured SPCA, however, deal with a mixed l1/l2 minimization: Right now it’s hard for me to understand the G and d
Problem Statement • In short, the norm constraints have the following effects • Dictionary has some structures • All non-zeros in the dictionary will be confined inside a grid
Outline • Introduction • Principal Component Analysis (PCA) • Sparse PCA (SPCA) • Structured Sparse PCA (SSPCA) • Problem Statement • The SSPCA Algorithm • Experiments • Conclusion and Other Thoughts
The SSPCA Algorithm • Making the dictionary sparser • The norm, • The new SSPCA problem:
The SSPCA Algorithm Methods to solve a sequence of convex problems
Excerpt from Author’s slide Excerpt from author’s slide:
Outline • Introduction • Principal Component Analysis (PCA) • Sparse PCA (SPCA) • Structured Sparse PCA (SSPCA) • Problem Statement • The SSPCA Algorithm • Experiments • Conclusion and Other Thoughts
Conclusion and Other Thoughts • Conclusion • This paper shows how to use SSPCA • SSPCA gets better performance in denoising, face recognition and classification • Other thoughts • Usually, the meaningful dictionary in communication signals is Fourier dictionary • But Fourier dictionary may not fit some transient signals or time-variant signals • How to manipulate the G, d and norms to set constraints for our needs?