30 likes | 171 Views
Lecture 14. PCA, pPCA, ICA. Principal Components Analysis. PCA is a data analysis technique to find the subspace of input space that carries most of the variance of the data. It is therefore useful as a tool to reduce the dimensionality of input space.
E N D
Lecture 14 PCA, pPCA, ICA
Principal Components Analysis • PCA is a data analysis technique to find the subspace of input space that carries most of the variance of the data. • It is therefore useful as a tool to reduce the dimensionality of input space. • The solution is found by an eigen-value decomposition of the sample covariance matrix. • PPCA is a probabilistic model that has ML solution equal to the PCA solution. It is a special case of FA with isotropic variance. • Therefore, the EM algorithm for FA is applicable for learning.
Independent Component Analysis • FA, PPCA have Gaussian prior models. In ICA we use non-Gaussian prior models (i.e. heavy tailed or bi-modal). • We also do not insist on dimensionality reduction, although that is also possible, but not necessary. • The canonical example is 2 speakers producing different mixtures of sound in 2 microphones that we wish to unmix. • The source distributions are non-Gaussian but independent, the noise model is typically Gaussian. • The simplest ICA model is square and has no noise. We can use a change of variable to go from sources to inputs. • Learning is through gradient descend with the ``natural gradient’’.