1 / 7

Sample complexity for Multiresolution ICA

Sample complexity for Multiresolution ICA. Doru Balcan Nina Balcan Avrim Blum. Introduction. Adaptive image representations Efficient encoding / Compression

carla-sloan
Download Presentation

Sample complexity for Multiresolution ICA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sample complexity for Multiresolution ICA Doru Balcan Nina Balcan Avrim Blum

  2. Introduction • Adaptive image representations • Efficient encoding / Compression • Although fixed transforms like DCT, or wavelets are successful (e.g. JPEG, JPEG2000), adapting the representation to the signals can improve coding efficiency • Feature extraction • ICA, Sparse coding features are similar to receptive fields of simple cells in primary visual cortex. [Olshausen&Field’97, Bell&Sejnowski’97] • Multiresolution ICA • Hybrid method: learns an ICA basis for each subband • Improves on traditional linear adaptive methods • can handle larger images (64£64), instead of small patches (8£8) • Improves on conventional multiscale transforms • adaptivity facilitates coefficient compression

  3. MrICA works well in practice • Improvement of ~4dB over the wavelet representation 64£64 images reconstructed at 20dB MrICA error Wavelets error MrICA reconstruction Wavelet reconstruction original

  4. MrICA – problem setup • S={x1,…,xm} a data sample (images), xi2Rd iid ~ p(¢) • M – a d£d invertible matrix (the MR transform) • Goal: 8k, find Wk2Rdk£dk s.t. y[k]=Wk(Mkx) has maximally independent components M1 d1 M2 d2 K subbands … MK dK d x MKx y[k] ICA Subband k Mk Wk

  5. ICA objectives Denote data r.v. u2Rn; demixing matrix B2Rn£n; new repres. vector z=Bu Assume data is centered and whitened (isotropic) • Minimize Mutual Information • Maximize non-gaussianity • (Approximately) Diagonalize certain functionals • Finite set of symmetric matrices T1, T2, …, Tr (depending on data) • X,Y indep. iff Ex,y(f(x)g(y))=Ex(f(x)) Ey(g(y)) constant true marginal entropy of bi¢u

  6. Related Sample Complexity Results • Sample Complexity for (classic) ICA • [Samarov&Tsybakov’04] • strong assumptions: • n-dim pdf is product of smooth 1-dim. pdfs • strong result: • estimated and true matrices are close in value • uses diagonalization of functional Eu[rp(u)rTp(u)] • … and for Kernel PCA • e.g., [Shawe-Taylor&coll.’05] • “whp, estimated and true matrices are close in quality” • The type of result we are seeking for ICA

  7. Results • Notation • , are the true and the estimatedquality measures (max) • , are the true and the estimatedoptimal matrices • We want to prove that • Suff. to prove is small for all B • (at least for and ) • Theorem: For kurtosis, O(d poly(1/)) sufficient to have whp simultaneously for all B. Please visit our poster!

More Related