1 / 15

Parallel and Distributed Audio (Image/Video) Concealment Using Nonlocal Sparse Representation

Parallel and Distributed Audio (Image/Video) Concealment Using Nonlocal Sparse Representation. Xin Li LDCSEE, WVU. Neural Basis of Media Content. Multimedia: a combination of content forms. “Missing Data” Problem. Skype. MSN. packet loss. p. … …. … …. p. q. Sequential. Parallel.

lamond
Download Presentation

Parallel and Distributed Audio (Image/Video) Concealment Using Nonlocal Sparse Representation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel and Distributed Audio (Image/Video) Concealment Using Nonlocal Sparse Representation Xin Li LDCSEE, WVU

  2. Neural Basis of Media Content Multimedia: a combination of content forms

  3. “Missing Data” Problem Skype MSN packet loss p …… …… p q Sequential Parallel

  4. Error Concealment in Image Communication image video Sequential Parallel

  5. Why Parallel? • Avoid choosing scanning order and its associated error propagation problem • Modeling multimedia signal at a patch (e.g., speech frame, image block) level instead of data (e.g., speech sample, image pixel) level • Parallelism has to work with distributed processing together (redundancy is exploited to fight against uncertainty)

  6. Prior Model in the Patch Space MRP x y z x y z y x Manifold constraint of sensory signals: dimensionality of local subspace is <<patch dimension p z

  7. Sparsity-based Prior Model Nonlinear Dimensionality Reduction By Locally Linear Embedding (LLE) Roweis and Saul, Science’2000 B0 B2 B1 B3 B4 Sparsifying transform

  8. Optimal sparsifying transform (KLT) Approximated solution (FFT/DCT) Nonlocal Sparse Representation n-D B0 B1 Bk … Pack into (n+1)-D Array D (n+1)D-FFT Thresholding ^ ^ ^ n-D B0 B1 Bk … Pack into (n+1)-D Array D (n+1)D-IFFT

  9. RPP B1 B3 B2 B0 B4 Why is it Nonlocal? Local neighbors in range (photometric similarity) nonlocal patches in domain (geometric proximity)

  10. Algorithm Flowchart Projection onto observation constraint Projection onto prior constraint Data clustering in Rp PDAC algorithm Outer loop Inner loop Better estimate of local neighborhood in the patch space Better estimate of missing data in the time domain

  11. Experimental Results (I) sequential-I (parametric) sequential-II (nonparametric) parallel (this work)

  12. Experimental Results (II) outer loop proceeds

  13. Extension into Images exemplar based Guleryuz’s scheme This work exemplar based Guleryuz’s scheme This work original damaged original damaged

  14. Parallel and Distributed Processing (PDP) • Dominant form of Connectionism (models mental or behavioral phenomena as the emergent processes of interconnected networks of simple units.) • More widely known as neural networks since 1980s

  15. Network Model Interpretation observation Y B2 B4 scene X B3 X1 B1 patch B X2 I hidden T Belief propagation in factor graph

More Related