1 / 21

CS-MUVI Video compressive sensing for spatial multiplexing cameras

CS-MUVI Video compressive sensing for spatial multiplexing cameras. Aswin Sankaranarayanan, Christoph Studer , Richard G. Baraniuk Rice University. Single pixel c amera. Photo-detector. Digital m icro-mirror device. Single pixel c amera. Each configuration of

jenna
Download Presentation

CS-MUVI Video compressive sensing for spatial multiplexing cameras

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CS-MUVIVideo compressive sensing for spatial multiplexing cameras Aswin Sankaranarayanan, ChristophStuder, Richard G. Baraniuk Rice University

  2. Single pixel camera Photo-detector Digital micro-mirror device

  3. Single pixel camera • Each configuration of • micro-mirrors yield ONE • compressive • measurement • Non-visible • wavelengths • Sensor material costly • in IR/UV bands • Light throughput • Half the light in the scene is directed to the photo-detector • Much higher SNR as compared to traditional sensors Photo-detector Digital micro-mirror device

  4. Single pixel camera • Each configuration of micro-mirrors yield ONEcompressive measurement • staticscene assumption • Key question: • Can we ignore motion • in the scene ? Photo-detector Digital micro-mirror device

  5. SPC on a time-varying scene time varying scene • Naïve approach: Collect Wmeasurements together to compute an estimate of an image what happens ? t=1 t=W measurements ? compressive recovery

  6. SPC on a time-varying scene t=W (large) t=W (small) t=1 Tradeoff Temporal resolution vs. spatial resolution Large W Higher spatial resolution More motion blur Small W Less motion blur Lower spatial resolution

  7. SPC on a time-varying scene sweet spot Lower spatial res. Higher temporal res. Higher spatial res. Lower temporal res.

  8. Dealing with Motion • Motion information can help in obtaining better tradeoffs [Reddy et al. 2011] • State-of-the-art video compression

  9. Dealing with Motion • Motion information can help in obtaining better tradeoffs [Reddy et al. 2011] • State-of-the-art video compression naïve reconstruction motion estimates

  10. Key points • Motion blur and the failure of the sparsity assumption • Use least squares recovery ? • Recover scene at lower spatial resolution • Lower dimensional problem requires lesser number of measurements • Tradeoff spatial resolution for temporal resolution • Least squares and random matrices • Random matrices are ill-conditioned • Noise amplification • Hadamardmatrices • Orthogonal (no noise amplification) • Maximum light throughput • Optimalfor least squares recovery [Harwit and Sloane, 1979]

  11. Hadamard + least sq. recovery Hadamard Random

  12. Hadamard+ least sq. recovery

  13. Designing measurement matrices • Hadamard matrices • Higher temporal resolution • Poor spatial resolution • Random matrices • Guarantees successful l1 recovery • Full spatial resolution • Can we simultaneouslyhave both properties in the same measurement matrix ?

  14. Dual-scale sensing (DSS) matrices Key Idea: Constructing high-resolution measurement matrices that have good properties when downsampled 1. Start with a row of the Hadamard matrix 3. Add high-freq. component 2. Upsample

  15. CS-MUVI: Algorithm outline 1.obtain measurements with DSS matrices t=t0 t=t0+W t=T t=1 t=W 2.low-resolution initial estimate 3.motion estimation 4.compressive recovery with motion constraints

  16. Simulation result

  17. CS-MUVI on SPC Single pixel camera setup Object InGaAs Photo-detector (Short-wave IR) SPC sampling rate: 10,000 sample/s Number of compressive measurements: M = 16,384 Recovered video: N = 128 x 128 x 61 = 61*M

  18. CS-MUVI: IR spectrum initial estimate Upsampled Recovered Video Joint work with Xu and Kelly

  19. CS-MUVI on SPC Naïve frame-to-frame recovery CS-MUVI Joint work with Xu and Kelly

  20. CS-MUVI summary • Key ingredients • Novel Measurement matrix design • Exploiting state-of-the-art motion model • One of first practicalvideorecovery algorithm for the SMC dsp.rice.edu

  21. CS-MUVI summary • Limitations • Need a priori knowledge of object speed • Motion at low-resolution • Robustness to errors in motion estimates • Future work • Dual-scale to multi-scale matrix constructions • Multi-frame optical flow • Online recovery algorithms dsp.rice.edu

More Related