1 / 20

Adaptive Temporal Compressive Sensing for Video

Adaptive Temporal Compressive Sensing for Video. Xin Yuan, Jianbo Yang, Patrick Llull , Xuejun Liao, Guillermo Sapiro , David J. Brady and Lawrence Carin Department of ECE, Duke University. xin.yuan@duke.edu prl12@duke.edu. Outline. New Hardware : CACTI

ronat
Download Presentation

Adaptive Temporal Compressive Sensing for Video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Adaptive Temporal Compressive Sensing for Video Xin Yuan, Jianbo Yang, Patrick Llull,Xuejun Liao, Guillermo Sapiro, David J. Brady and Lawrence Carin Department of ECE, Duke University xin.yuan@duke.edu prl12@duke.edu

  2. Outline • New Hardware: CACTI Coded Aperture Compressive Temporal Imaging • New Concept: Adaptive TemporalCompressive Sensing in Real-Time • Experimental results

  3. Coded Aperture Compressive Temporal Imaging P. Llull, X. Liao, X. Yuan, J. Yang, D. Kittle, L. Carin, G. Sapiro, and D. J. Brady, “Coded aperture compressive temporal imaging,” Optics Express, vol. 21, no. 9, pp. 10526–10545.

  4. New Results of Color CACTI

  5. New Results of Color CACTI

  6. Coding Mechanism This opens the door of “adaptive temporal compressive sensing”.

  7. Our New Contribution: Adaptive Temporal Compressive Sensing

  8. Adaptive Temporal Compressive Sensing • Conventional Adaptive CS Design the measurement matrix H in the model When used in image CS, H is designed to adapt to the specific image • New concept: Adaptive the integration period of camera. We adaptively determine the number of frames collapsed to one measurement, using motion estimation in the compressed domain

  9. How? Motion Estimate Block Matching:

  10. Challenge Measurement Reconstruction Block Matching Computationally Infeasible Real-time?

  11. Solution: Block-Matching in Compressed Domain • Motion estimation is implemented on the measurements directly. • Adapting the compression ratio onlineis feasible due to the computational simplicity of block-matching.

  12. Proposed Method • Perform the block-matching on the current measurement and the previous one • Calculate the motion by averaging the largest several blocks • Determine the compression ratio by the table: This table is learnt with training dataset to maintain a constant PSNR of the reconstructed frames

  13. Experimental Results • Generalized Alternative Projection (GAP) algorithm is used for the reconstruction • Synthetic Traffic Video Result • Realistic Surveillance Video Result X. Liao, H. Li, and L. Carin, “Generalized alternating projection for weighted l_2,1 minimization with applications to model-based compressive sensing,” submitted to SIAM Journal on Imaging Sciences, 2012.

  14. Result 1: Synthetic Traffic Video

  15. Measurement

  16. Motion Estimation Comparison of the Compressed Measurement and the Reconstructed Video

  17. Result 2: Realistic Surveillance Video

  18. Conclusion & Future Work • Adaptive temporal compressive sensing is proposed for video CS, not limited to CACTI • Effective method using block-matching on the measurements leads to the real-time adaption, since good hardware exists. • Will implement this adaptive CS ratio in our CACTI camera

  19. Q & A

More Related