1 / 27

Light Field Compression Using 2-D Warping and Block Matching

Light Field Compression Using 2-D Warping and Block Matching. Shinjini Kundu Anand Kamat Tarcar EE398A Final Project. Outline. Motivation and Goals Overview of Our Method Results and Analysis Summary Future Work References. Motivation.

saad
Download Presentation

Light Field Compression Using 2-D Warping and Block Matching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Light Field Compression Using 2-D Warping and Block Matching ShinjiniKundu AnandKamatTarcar EE398A Final Project EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  2. Outline • Motivation and Goals • Overview of Our Method • Results and Analysis • Summary • Future Work • References EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  3. Motivation • Light field images are used in computer graphics to compute new views of a scene without need for scene geometry model1. • Need to compress large set of images • Exploit inter-view coherence to achieve compression. 1. M. Levoy and P. Hanrahan, “Light field rendering,” in Computer Graphics (Proceedings SIGGRAPH 96), August 1996, pp. 31-42. EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  4. Light Fields • Represents a 3D scene or object from all viewing positions and directions • 2D array of 2D images • Difficult to Acquire • Very Large • Perfect representation requires images of the order of the resolution

  5. Light Field Views EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  6. Credit: Andrew Adams Light Field Data Set http://lightfield.stanford.edu/aperture.swf?lightfield=data/lego_lf/preview.zip&zoom=1 8.4 MB uncompressed data sets EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  7. Related Work • Intra-frame coding • Vector quantization, DCT coding, transform coding yield compression ratios of less than 30:1 • Inter-frame coding (compression in the hundreds, thousands) • Disparity compensation • 3D geometry models • Blockwise Compression ideal: maximally use coherence between two images EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  8. Our Method: 2-D Warping • Each consecutive view is a projection of the previous view due to constant predictable movement of camera • Find this relation between the views by obtaining projection matrix for each pair of views • Predict the view and encode the residual EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  9. Our Encoding Scheme Input View Cost=R1+λD1 -- Lagrangian Cost Function Residual and MV Reconstructed Previous View 2-D DCT for the Residual Cost=R2+λD2 ? 2D Warping Algorithm Previous Frame 2-D Warped Use for Reconstruction EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  10. Notes • DCT used on 8x8 blocks to encode residual • Laplacian distribution assumed for motion vectors • Projection matrix was encoded by normalizing values with respect to 10, and assuming Laplacian distribution of bitrate. The min and max values are encoded separately using binary encoding. • H =   -0.5780.005   -0.720   -0.003   -0.5720.0070.0000.000   -0.582 EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  11. 1. Feature match by correlation 2. Projective matrix computed Lagrangian Mode Decision using two references 3. Clipped edges are interpolated using motion compensation EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  12. Getting a predicted projection:Step 1: Feature matching by Correlation Features detected by Harris corner detection algorithm, and matching points identified by maximum correlation EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  13. Computing the Homography Matrix • A homography is an invertible transformation from the real projective plane to the projective plane that maps straight lines to straight lines EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  14. Results for 2-D Projection Warping EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  15. Results for 2-D Projective Warping EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  16. Results for 2D Projective Warping EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  17. Compression Ratios EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  18. Conclusion • Advantages: decreased coding complexity, and increased rate/PSNR as well as compression • Experimental results demonstrate improved coding efficiency with our 2D warp method when compared with MVC. EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  19. Future Work Possible • Optimize the code to give better PSNR values and check performance by introducing extra modes like copy mode • Explore other methods of using inter-view redundancy in detail like disparity compensation at sub-pel accuracy • Run for larger data sets and optimize complexity of the algorithm EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  20. Summary • Light fields represent a 3D scene using sequence of 2-D images • Large amounts of data • Can use redundancy between images using 2-D warping with motion compensated block matching • Results in a sleek method for compression • Performance wise.. EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  21. Acknowledgement • Prof. Girod for pointing us in the right direction • Mina Makar for his help • Chuo-Ling Chang for DAPBT code • Huizhong Chen and Derek Pang for their help • Prof. Peter Kovesi for open source matlab function library • Prof. Levoy’s group and Andrew Adams for access to light field images EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  22. Questions? EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  23. Other Projects • Use Motion Compensation with Directional Transforms • Result: Gain in PSNR due to directionality is approximately 0.1dB at high Quantization; almost nil increase seen at low quantization • So, We adapted the direction of out project to study a new approach of compression presented next. EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  24. Results with Motion Compensation and DAPBT for Crystal light field EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  25. Results with Motion Compensation and DAPBT for Lego light field EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  26. This is how blocking is done and direction selection happens!IAP(DAT)+IRP(DCT) for QP=44, Crystal Light Field EE398A - Compression of Light Fields using 2-D Warping and Block Matching

  27. For Lego light field IAP(DAT)+IRP(DCT) for QP=44, Crystal Light Field EE398A - Compression of Light Fields using 2-D Warping and Block Matching

More Related