1 / 30

Shota Hino, Kishore Sriadibhatla & Jingyang Xue Stanford University EE 398A Final Project

Shota Hino, Kishore Sriadibhatla & Jingyang Xue Stanford University EE 398A Final Project. Direction-Adaptive Transform of Motion-Compensated Residuals in Video Sequences. Overview. Motivation Directional Adaptive Transforms for Residuals Motion compensated residual computation

connor
Download Presentation

Shota Hino, Kishore Sriadibhatla & Jingyang Xue Stanford University EE 398A Final Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Shota Hino, Kishore Sriadibhatla & Jingyang Xue Stanford University EE 398A Final Project Direction-Adaptive Transform of Motion-Compensated Residuals in Video Sequences

  2. Overview • Motivation • Directional Adaptive Transforms for Residuals • Motion compensated residual computation • Residual clustering • Training KLT on each cluster • Results & Performance Comparison • Coding Gain vs. Directions • Rate-Distortion comparison with standard transforms • Conclusion • Questions

  3. Motivation • Direction adaptive transform achieves better coding performance for still images • Same is true for residual images? • Maximum coding gain using KLT?

  4. Motion Compensated Residual Images • Motion compensated residuals for six different quantization levels • Block matching based on MSE and motion vector • Search range of [-16,16]

  5. Block Based Residual Clustering • Clustering using dominant direction in 8x8 block • Histogram of gradients, find dominant angle based on sum of magnitude threshold • 9 groups: -67.5 to 90 deg @ 22.5 interval + No direction • Input source can be residual or predicted signals 45 deg in prediction picture Original Image Residual Image

  6. Trained KLT on Cluster Basis Function for direction -67.5 KLT Covariance Matrix

  7. Mode Control Encoder XOR Decoder MC Hybrid Coder with KLT Clustering & KLT Selection Intraframe KLT coder Intraframe KLT Decoder Motion compensated predictor

  8. Coding Gain Coding Gain = • Coding gain of DA-KLT compared with 3 other standard transforms. • -- DCT • -- KLT trained on whole image • -- DAPBT

  9. Coding Gain Comparison, Train = Test Prediction Signal Based Clustering Coding Gain Train: Stefan.cif Test: Stefan.cif

  10. Coding Gain Comparison, Train = Test Residual Based Clustering Coding Gain Train: Stefan.cif Test: Stefan.cif 10

  11. Coding Gain Comparison, Train = Test Prediction Based Clustering, train on 5 video clips Train: Flower.cif, Highway.cif, Stefan.cif, Tempete.cif, Highway.cif Test: Flower.cif, Highway.cif, Stefan.cif, Tempete.cif, Highway.cif 11

  12. Coding Gain Comparison, Train != Test Prediction Based Clustering, train on 5 video clips Train: Flower.cif, Highway.cif, Stefan.cif, Tempete.cif, Highway.cif Test: Coastguard.cif 12

  13. DA-KLT vs other Transforms • Uniform Quantization - 6 levels • Trained & Tested on 40 frames of Stefan • Direction 6, prediction signal based Prediction Signal Based Clustering

  14. DA-KLT vs DCT Direction 6 - Zoom Prediction Signal Based Clustering

  15. DA-KLT vs DCT Direction 8 Prediction Signal Based Clustering

  16. DA-KLT vs DCT Direction 8 - Zoom Prediction Signal Based Clustering

  17. DA-KLT vs DCT Direction 6 Residual Based Clustering

  18. DA-KLT vs DCT Direction 6 - Zoom Residual Based Clustering

  19. DA-KLT vs DCT Direction 8 Residual Based Clustering

  20. DA-KLT vs DCT Direction 8 - Zoom Residual Based Clustering

  21. Frame Comparison Original Image DA-KLT reconstructed

  22. Frame Comparison - Detail DCT Reconstructed DA-KLT Reconstructed Original q_step = 16

  23. Frame Comparison - Detail DCT Reconstructed DA-KLT Reconstructed Original q_step = 16

  24. Conclusion & Future Directions • Directional Adaptive Transforms can outperform DCT and DA-PBT if train KLT for specific video source • Can achieve 0.1 to 0.4 db at high rate in ideal case • Prediction based clustering more practical at the expense of less accuracy • Currently seeing slightly worse coding gain performance using “universal” KLT on other video sources • Future Directions • Explore different quantization scheme for KLT • Better clustering algorithm based on prediction signal • Searching for better universal KLT or separable transforms 24

  25. Thank you • Questions? 25

  26. Backup Slides 26

  27. Coding Gain Comparison, Train = Test Residual Based Clustering, train on 5 video clips Train: Flower.cif, Highway.cif, Stefan.cif, Tempete.cif, Highway.cif Test: Coastguard.cif 27

  28. Coding Gain Comparison, Train != Test Residual Based Clustering, train on 5 video clips Train: Flower.cif, Highway.cif, Stefan.cif, Tempete.cif, Highway.cif Test: Coastguard.cif 28

  29. DA-KLT vs DCT Direction 1

  30. DA-KLT vs DCT Direction 1 Prediction Signal Based Clustering

More Related