Real-Time Foveation Techniques for Low Bit Rate Video Coding

1. A study of Real-Time Foveation Techniquesfor Low Bit Rate Video Coding authored by Hamid R. Sheikh, Brian L. Evans, and Alan C. Bovik

2. Intuition • Encoded video should most closely resemble the original to a human observer • The observability of infidelity depends upon the acuity of the eye observing that infidelity • Acuity can be calculated based on foveation • We can predict likely foveation targets (with accuracy better than chance) while adding only an insignificant amount of additional processing during encoding

3. Sensitivity of the eye is modeled approximately

5. (x,y) position of stimulating pixel (xf,yf) position of center of foveated pixels V viewing distance NOTE: “e” in exponent is defined above, “e” in base is 2.7182818…

6. CT0 Minimum contrast threshold (limit of sensitivity) α Spatial frequency decay constant e2 “Half-resolution” eccentricity

7. fc,e (e) is the cutoff frequency for the eye, given in cycles per degree (cpd)

8. [ ] [ Sampling interval of meters ]

10. Significant Change (e.g., affine warp, color change) Negligible Change (e.g., translation, static)

11. Transmits signal when error is non-negligible or frequency cutoff has changed (in which case the newly-uncut freq. areadded, and newly-cut freq. are removed – the latter is a weakness of this approach, as it is unnecessary at best). LPF Lowpass Filter DCT Discrete Cosine Transform (e.g., Discrete Fourier Transform) IDCT Inverse Discrete Cosine Transform MCP Motion Compensated Prediction Q Quantization IQ Inverse Quantization M “Prediction” of visible data E Prediction error M(f1) Data for M with f1 freq. limit M(f2) Data for M with f2 freq. limit E(f2) Error for M with f2 freq. limit

12. Transmits signal when error is non-negligible (noting that MW2 + EW2 = M(f2) + E(f2)) or if the cutoff frequency has increased (identified by binary weights). LPF Lowpass Filter DCT Discrete Cosine Transform (e.g., Discrete Fourier Transform) IDCT Inverse Discrete Cosine Transform MCP Motion Compensated Prediction Q Quantization IQ Inverse Quantization M “Prediction” of visible data E Prediction error M(f1) Data for M with f1 freq. limit M(f2) Data for M with f2 freq. limit E(f2) Error for M with f2 freq. limit

13. Original Spatial DCT

14. "Automatic Foveation for Video Compressionusing a Neurobiological Model of Human Attention" (L. Itti)

15. "A Practical Foveation-Based Rate ShapingMechanism for MPEG Videos" (C. C. Ho, J. L. Wu, W. H. Cheng)

16. Questions?