Image Quality Assessment

1. H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and Video Engi., Dept. of ECE Univ. of Texas at Austin Image Quality Assessment

2. Outline • Introduction of Image Quality Assessment • Visual Information Fidelity • Experiments and Results • Conclusion

3. Quality Assessment (QA) • For testing, optimizing, bench-marking, and monitoring applications. Quality?

4. Three Broad QA Categories • Full-Reference (FR) QA Methods • Non-Reference (NR) QA Methods • Reduced-Reference (RR) QA Methods FR QA Reference Image Quality Distorted Image

5. PSNR • Simple but not close to human visual quality Contrast enhancement JPEG compressed Blurred VIF = 1.10 VIF = 0.10 VIF = 0.07

6. Prior Arts • Image Quality Assessment based on Error Sensitivity CSF: Contrast Sensitivity Function Channel Decomposition: DCT or Wavelet Transform Error Normalization: Convert the Error into Units of Just Noticeable Difference (JND) Error Pooling:

7. Problems of Error-Sensitivity Approaches • The Quality Definition Problem • The Suprathreshold Problem • The Natural Image Complexity Problem • The Decorrelation Problem • The Cognitive Interaction Problem

8. Visual Information Fidelity HVS E Natural Image Source Channel (Distortion) HVS F C D Human Visual System Reference Image Reference Image Information Human Visual System Test Image Test Image Information

9. Definition of VIF HVS E Natural Image Source Channel (Distortion) HVS F C D

10. Source Model • The natural images are modeled in the wavelet domain using Gaussian scale mixtures (GSMs). The subband coefficients are partitioned into nonoverlapping blocks of M coefficients each

11. Gaussian scale mixture (GSM) • Wavelet coefficient => non-Gaussian The variance is proportional to the squared magnitudes of coefficients at spatial positions. V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.

12. Implementation Issues • Assumption about the source model: V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.

13. Distortion Model

14. Distorted Images Distorted Images Synthesized versions

15. Distorted Images

16. Human Visual System (HVS)Model HVS E Natural Image Source Channel (Distortion) HVS F C D

17. Visual Information Fidelity Criterion (IFC) HVS Mutual Information I(C;E) E C E Natural Image Source Channel (Distortion) HVS F C D

18. Mutual Information • Assuming that G, and are known Mutual Information I(C;E) C E

19. Mutual Information

20. Implementation Issues • Assumption about the source model: V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.

21. Implementation Issues • Assumption about the distortion model: use B x B window centered at coefficient i to estimate and at i • Assumption about the HVS model: Hand-optimize the value of (by linear regression)

22. Definition of VIF HVS E Natural Image Source Channel (Distortion) HVS F C D

23. Experiments • Twenty-nine high-resolution(768x512) 24-bits/pixel RGB color images • Five distortion types: JPEG 2000, JPEG, white noise in RGB components, Gaussian blur, and transmission errors • 20-25 human observers • Perception of quality: “Bad,” “Poor,” “Fair,” “Good,” and “Excellent” • Scale to 1-100 range and obtain the difference mean opinion score (DMOS) for each distorted image • Data base: http://live.ece.utexas.edu/research/quality/

24. Scatter Plots for Four Objective Quality Criteria (x) JPEG2000, (+) JPEG, (o) white noise in RGB space, (box) Gaussian blur, and (diamond) transmission Errors in JPEG2000 stream over fast-fading Rayleigh channel

25. Scatter Plots for the Quality Prediction

26. Validation Scores Two version of VIF: VIF using the finest resolution at all orientations and Using the horizontal and vertical orientations only THE VALIDATION CRITERIA ARE: CORRELATION COEFFICIENT (CC), MEAN ABSOLUTE ERROR (MAE), ROOT MEAN-SQUARED ERROR (RMS), OUTLIER RATIO(OR), AND SPEARMAN RANK-ORDER CORRELATION COEFFICIENT (SROCC)

27. Cross-Distortion Performance

28. Cross-Distortion Performance (dark solid) JPEG2000, (dashed) JPEG, (dotted) white noise, (dash-dot) Gaussian blur, and (light solid) transmission errors in JPEG2000 stream over fast-fading Rayleigh channel

29. Dependence on the HVS Parameter Dependence of VIF performance on the parameter. (solid) VIF, (dashed) PSNR, (dash-dot) Sarnoff JNDMetrix 8.0, and (dotted) MSSIM.

30. Conclusion • A VIF criterion for full-reference image QA is presented. • The VIF was demonstrated to be better than a state-of-the-art HVS-based method, the Sarnoff’s JND-Metrix, as well as a state-of-the-art structural fidelity criterion, the SSIM index • The VIF provides the ability to predict the enhanced image quality by contrast enhancement operation.

31. Reference • H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006. • H. R. Sheikh, A. C. Bovik, and G. de Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117–2128, Dec. 2005. • Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error measurement to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004. • V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.