580 likes | 1.81k Views
H. R. Sheikh, A. C. Bovik , “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and Video Engi ., Dept. of ECE Univ. of Texas at Austin. Image Quality Assessment. Outline. Introduction of Image Quality Assessment
E N D
H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006 Lab for Image and Video Engi., Dept. of ECE Univ. of Texas at Austin Image Quality Assessment
Outline • Introduction of Image Quality Assessment • Visual Information Fidelity • Experiments and Results • Conclusion
Quality Assessment (QA) • For testing, optimizing, bench-marking, and monitoring applications. Quality?
Three Broad QA Categories • Full-Reference (FR) QA Methods • Non-Reference (NR) QA Methods • Reduced-Reference (RR) QA Methods FR QA Reference Image Quality Distorted Image
PSNR • Simple but not close to human visual quality Contrast enhancement JPEG compressed Blurred VIF = 1.10 VIF = 0.10 VIF = 0.07
Prior Arts • Image Quality Assessment based on Error Sensitivity CSF: Contrast Sensitivity Function Channel Decomposition: DCT or Wavelet Transform Error Normalization: Convert the Error into Units of Just Noticeable Difference (JND) Error Pooling:
Problems of Error-Sensitivity Approaches • The Quality Definition Problem • The Suprathreshold Problem • The Natural Image Complexity Problem • The Decorrelation Problem • The Cognitive Interaction Problem
Visual Information Fidelity HVS E Natural Image Source Channel (Distortion) HVS F C D Human Visual System Reference Image Reference Image Information Human Visual System Test Image Test Image Information
Definition of VIF HVS E Natural Image Source Channel (Distortion) HVS F C D
Source Model • The natural images are modeled in the wavelet domain using Gaussian scale mixtures (GSMs). The subband coefficients are partitioned into nonoverlapping blocks of M coefficients each
Gaussian scale mixture (GSM) • Wavelet coefficient => non-Gaussian The variance is proportional to the squared magnitudes of coefficients at spatial positions. V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.
Implementation Issues • Assumption about the source model: V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.
Distorted Images Distorted Images Synthesized versions
Human Visual System (HVS)Model HVS E Natural Image Source Channel (Distortion) HVS F C D
Visual Information Fidelity Criterion (IFC) HVS Mutual Information I(C;E) E C E Natural Image Source Channel (Distortion) HVS F C D
Mutual Information • Assuming that G, and are known Mutual Information I(C;E) C E
Implementation Issues • Assumption about the source model: V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.
Implementation Issues • Assumption about the distortion model: use B x B window centered at coefficient i to estimate and at i • Assumption about the HVS model: Hand-optimize the value of (by linear regression)
Definition of VIF HVS E Natural Image Source Channel (Distortion) HVS F C D
Experiments • Twenty-nine high-resolution(768x512) 24-bits/pixel RGB color images • Five distortion types: JPEG 2000, JPEG, white noise in RGB components, Gaussian blur, and transmission errors • 20-25 human observers • Perception of quality: “Bad,” “Poor,” “Fair,” “Good,” and “Excellent” • Scale to 1-100 range and obtain the difference mean opinion score (DMOS) for each distorted image • Data base: http://live.ece.utexas.edu/research/quality/
Scatter Plots for Four Objective Quality Criteria (x) JPEG2000, (+) JPEG, (o) white noise in RGB space, (box) Gaussian blur, and (diamond) transmission Errors in JPEG2000 stream over fast-fading Rayleigh channel
Validation Scores Two version of VIF: VIF using the finest resolution at all orientations and Using the horizontal and vertical orientations only THE VALIDATION CRITERIA ARE: CORRELATION COEFFICIENT (CC), MEAN ABSOLUTE ERROR (MAE), ROOT MEAN-SQUARED ERROR (RMS), OUTLIER RATIO(OR), AND SPEARMAN RANK-ORDER CORRELATION COEFFICIENT (SROCC)
Cross-Distortion Performance (dark solid) JPEG2000, (dashed) JPEG, (dotted) white noise, (dash-dot) Gaussian blur, and (light solid) transmission errors in JPEG2000 stream over fast-fading Rayleigh channel
Dependence on the HVS Parameter Dependence of VIF performance on the parameter. (solid) VIF, (dashed) PSNR, (dash-dot) Sarnoff JNDMetrix 8.0, and (dotted) MSSIM.
Conclusion • A VIF criterion for full-reference image QA is presented. • The VIF was demonstrated to be better than a state-of-the-art HVS-based method, the Sarnoff’s JND-Metrix, as well as a state-of-the-art structural fidelity criterion, the SSIM index • The VIF provides the ability to predict the enhanced image quality by contrast enhancement operation.
Reference • H. R. Sheikh, A. C. Bovik, “Image Information and Visual Quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006. • H. R. Sheikh, A. C. Bovik, and G. de Veciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117–2128, Dec. 2005. • Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error measurement to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004. • V. Strela, J. Portilla, and E. Simoncelli, “Image denoising using a local Gaussian scale mixture model in the wavelet domain,” Proc. SPIE, vol. 4119, pp. 363–371, 2000.