1 / 18

SELF-SIMILARITY MEASURE FOR ASSESSMENT OF IMAGE VISUAL QUALITY

SELF-SIMILARITY MEASURE FOR ASSESSMENT OF IMAGE VISUAL QUALITY Nikolay Ponomarenko (*), Lina Jin(**), Vladimir Lukin (*) and Karen Egiazarian (**) (*) National Aerospace University, Kharkov, Ukraine (**) Tampere University of Technology, Tampere, Finland. Introduction.

louie
Download Presentation

SELF-SIMILARITY MEASURE FOR ASSESSMENT OF IMAGE VISUAL QUALITY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SELF-SIMILARITY MEASURE FOR ASSESSMENT OF IMAGE VISUAL QUALITY Nikolay Ponomarenko (*), Lina Jin(**), Vladimir Lukin (*) and Karen Egiazarian (**) (*) National Aerospace University, Kharkov, Ukraine (**) Tampere University of Technology, Tampere, Finland Karen Egiazarian ACIVS 2011

  2. Introduction Human visual sensitivityvaries as function of several key image properties Image processing (compression, denoising, watermarking, etc Image retrieval, estimation of image quality in digital cameras • Light level • Spatial frequency • Color • Local image contrasts • Self-similarity • Temporal frequency Visual quality metrics Full reference No-reference Mathematic metrics (MSE, PSNR) Heuristic metrics (UQI, SSIM) Metrics corresponding to some HVS properties (MSSIM, PSNR-HVS-M) Goal of the research: Consider a possibility to take into account image self-similarity in image visual quality assessment methods Karen Egiazarian ACIVS 2011

  3. Example of non-eccentricity distortions (distortions are similar to non-distorted regions) Distorted image Reference image MSE=350, PSNR=22.69 dB, PSNR-HVS-M=21.39 dB, MSSIM=0.949 Karen Egiazarian ACIVS 2011

  4. Basic Definitions Denote A a given image block (patch) of size NxM pixels. Let us find the difference between A and another patch B of the same size: U A characteristic of noise for the image fragment occupied by the patch A can be determined as: A B where U is an image area in some neighborhood of the A, n denotes the number of patches D in the area U, k=NxM, and P(n,k) is a correcting factor. Let us define an image block self-similarity parameter for A as: Sim(A) = max(0, σA2-Dissim(A)T)/σA2 where σA2 is the image local variance in A, T denotes a factor that determines a minimal ratio σA2/Dissim(A) for which smaller self-similarity for a given patch is considered zero. Karen Egiazarian ACIVS 2011

  5. Distribution of Diff(A,B) for noisy homogeneous regions • Properties of distribution of Diff(A,B) for noisy homogeneous regions • Suppose that a given image contains only noise and it has a Gaussian distribution with zero mean and variance σ2. Then it is easy to show that Diff(A,B) is a random variable with the distribution χ2(k). • Taking into account central limit theorem, for a large degree of freedom (NxM>30) the distribution χ2(k) can be approximately considered Gaussian and having the mean equal to 2σ2 and the variance 8σ4/k. Derivations • Variance of the parameters Dissim(A) and Sim(A) can be decreased by using blocks with larger size (then parameter k for the distribution χ2 becomes larger). • For blocks of larger size it is less probable to find similar image blocks, this can lead to overestimated parameters Dissim(A) for informative image fragments. • Thus, below we will analyze 8x8 pixel blocks (k=64). Such block size is “convenient” due to simplicity of calculating discrete cosine transform (DCT) in blocks. In general, we recommend to use block size from 8x8 to 32x32 pixels (for images of larger size the block size can be larger). Karen Egiazarian ACIVS 2011

  6. Calculation of correcting factor P Dependence of the parameter min(Diff(A,B)) mean on the n obtained by simulations for i.i.d noise (k=64, σ2=100) Taking into account that the variance of estimates Diff(A,B) is equal to 8σ4/k, P(n,k) can be written as: For k=64, it simplifies to: Karen Egiazarian ACIVS 2011

  7. Calculation of correcting factor T • If the correcting factor T is set to be equal to 0.5, then the mean of σA2-Dissim(A) equals to 0 and about half of noisy blocks will have Sim(A) larger than zero. • Let us analyze what should be set for T in order to provide the number of blocks with non-zero estimate of self-similarity to be small. • For n=1 and k=64, the estimates σA2-Dissim(A) are Gaussian random variables with variance σ4/32+T2σ4/8 and mean equals to σ2-2σ2T. Then for T=1, the probability Ppos that the considered random variable is positive is equal to 0.6%. • Our experiments have demonstrated that with n increasing the distribution of Dissim(A) becomes non-Gaussian and its variance decreases (it approximately equals to σ4/40 for n=500) and then increases. For n=100, this variance becomes approximately equal to σA2 or to σ4/32. • Let us estimate Ppos for different T: • For T=0.5, one has distribution with zero mean and variance σ4(1/32+1/128). Such a random variable will be positive in almost 50% of cases. • for T=0.7 one has distribution with the mean equal to -0.4 σ2 and variance σ4(1/32+1/64). Then, the probability Ppos equals to 3.2%. • For T=1, one has distribution with the mean equal to -σ2 and variance σ4/16. Probability Ppos is very close to zero. • T should be larger than 0.5 and less than 1 Karen Egiazarian ACIVS 2011

  8. Example of map of Sim(A)parameter (T=1, n=100) Map of Sim(A) for reference image Barbara, mean value of Sim(A) is 0.26 The map for noisy image (σ2=100), mean Sim(A) is 0.14 Noise significantly decreases similarity level of an image, especially for low contrast details Karen Egiazarian ACIVS 2011

  9. Accounting for Contrast Sensitivity Function • Similarity between blocks is calculated in discrete cosine transform (DCT) domain by multiplying each element of the sum by the corresponding coefficient of CSF. Then, for 8x8 pixels blocks, expression for Diff can be rewritten as: where DCT(A) and DCT(B) are DCTs for patches A and B, respectively, Tc denotes the matrix of CSF of correcting factors given in the following table Karen Egiazarian ACIVS 2011

  10. Visual Quality Metric Based on the Proposed Self-Similarity Parameter • DMap and DMapd are maps of noise components (parameter Dissim) for reference and distorted images respectively. • The search area U is selected in such a manner that the area of width 4 pixels from all sides of an analyzed block is excluded from search (to provide correctness of parameter calculation for the cases of spatially correlated noise). • The analyzed block size is 8x8 pixels, n is set fixed and equal to 96. Half overlapping of blocks is used (i.e. a neighboring block mutual shift is 4 pixels). • For color images, they are first converted into YCbCr color space and then only intensity component Y is analyzed. The source code in Delphi and the Matlab code can be found at http://ponomarenko.info/msddm.htm Karen Egiazarian ACIVS 2011

  11. Analysis of MSDDM Efficiency for Specialized Databases Spearman rank order correlation coefficients values for different metrics and databases Karen Egiazarian ACIVS 2011

  12. Analysis of MSDDM Efficiency for Specialized Databases Scatter-plot of MSDDM wrt MOS for LIVE database Karen Egiazarian ACIVS 2011

  13. Analysis of MSDDM Efficiency for Specialized Databases Scatter-plot of MSDDM wrt MOS for CSIQ database Karen Egiazarian ACIVS 2011

  14. Analysis of MSDDM Efficiency for Specialized Databases Scatter-plot of MSDDM wrt MOS for TID2008 database Karen Egiazarian ACIVS 2011

  15. Analysis of MSDDM Efficiency for Different Types of Distortions in TID2008 • MSDDM adequately estimates distortions due to high-frequency noise • MSDDM overestimates visual quality for the cases of spatially correlated and masked noise • Quantization noise that does not distort self-similarity too much are considerably overestimated Karen Egiazarian ACIVS 2011

  16. Analysis of MSDDM Efficiency for Different Types of Distortions in TID2008 • Gaussian blur and distortions due to image denoising are estimated adequately • MSDDM overestimates visual quality for images distorted due to lossy compression by JPEG and JPEG2000 if these distortions are large Karen Egiazarian ACIVS 2011

  17. Analysis of MSDDM Efficiency for Different Types of Distortions in TID2008 • MSDDM reacts well on mean shift distortions and on non-eccentricity pattern noise • Assessments for block-wise distortions are overestimated • Quality assessments for contrast change distortions are considerably underestimated and the dependence of MOS on MSDDM is nonlinear Karen Egiazarian ACIVS 2011

  18. Summary and Conclusion • Summary • A new self-similarity parameter for images that can be used in image visual quality assessment is considered; • A new quality metric based on self-similarity is proposed; • High correlation between image self-similarity and mean opinion score (human perception) is clearly demonstrated. • Conclusions The paper presents a new self-similarity parameter for images that can be used in image visual quality assessment. We also expect that it can be useful for other image processing applications as blind estimation of noise variance. The parameter properties are analyzed and it is demonstrated that visual quality metrics based on the self-similarity parameter has a good correspondence with a human perception. The metric shows very high level of correlation with mean opinion scores, second only the MSSIM metric. In the same time our metric considers only self-similarity, while the other compared metrics take into account many different factors. This shows that is worth taking into account a self-similarity in a design of adequate models of human visual system. Karen Egiazarian ACIVS 2011

More Related