Artefact-based Methods for Video Quality Prediction

1. Artefact-based methods for video quality prediction – Literature survey and state-of-the-art Towards hybrid video quality models

2. Video quality prediction • Parameter-based vs signal-based models Savvas Argyropoulos, Deutsche Telekom Labs

3. Artifact-based video quality predictionNo-Reference error concealment effectiveness • Estimate video quality caused by packet losses (but not error propagation!) • Error concealment effectiveness based on: • Motion level information • Luminance discontinuity • Video quality score is based on number of ineffectively concealed macroblocks [1] T. Yamada, Y. Miyamoto, M. Serizawa, “No Reference Video Quality Estimation based on Error Concealment Effectiveness,” Packet Video Workshop, 2007. Savvas Argyropoulos, Deutsche Telekom Labs

4. Artifact-based video quality predictionNo-Reference error concealment effectiveness • MVs for missing macroblocks are estimated from previous frames • EC is considered ineffective if: where: mvx, mvy : (estimated) motions vectors of a missing macroblock an: luminance value in the error region along the boundary An: luminance value in the correctly decoded region along the boundary N : number of pixels along the boundary • Finally: • Where x is the number of ineffectively concealed macroblocks Savvas Argyropoulos, Deutsche Telekom Labs

5. Artifact-based video quality predictionThe ringing effect • Ringing: “Ripples or oscillations around high contrast edges” • Detect regions with perceived ringing • The annoyance caused by the ringing effect is determined by: • Luminance masking • Texture masking • Two-step approach: • Detect spatial location of regions with ringing • Estimate the visibility of the ringing effect • [2] H. Liu, N. Klomp, and I. Heynderickx, “A no-reference metric for perceived ringing,” in Proc. of VPQM 2009. Savvas Argyropoulos, Deutsche Telekom Labs

6. Artifact-based video quality predictionThe ringing effect Block diagram of the algorithm for the perceptual effect of ringing artifacts Savvas Argyropoulos, Deutsche Telekom Labs

7. Artifact-based video quality predictionThe ringing effect (b) Perceptual edge map (a) Original image (c) computational ringing region map (d) perceived ringing map Savvas Argyropoulos, Deutsche Telekom Labs

8. Artifact-based video quality predictionBlurriness (1) • Blurriness is caused by the loss/attenuation of high spatial frequency values. • Blur is perceptually apparent along edges and in textured areas. • The algorithm measures the spread of the edges. • Subjective tests with ten subjects • Artificial blurriness noise: • Gaussian noise • JPEG 2000 compression • [3] P. Marziliano et al., “A no-reference perceptual blur metric,” ICIP, 2002. Savvas Argyropoulos, Deutsche Telekom Labs

9. Artifact-based video quality predictionBlurriness (1) Savvas Argyropoulos, Deutsche Telekom Labs

10. Artifact-based video quality predictionBlurriness (1) • Dashed lines: detected edges • Dotted lines: local minima/maxima around edges • Edge width at P1 Savvas Argyropoulos, Deutsche Telekom Labs

11. Artifact-based video quality predictionBlurriness (1) (a) Original image (b) Gaussian blurriness (c) JPEG 2000 compression blurriness Savvas Argyropoulos, Deutsche Telekom Labs

12. Artifact-based video quality predictionBlurriness (2) • Global blur is estimated from a histogram of DCT coefficients. • 8x8 DCT coefficients from all the blocks in the frame. • The distribution rather than the values of the DCT coefficients are considered. • Normalization with the number of non-zero DC coefficients. • [4] X. Marichal et al., “Blur determination in the compressed domain using DCT information,” in ICIP., Oct. 1999. I-frame P-frame B-frame Savvas Argyropoulos, Deutsche Telekom Labs

13. Artifact-based video quality predictionBlurriness (2) • The blur estimation algorithm examines the number of coefficients that are always zero in the image. • A weighted grid is applied to give more importance to the coefficients in the central diagonal since they better represent global blur. • The weighted sum of number of occurrence of each DCT coefficient provides a measure of the total blur. Savvas Argyropoulos, Deutsche Telekom Labs

14. Artifact-based video quality predictionBlurriness - Sharpness (3) • Blurriness is calculated as the inverse of sharpness. • Sharpness is estimated from kurtosis. • Kurtosis: measure of the non-gaussianity of a random variable • [5] J. Caviedes and S. Gurbuz, “No-Reference sharpness metric based on local edge kurtosis,” ICIP, 2002. Savvas Argyropoulos, Deutsche Telekom Labs

15. Artifact-based video quality predictionBlurriness - Sharpness (3) • DCT is applied to 8x8 pixel blocks and the bivariate probability distribution p(x,y) is used to calculate kurtosis. • Edges are detected – Each edge pixel is assigned to the center of a 8x8 block. • The 2-D kurtosis is calculated for each block of the edge profile. • The average kurtosis over all 8x8 blocks is the frame sharpness measure. Savvas Argyropoulos, Deutsche Telekom Labs

16. References • T. Yamada, Y. Miyamoto, M. Serizawa, “No Reference Video Quality Estimation based on Error Concealment Effectiveness,” Packet Video Workshop, 2007. • H. Liu, N. Klomp, and I. Heynderickx, “A no-reference metric for perceived ringing,” in Proc. of VPQM 2009. • P. Marziliano et al., “A no-reference perceptual blur metric,”ICIP, 2002, vol. 3, pp. 57–60. • X. Marichal et al., “Blur determination in the compressed domain using DCT information,” in ICIP., Oct. 1999. • J. Caviedes and S. Gurbuz, “No-Reference sharpness metric based on local edge kurtosis,” ICIP, 2002. Savvas Argyropoulos, Deutsche Telekom Labs