Color Image Fidelity Assessor *

1. Color Image Fidelity Assessor * Wencheng Wu (Xerox Corporation) Zygmunt Pizlo (Purdue University) Jan P. Allebach (Purdue University) *Research supported by HP Company while Wencheng Wu was at Purdue

2. Outline • Introduction • Spatial color descriptor: chromatic difference • Structure of Color Image Fidelity Assessor (CIFA) • Psychophysical experiment and its results • Test examples • Conclusion

3. Introduction(Motivation) • Image fidelity assessment is important in the development of imaging systems and image processing algorithms • Create visually lossless reproduction • Allocate efforts on most visible area • Subjective evaluation is expensive and slow.

4. Introduction(Prior work) • Simple but not working • Root-Mean-Square Error • Consider structure of HVS and perceptual process • Achromatic: Daly’s VDP, Lubin’s VDM, Taylor’s Achromatic IFA (IFA) • Color: Jin’s CVDM (Daly’s VDP + Wandell’s Spatial CIE Lab)

5. Introduction(CVDM vs. CIFA) • Both operate along opponent-color coordinates • Both incorporate results from electrophysiological and psychophysical exp. • They differ in a similar way as VDP vs. IFA • CIFA has closer link between the structure of the model and the psychophysical data used by the model • CIFA normalize the chromatic responses • This discounts luminance effect in chromatic channels • This reduces the dimension of psychometric LUT

6. Color Image Fidelity Assessor (CIFA) Ideal Image maps of predicted visible differences Rendered Viewing parameters Introduction(Overview of CIFA) • Color extension of Taylor’s achromatic IFA • The model predicts perceived image fidelity • Assesses visible differences in the opponent channels • Explains the nature of visible difference (luminance change vs. color shift)

7. Luminance  Red-Green Blue-Yellow Chromatic difference(Definition) • Objective: evaluate the spatial interaction between colors • First transform CIE XYZ to opponent color space (O2,O3)* • Then normalize to obtain opponent chromaticities (o2,o3) • Define chromatic difference (analogous to luminance contrast c1) *X. Zhang and B.A. Wandell, “A SPATIAL EXTENSION OF CIELAB FOR DIGITAL COLOR IMAGE REPRODUCTION”, SID-97

8. (Y,o2,o3) (Y,0.24,0.17) (13.3,o2,0.17) (13.3,0.24,o3) Opponent color representation

9. 0.1 0.05 0.2 0.1 Chromatic difference(illustration) • Chromatic difference is a measure of chromaticity variation • Chromatic difference is a spatial feature derived from opponent chromaticity that has little dependence upon luminance • Chromatic difference is the amplitude of the sinusoidal grating

10. Imagemap of predicted visible luminance differences Achromatic* IFA Ideal Y Image Rendered Y Image Multi-resolution Y images Imagemap of predicted visible red-green differences Ideal O2 Image Red-green IFA Rendered O2 Image Imagemap of predicted visible blue-yellow differences Ideal O3 Image Blue-yellow IFA Rendered O3 Image ChromaticIFAs * Previous work of Taylor et al CIFA (Y,O2,O3): Opponent representation of an image

11. Achromatic IFA Lum. contrast discrimination Chromatic diff. discrimination Psychometric LUT (f,Y,c1) Psychometric LUT (f,o2,c2) Adaptation level Lowpass Pyramid Chromatic Diff. Decomposition Contrast Decomposition + – S Lowpass Pyramid Chromatic Diff. Decomposition Contrast Decomposition Contrast: luminance contrast & chromatic difference Red-green IFA Psychometric Selector Channel Response Predictor Limited Memory Prob. Sum.

12. IFA components • Psychometric LUT • Results from psychophysical experiment • Stored in the form of Lookup-Table: (f,Y,c1), (f,o2,c1), (f,o3,c1) • Time consuming, but it is done off-line • Image processing: • Lowpass pyramid: create 5 multi-resolution images • Lowpass filtering +  2 in horizontal and vertical direction • Normalized by Y images if it is a chromatic IFA • Signal decomposition: create 8 orientation-specific contrast or chromatic-difference images at each resolution • Lowpass pyramid + Signal decomposition: 40 (5 levels  8 orientations) visual channels for each image pixel

13. IFA components(cont’d) • Image processing (continued): • Psychometric selector: for each pixel at each visual channel, find discrimination threshold by choosing appropriate data from LUT • Channel response predictor: for each pixel at each visual channel, convert chromatic difference to discrimination probability • Limited memory probability summation: for each pixel, combine discrimination probability across all 40 visual channel

14. Gabor patch  f, o2, c2 Estimating parameters of LUT(Stimulus: Isoluminant Gabor patch) • Red-green (O2 or o2)stimulus • Keep Y, O3 (o3)constant • Let O2=Yo2+Yc2cos(.)e(.) or equivalently o2’ =o2+c2cos(.)e(.) • (Y,o2,o3) specifies the background color, c2 is the chromatic difference

15. Estimating parameters of LUT(Psychophysical method) • Red-green stimulus: (Y,o2,o3) specifies the background color, c2 is the ref. chromatic difference • Which stimulus has less chromatic difference?

16. Subject WW’s responses probability Estimating parameters of LUT (Data analysis) • Fit subject’s responses to a Normal distribution using probit analysis • Record the standard deviation as the discrimination threshold  • LUT: rg(f,o2,c2) 

17. Estimating parameters of LUT(List of experimental conditions) • indicate spatial frequency of 1, 2, 4, 8, 16 cpd

18. Representative results Red-green discrimination at RG1:(Y,o2,o3)=(5,0.2,-0.3) Blue-yellow discrimination at BY1:(Y,o2,o3)=(5,0.3,0.2) • Results for f = 16, 8, 4, 2, 1 cycle/deg are drawn in red, green, blue, yellow, and black. • Threshold is not affected strongly by the reference chromatic difference • Chromatic channels function like low-pass filters Threshold  Threshold  Reference c2 Reference c3

19. CIFA output for example distortions(Hue change) Luminance R-G B-Y

20. Luminance R-G B-Y CIFA output for example distortions(Blurring)

21. CIFA output for example distortions(Limited gamut) Luminance R-G B-Y

22. Conclusion • CIFA provides good assessment of the perceived visible differences over a range of image contents and distortion types • Chromatic difference describes the color percept of HVS efficiently • Suggestions on future directions • Add DC component in the LUT in chromatic IFAs • Subjective validation • Improve spatial localization • Take dependency between visual channels into account (in prob. Sum. stage)

23. CIFA output for example distortions(Limited color quantization) Luminance R-G B-Y

24. CIFA output for example distortions(Limited gamut) Luminance R-G B-Y

25. CIFA output for example distortions(Increased saturation) Luminance R-G B-Y