On the perception of Bandlimited Phase Distortion in Natural scenes

1. On the perception of Bandlimited Phase Distortion in Natural scenes KedarVilankar, LogeshVasu and Damon Chandler Computational Perception and Image Quality Laboratory School of Electrical and Computer Engineering Oklahoma State University

2. Importance of Phase • Oppenheim and Lim (1981) demonstrated the importanceof phase in signals. • Phase spectrum contributesmore to the image’s visual appearance. Magnitude Spectrum Phase Spectrum

3. Cells Compute Local Information • Primates V1 is dominated by complex cells. • V1complex cells are insensitive to phase • V1complex cells encode the magnitude information only. • V1complex cells are localized to receptive fields.

4. Local Magnitude is all we need • Phaseinformation is implicitlyencodedin local magnitude. • Morrone and Burr demonstrated computation of location of lines and edges (Phase congruency) from local magnitude (Complex cell response). • Also, other researchers have shown only local magnitude is required for scene recognition and categorization.

5. Local Magnitude is all we need • Morgan et. al demonstrated local phase is of lesser importance than local magnitude. • HVS uses local magnitude information to determine global (image-wide) phase information. Phase Spectrum Magnitude Spectrum

6. Does HVS use Local Phase ? • Signal processing perspective, local phase is important. • If HVS uses only local magnitude, then we should not see any impact of distortion in local phase. Local Phase Distorted Image Original Image

7. Does HVS use Local Phase ? • Complex cells compute local magnitude by combining the responses of two simple cells. • May also exist a visual mechanism to compute local phase using simple cells.

8. Does HVS use Local Phase ? • Complex cells compute local magnitude by combining the responses of two simple cells. • May also exist a visual mechanism to compute local phase using simple cells. • If HVS has mechanism to compute local phase, • then do we inferglobal phase from • only local magnitude or • both local magnitude and local phase

9. Summary • Global phase ismost important for image appearance. (Oppenheim & Lim) • Local magnitude can implicitlyencode global phase. (Morroneet al. and Shams et al.) • Local phase is of lesser importance for image appearance • However, local phase distortion has substantial impact on image quality.

10. Outline • Experiment (Local Phase Contribution) • Results (Interesting and surprising) • Discussion (Open questions. We need help) • Algorithm (Local magnitude and phase Distortion Rater) • Conclusion (Our belief)

11. Experiment • Measure the relative contribution of local magnitude and local phase towards image appearance. • Stimuli were created by forming hybrid in complex wavelet subbands. • Each subbands local magnitude and local phase was taken from 2- 4 original images.

12. Experiment Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

13. Experiment Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

14. Experiment • Based on permutations to make hybrid images 14 combination were created. • For each combination 12 stimuli were created. • Five subjects were asked to rate how much each original image contribute to the appearance of the stimulus. • Viewing distance : 60cm. • Five choices : 5%, 10%, 25%, 50% and 75%

15. Result : Combination 1 Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

16. Result : Combination 1 Local Magnitude Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

17. Result : Combination 1 59% Local Magnitude 6% Local Phase 27% 8% Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

18. Result : Combination 1 59% Local Magnitude 6% Local Phase 27% 8% Stimulus Low Frequency High Frequency Frequency (cyc/deg) When across frequencies, local magnitude and phase have non cooperative information, then HVS relies mostly on high frequency local magnitude information. Original Images

19. Result : Combination 3 Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

20. Result : Combination 3 Local Magnitude Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

21. Result : Combination 3 39% Local Magnitude 6% 55% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

22. Result : Combination 3 39% Local Magnitude 6% 55% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) When local phase across entire frequency channels cooperate, then local phase information dominates local magnitude information. Original Images

23. Result : Combination 12 Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

24. Result : Combination 12 Local Magnitude Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

25. Result : Combination 12 48% Local Magnitude 52% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

26. Result : Combination 12 48% Local Magnitude 52% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Local phase and local magnitude have equal importance for the appearance of an image Original Images

27. Result : Combination 13 Local Magnitude Local Phase Low Frequency High Frequency Frequency (cyc/deg) Original Images

28. Result : Combination 13 Local Magnitude Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

29. Result : Combination 13 76% Local Magnitude 24% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

30. Result : Combination 13 76% Local Magnitude 24% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) High frequency information is more important than low frequency for image appearance. Original Images

31. Discussion • Do we infer global phase from local phase?

32. Result : Combination 12 48% Local Magnitude 52% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

33. Discussion • Do we infer global phase from local phase? • Is there visual summation of local phase across frequency channels?

34. Result : Combination 3 39% Local Magnitude 6% 55% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

35. Discussion • Do we infer global phase from local phase? • Is there visual summation of local phase across frequency channels? • Why HVS needs local phase information?

36. Discussion • Do we infer global phase from local phase? • Is there visual summation of local phase across frequency channels? • Why HVS needs local phase information? • How local phase is computed in HVS?

37. Discussion • Do we infer global phase from local phase? • Is there visual summation of local phase across frequency channels? • Why HVS needs local phase information? • How local phase is computed in HVS? • What is the neural basis for this computation?

38. Discussion • What is more important low frequency or high frequency?

39. Result : Combination 13 76% Local Magnitude 24% Local Phase Stimulus Low Frequency High Frequency Frequency (cyc/deg) Original Images

40. Discussion • What is more important low frequency or high frequency? Previous research: • Low and high convey independent information about image structure • For categorization task • Between class – Low frequency • Within class – High frequency • Information content in low and high frequency. Is this task dependent?

41. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Algorithm was developed to rate the quality of local phase distorted images using experimental results. • Algorithm computes local phase as well as local magnitude distortions in an image.

42. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Local Magnitude distortion • Decompose original and distorted images in five scale and ten orientation log-Gabor subbands. • For each scale • Compute local energy maps of the original and distorted images. • Compute local MSE map between local energy maps of the original and distorted images. Use block size of 16 × 16 for local MSE. • Collapse local MSE map via the L2 − norm into a single scalar value. • Compute correlation between local energy maps of original and distorted images. • Compute local magnitude distortion Si(where, i is the index for current scale) by multiplying scalar values obtained in step (c) and (d).

43. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Using Equation combine the local magnitude distortion obtained for each scale in step (e) to compute local magnitude distortion rating in the distorted image. = () + () + () + () + () 1, 2, 3, 4, 5 are power coefficients with values of 4, 4, 2, 1.5 and 0.143 respectively

44. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Local phase distortion • Decompose original and distorted images in four levels and four orientation complex wavelet subbands. • For each level of the complex wavelet subband • Extract local phase information of the original and distorted image. • Compute local phase distortion Ei(where, i is the index for current level) by computing MSE between local phase of the original and distorted image obtained in step (a).

45. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Using Equation combine the local Phase distortion obtained for each scale in step (b) to compute local phase distortion rating in the distorted image. = () + () + () + () 1, 2, 3, 4are power coefficients with values of 2.1, 2.4, 2.3, and 2.2 respectively

46. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Final quality rating of distorted image α = 0.6

47. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Database of 48 phase distorted images. • Five subjects rated the distorted images. • Performance of LMPD compared with other image quality assessment algorithms.

48. Algorithm LMPDLocal Magnitude and Phase Distortion Rater • Database of 48 phase distorted images. • Five subjects rated the distorted images. • Performance of LMPD compared with other image quality assessment algorithms.

49. Conclusion • Local magnitude is most important information for image appearance. • Local phase also has a substantial, sometimes dominating contribution. • Local phase distortion of the images also has a substantial effect on image quality.

50. Conclusion • Local magnitude is most important information for image appearance. • Local phase also has a substantial, sometimes dominating contribution. • Local phase distortion of the images also has a substantial effect on image quality. We believe that an explicit mechanism does exist in visual cortex for the computation of local phaseinformation.