A model for detecting illusory contours

1. A model for detecting illusory contours (Peterhans and von der Heydt, 1989) Caveat: Response decreases gradually with increasing intermeshing of gratings. This does not correspond with subjective percepts.

2. Convergence of cues in contour coding (Studies with Tilt AfterEffect) Adapting stimulus Test stimulus TAE is orientation specific and exhibits inter-ocular transfer. Therefore, TAE is likely to be of cortical origin. Does the TAE transfer from one kind of contour (say ‘real’) to another (say ‘illusory’, or ‘motion defined’)? Yes. (Berkley, DeBruyn & Orban, 1994; Paradiso, Shimojo & Nakayama, 1989)

3. Convergence of cues in contour coding

4. From zero-crossings to a raw primal sketch (collection of tokens) Zero crossings at a fine scale Zero crossings at a coarse scale ‘Blobs’ Regions where Zero crossings Form closed contours ‘Edge segments’ Regions of coincidence of ZCs across scales

5. From a raw primal sketch to a full primal sketch Requires contour linking and extracting globally salient structures

6. What assumptions does the visual system use to link Edge fragments and identify ‘important contours’? Ideas from psychophysical studies • Collinearity (aligned fragments typically belong to the same contour) • Length (longer contours are more salient) • Low curvature • Completion (closed contours are more salient) • Object recognizability (controversial)

7.  i Computing saliency [Ullman and Shashua, 1988] How to assign a saliency measure to each edge fragment? S (p) = w  i i  i  3 = 1 if element present = 0 otherwise  2 p -c w = e i Ci is the total curvature of the contour from p upto the ith element i i  1 • Wi is 1 for a straight line and decreases as the curvature increases • penalizes high curvature • Weights distant elements less than closer ones

8. Final saliency at p  3  2 p i  1 S(p) = max S (p)   Very high computational requirements, but can be implemented Highly efficiently in a locally connected network. [Ullman, 1995]

9. Computing saliency – some results

10. Psychophysical and physiological correlates Of saliency computations • Fields et al • Polat and Sagi Facilitation of detection by aligned neighboring elements Gilbert and Wiesel found long range connections between similarly tuned hypercolumns that might subserve the perceptual facilitation.

11. From a raw primal sketch to a full primal sketch Requires segregating regions with different textures

12. Moving beyond the first stages of image-processing… Information about object shape and surface properties

13. Open questions about early processes Do texture segregation processes work on the 2D image or at the level of Surface representation?

14. Processing Framework Proposed by Marr Recognition 3D structure; motion characteristics; surface properties Shape From stereo Motion flow Shape From motion Color estimation Shape From contour Shape From shading Shape From texture Edge extraction Emphasis on ‘Bottom-up’ processing Image

15. Stereopsis The process of perceiving the relative distance to objects based on their Lateral displacement in the two retinal images (aka binocular disparity). Challenges: 1. Trigonometric calculations 2. Correspondence problem Other potential cues to depth (convergence and accommodation) do not seem to be too Important for humans.

16. The Stereo Correspondence Problem “During binocular regard of an objective image, each uniocular Mechanism develops independently a sensual image of considerable Completeness. The singleness of binocular perception results from The union of these elaborated uniocular sensations. The singleness id Therefore the product of a synthesis that works with already elaborated Sensations contemporaneously proceeding.” - Sherrington, 1906

17. The Stereo Correspondence Problem Hand Hand

18. However, different looking images could also be fused… “Although a perfect stranger to you, and living on the reverse side of the globe, I have taken the liberty of writing to you on a small discovery I have made in Binocular vision in the stereoscope. I find by taking two ordinary photos of two Different persons’ faces, the portraits being about the same sizes, and looking About the same direction, and placing them in a stereoscope, the faces blend into One in a most remarkable manner, producing in the case of some ladies’ Portraits, in every instance, a decided improvement in beauty.” - From a letter to Charles Darwin by A. L. Austin of New Zealand

19. Is monocular shape analysis a necessary pre-requisite for stereo correspondence?

20. Open questions: 1.Does RDS stereopsis conclusively prove that the shape-first theory is incorrect? 2. Are RDSs representative of real-world scenes? 3. Are RDSs completely devoid of monocular shape cues?