A Bayesian multi-scale model of perceptual organization

1. A Bayesian multi-scale model of perceptual organization James Elder & Francisco Estrada Centre for Vision Research York University Toronto, Canada

2. Goal

3. Outline • Probabilistic foundations • Limitations • A promising direction

4. Markov Chain Model • Model contours as Markov chains: assume long-range statistics completely determined by local statistics.

5. li1 lj1 r li2 lj2 ij t t i j Gestalt Cues: Contours

6. Gestalt Cues: Natural Image Statistics Elder & Goldberg, 2002

7. Parallel Constructive Search

8. Parallel Constructive Search

9. Parallel Constructive Search

10. Parallel Constructive Search

11. Parallel Constructive Search

12. Parallel Constructive Search

13. Parallel Constructive Search

14. Problem: Natural images are rich in detail – leads to large searchspace

15. Problem: smaller parts and structures can easily dominate

16. A Possible Solution: Prior Knowledge • Theory 1. Perceptual organization is a general, bottom-up process that precedes scene recognition (e.g. Kanizsa 79) • Theory 2. Perceptual organization is influenced by higher-level objectives, prior knowledge and specific context (e.g. Rock 83, Cavanagh 91).

17. Using Prior Knowledge: Example

18. But this does not solve the general problem • Some Observations • Limitations of Model: • Naïve 1st-order Markov model almost certainly incomplete – fails to capture important global structure (e.g., Ren et al., 2007) • Raising the order of the model is unlikely to fix the problem. • Computational complexity: • Approximate search algorithms may miss global optimum • Myopic algorithms • Difficult to capture global structure

19. Idea: Coarse-to-Fine Grouping • BYOP (Build Your Own Prior) : • Compute closed contour hypotheses at low resolution. • Project these into higher resolution as statistical priors

20. Coarse-to-Fine Visual Computation: Not a New Idea • Stereopsis (Marr & Poggio, 1979) • Edge Localization (Bergholm, 1987) • Object Detection (Geman, 2006) • Perceptual Organization (???????)

22. Coarse-to-Fine Grouping: Observations • Reduced complexity at low resolutions increases chances of hitting global optimum. • Model is still Markov (computationally efficient), but now over scale as well as space • Global structure is captured first (at low complexity) and then controls complexity of grouping at finer scales.

23. Experimental Results

24. Mean error across all images 1 0.8 0.6 Error 0.4 0.2 0 MS SS RC EJ Experimental Results

25. Mean relative errors 0.45 0.4 0.35 0.3 0.25 Relative Error 0.2 0.15 0.1 0.05 0 SS vs. MS RC vs. MS EJ vs. MS Experimental Results

26. Human Perception: Limitations Elder & Morgenstern, 2003

27. Human efficiency declines with contour length 7000 Ideal Participant 1 Pooled 0.6 0.6 Participant 1 Participant 2 6000 Participant 2 0.5 0.5 5000 0.4 0.4 4000 Efficiency Efficiency Number of elements at threshold 0.3 0.3 3000 0.2 0.2 2000 0.1 0.1 1000 0 0 0 0 5 10 0 5 10 0 5 10 Path length Path length Path length Elder & Morgenstern, 2003

28. Human Perception: The Global Precedence Effect Navon, 1977; Han et al., 1999

29. Coarse-to-Fine Perceptual Grouping • Computational Directions • A more interesting feed-forward computation • Monte Carlo sampling to accelerate feedback • Formulation as closed-loop (recurrent) process • Extension to complete scenes • Experimental Directions • Psychophysical methods to identify dynamics?