Bayesian modeling in the context of robust cue integration

1. Bayesian modeling in the context of robust cue integration David C. Knill Center for Visual Science University of Rochester

2. The Bayesian approach as a framework for psychophysics David C. Knill Center for Visual Science University of Rochester

3. Some properties of a useful psychophysical framework • Support building predictive models of perceptual performance. • Support bridging statements between models and descriptions of behavior. • Explain “why” perception / sensorimotor control works the way it does. • Help guide psychophysical research • Suggests new and interesting theoretical questions. • Supports scaling down perceptual / sensorimotor problems to bring them into the lab. • Scales up naturally

4. World model

5. World model Noise Sensory processing Generative model Sensory Features Information

6. World model Noise Sensory processing Generative model Sensory Features p(S | I) Bayesian Computations

7. World model Noise Sensory processing Generative model Estimate Sensory Features p(S | I) Bayesian Computations * Task model

8. World model Noise Sensory processing Generative model Estimate Sensory Features p(S | I) Bayesian Computations * Ideal Observer Task model

9. World model Noise Sensory processing Generative model Estimate Sensory Features p(S | I) Bayesian Computations * Ideal Observer Task model

10. Estimate Sensory Features Human Observer

11. Estimate Sensory Features p(S | I) Bayesian Computations * Task model Rational Observer

12. World model Generative model Estimate Sensory Features p(S | I) Bayesian Computations * Task model Rational Observer

13. Ideal observer models Rational observer models The domain of Bayesian models Description of sensorimotor / perceptual behavior

14. Cue integration:Estimating slant from monocular and binocular cues

16. Linear process model Texture data (It) Slant from texture St wt Sst Action / Decision + Stereo data (Is) Ss ws Slant from stereo

17. Normative (ideal observer) model Stereo likelihood

18. Normative (ideal observer) model Texture likelihood

19. Normative (ideal observer) model Joint likelihood

20. Humans weight sensory cues “optimally” • Discrimination thresholds in single cue conditions predict weights measured in multi-cue experiments. • Ernst and Banks, 2002; Knill and Saunders, 2003; Alais and Burr (2004); etc., etc., etc.

21. Texture information Binocular information Least Reliable Equally reliable Most Reliable

22. What are cue weights?

23. What are cue weights? • Summary descriptions of perceptual performance.

24. What are cue weights? • Summary descriptions of perceptual performance. • Summary descriptions of the information available for a task.

25. What are cue weights? • Summary descriptions of perceptual performance. • Summary descriptions of the information available for a task. • Support logical links between behavior and rational / normative models of performance.

26. Robust non-linear cue integration

27. Robust non-linear cue integration • Classical question • How does the brain interpret multiple sensory cues when they have large “conflicts”

28. Robust non-linear cue integration • Classical question • How does the brain interpret multiple sensory cues when they have large “conflicts” • For visual depth cues • Re-conceptualize the problem • what normally gives rise to what we call large cue “conflicts?”

29. Answer • Most depth cues are informative because of statistical regularities in the world

30. Answer • Most depth cues are informative because of statistical regularities in the world • Examples • Texture - isotropy, homogeneity • Figure shape - isotropy, symmetry • Motion - rigidity

31. Answer • Most depth cues are informative because of statistical regularities in the world • Examples • Texture - isotropy, homogeneity • Figure shape - isotropy, symmetry • Motion - rigidity • Constraints don’t always apply • True prior model = mixture of constraints

32. Answer • Most depth cues are informative because of statistical regularities in the world • Examples • Texture - isotropy, homogeneity • Figure shape - isotropy, symmetry • Motion - rigidity • Constraints don’t always apply • True prior model = mixture of constraints • Large “conflicts” arise when strong constraint does not hold.

33. Compression cue = slant suggested by circle interpretation

34. Likelihood function - p(circle) + p(ellipse)

35. Figure shape cue

36. Disparity cue

37. Combined cues

46. Predictions of mixture model All circles Circles + narrow range of ellipses Circles + broad range of ellipses

48. Stereoscopic slant = 35o

49. Model fits Stereoscopic slant = 35o

50. Model fits Stereoscopic slant = 55o