Modeling event perception in infancy

1. Modeling event perception in infancy Arthur Franz Frankfurt Institute for Advanced Studies http://fias.uni-frankfurt.de FIAS, 2008-4-29

2. Motivation • Where does knowledge come from? • Study “simple” systems: infants • Perception and conception of spatial events seem to be crucial • E.g.: occlusion, launching, object unity and, permanence, continuity, object solidity, support,… ”naïve physics” • Hypothesis: most of them can be learned from purely statistical properties of visual input.

3. The habituation paradigm How do people investigate what infants know? • Habituation paradigm Example: Perception of object unity rod movement baseline Mean looking time (sec) habituation displays test habituation test displays habituation test

4. How can we model this? Input example for the object unity experiment 7 x 7 pixel retina BACKGROUND FOREGROUND We build a network that learns to represent occluded objects.

5. Input coding Assumption: Neurons tuned to velocity AND disparity In MT?

6. Neural network Got it?

7. Calculation details Learning: backpropagation of error with real inputs and outputs. Objective function:

8. Pre-training Pre-training corresponds to the infant’s visual experience with the world • Varying the pre-training time allows for modeling infant’s of various ages! Pretraining with random moving or stationary rectangles

9. What the network “imagines” real inputs full inputs imagined outputs real outputs

10. Relation to infant experiments In infant experiments the looking time is measured. Looking time ~ attention, novelty ~ habituation (“tiring”) of certain neurons in the infant’s brain. New stimulus => other neurons get active => dishabituation Dishabituation in the model = difference between the hidden layer activity during habituation and the activity during a test stimulus.

11. Modeling object unity (1) full inputs rod movement baseline habituation displays complete rod broken rod test displays Rod movement => preference for broken rod Baseline => no preference

12. Modeling object unity (2) rod occlusion complete rod broken rod control control habituation displays test displays Rod occlusion => preference for broken rod (age effect!) Complete rod control => pref. for broken rod Broken rod control => pref. for complete rod

13. Modeling object unity (3) full inputs rod occlusion rod pieces habituation displays test displays Rod occlusion => preference for broken rod Rod pieces => preference for broken rod

14. Modeling object unity (4) rod moves rod & block move block moves no movement habituation displays test displays Result: after long pre-training the network shows a preference for the broken rod in each condition! => Age effect, see adult data

15. Modeling object unity (5) rod-polygon baseline habituation displays complete broken test displays rod-polygon => preference for broken rod Baseline => no preference

16. Adult data

17. Modeling perception of occluded trajectories (1) thick occluder 4-month-olds thin occluder habituation continuous test discontinuous test

18. Modeling perception of occluded trajectories (2) long pre-training short pre-training exp. condition baseline exp. condition baseline habituation displays test displays Exp. condition => preference for discontinuous display Exp. condition => preference for continuous display Baseline => no preference

19. Modeling perception of occluded trajectories (3) thin thick preference 2 mo 4 mo 6 mo Pre-training time / 1000 • Natural explanation for data • Model explains how and why preferences change • Object permanence develops in the network!

20. Summary • The neural network provides a model of infant’s perception of occluded objects, object unity and object permanence. • In total 9 fundamental experiments from 2 different laboratories have been explained. • The network is a developmental model and can reveal the mechanisms of change. Especially, the how and why questions can be adressed. • It demonstrates that much of infants’ perception can be learned and explained solely on the basis of statistical regularities of raw visual input. No innate principles or modules need to be postulated.

21. Drawbacks and open questions • Backpropagation of error => Andrea’s network? • Dishabituationmeasurment is done only with first hidden layer. What not the second? Why not the whole network? Habituation with intrinsic plasticity? • Stimuli are “flat” on the screen in the lab => no bottom-up disparity-based separation possible! • Evidence for neurons tuned to both velocity AND disparity? • In some experiments the prediction error is more suitable as a dishabituation measure. How to combine? • Imagined outputs are noisy. The calculation of the full inputs is too “constructed”.

22. Future work • Predictions of the model • Elaborate the relation of this model to the experimenters verbal accounts • Include other event categories into pre-training (blocked motion, launching). Many other experiments can then be explained. continuity, solidity, object permanence,…

23. Thank you!

24. References • Kellman, Spelke (1983). Perception of Partly Occluded Objects in Infancy. Cognitive Psychology, 15, 483-524 • S.P. Johnson, J.G. Bremner, A. Slater, U. Mason, K. Foster, A. Cheshire (2003). Infants' perception of object trajectories. Child Development, 74, 94-108

25. Talk feedback • Hidden+context layer doing everything? • Feedback from im. Layer to hidden/context? • Bayesian / optimization approach • Disparity cells not present before 4 months • Modeling with Kalman filters