Abstract

1. The 27th Conference on Uncertainty in Artificial Intelligence Active Inference and Uncertainty Karl Friston uai2011 Abstract In this presentation, I will rehearse the free-energy formulation of action and perception, with a special focus on the representation of uncertainty: The free-energy principle is based upon the notion that both action and perception are trying to minimise the surprise (prediction error) associated with sensory input. In this scheme, perception is the process of optimising sensory predictions by adjusting internal brain states and connections; while action is regarded as an adaptive sampling of sensory input to ensure it conforms to perceptual predictions (this is known as active inference). Both action and perception rest on an optimum representation of uncertainty, which corresponds to the precision of prediction error. Neurobiologically, this may be encoded by the postsynaptic gain of prediction error units. I hope to illustrate the plausibility of this framework using simple simulations of cued, sequential, movements. Crucially, the predictions driving movements are based upon a hierarchical generative model that infers the context in which movements are made. This means that we can temporarily confuse agents by changing the context (order) in which cues are presented. These simulations provide a (Bayes-optimal) simulation of contextual uncertainty and set-switching that can be characterised in terms of behaviour and electrophysiological responses. Interestingly, one can lesion the encoding of precision (postsynaptic gain) to produce pathological behaviours that are reminiscent of those seen in Parkinson's disease. I will use this as a toy example of how information theoretic approaches to uncertainty may help understand action selection and set-switching UNCERTAINTY, ansen seale (2005)

2. “Objects are always imagined as being present in the field of vision as would have to be there in order to produce the same impression on the nervous mechanism” - Hermann Ludwig Ferdinand von Helmholtz Richard Gregory Geoffrey Hinton From the Helmholtz machine to the Bayesian brain and self-organization Thomas Bayes Richard Feynman Hermann Haken

3. Overview Ensemble dynamics Entropy and equilibria Free-energy and surprise The free-energy principle Perception and generative models Hierarchies and predictive coding Perception Birdsong and categorization Simulated lesions and perceptual uncertainty Action Cued reaching and affordance Simulated lesions and behavioral uncertainty

4. What is the difference between a snowflake and a bird? Phase-boundary temperature …a bird can act (to avoid surprises)

5. What is the difference between snowfall and a flock of birds? Ensemble dynamics, clumping and swarming …birds (biological agents) stay in the same place They resist the second law of thermodynamics, which says that their entropyshould increase

6. But what is the entropy? …entropy is just average surprise High surprise (I am never here) Low surprise (we are usually here) This means biological agents must self-organize to minimise surprise. In other words, to ensure they occupy a limited number of states (cf homeostasis).

7. But there is a small problem… agents cannot measure their surprise ? But they can measure their free-energy, which is always bigger than surprise This means agents should minimize their free-energy. So what is free-energy?

8. What is free-energy? …free-energy is basically prediction error sensations – predictions = prediction error where small errors mean low surprise

9. More formally, Sensations External states in the world Internal states of the agent (m) Action Free-energy is a function of sensations and a proposal density over hidden causes and can be evaluated, given a generative model (Gibbs Energy) or likelihood and prior: So what models might the brain use?

10. Hierarchal models in the brain lateral Backward (modulatory) Forward (driving)

11. So how do prediction errors change predictions? sensory input Forward connections convey feedback Prediction errors Adjust hypotheses Predictions prediction Backward connections return predictions …by hierarchical message passing in the brain

12. David Mumford More formally, Synaptic activity and message-passing Forward prediction error Backward predictions cf., Predictive coding or Kalman-Bucy filtering

13. Summary Biological agents resist the second law of thermodynamics They must minimize their average surprise (entropy) They minimize surprise by suppressing prediction error (free-energy) Prediction error can be reduced by changing predictions (perception) Prediction error can be reduced by changing sensations (action) Perception entails recurrent message passing in the brain to optimise predictions Predictions depend upon the precision of prediction errors

14. Overview Ensemble dynamics Entropy and equilibria Free-energy and surprise The free-energy principle Perception and generative models Hierarchies and predictive coding Perception Birdsong and categorization Simulated lesions and perceptual uncertainty Action Cued reaching and affordance Simulated lesions and behavioral uncertainty

15. Making bird songs with Lorenz attractors Vocal centre Syrinx Sonogram Frequency causal states 0.5 1 1.5 time (sec) hidden states

16. Predictive coding and message passing prediction and error 20 15 10 5 0 -5 10 20 30 40 50 60 Backward predictions causal states 20 15 stimulus 10 5000 5 4500 Forward prediction error 0 4000 -5 3500 -10 10 20 30 40 50 60 3000 hidden states 20 2500 2000 15 0.2 0.4 0.6 0.8 time (seconds) 10 5 0 -5 10 20 30 40 50 60

17. Perceptual categorization Song a Song b Song c Frequency (Hz) time (seconds)

18. Hierarchical (itinerant) birdsong: sequences of sequences Neuronal hierarchy Syrinx sonogram Frequency (KHz) 0.5 1 1.5 Time (sec)

19. Simulated lesions and false inference percept LFP 60 40 20 Frequency (Hz) LFP (micro-volts) 0 -20 -40 no top-down messages LFP 60 40 20 no structural priors Frequency (Hz) LFP (micro-volts) 0 -20 -40 -60 no lateral messages LFP 60 40 20 no dynamical priors Frequency (Hz) LFP (micro-volts) 0 -20 -40 -60 0.5 1 1.5 0 500 1000 1500 2000 time (seconds) peristimulus time (ms)

20. Overview Ensemble dynamics Entropy and equilibria Free-energy and surprise The free-energy principle Perception and generative models Hierarchies and predictive coding Perception Birdsong and categorization Simulated lesions and perceptual uncertainty Action Cued reaching and affordance Simulated lesions and behavioral uncertainty

21. Premotor cortex affordance Motor cortex joint positions Parietal cortex finger location Anatol Feldman Prefrontal cortex changes in set Active inference Striatum set selection Superior colliculus salience Motoneurones

22. Lotka-Volterra dynamics: winnerless competition Misha Rabinovich

23. Motor cortex Premotor cortex Parietal cortex Prefrontal cortex Striatum Mesocortical DA projections Superior colliculus Nigrostriatal DA projections SN/VTA Mesorhombencephalic pathway Dopamine and precision Motoneurones

24. prediction and error prediction and error prediction and error prediction and error prediction and error prediction and error prediction and error prediction and error 4 4 4 4 4 4 4 4 3 3 3 3 3 3 3 3 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1 -1 -1 -2 -2 -2 -2 -2 -2 -2 -2 20 20 20 40 40 40 60 60 60 80 80 80 100 100 100 120 120 120 20 20 40 40 60 60 80 80 100 100 120 120 20 20 20 40 40 40 60 60 60 80 80 80 100 100 100 120 120 120 time time time time time time time time hidden causes hidden causes hidden causes hidden causes hidden causes hidden causes hidden causes hidden causes 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1 1 1 1 1 1 1 1 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0 0 0 0 0 0 0 0 -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 -0.5 20 20 20 40 40 40 60 60 60 80 80 80 100 100 100 120 120 120 20 20 40 40 60 60 80 80 100 100 120 120 20 20 20 40 40 40 60 60 60 80 80 80 100 100 100 120 120 120 time time time time time time time time -2 -2 -2 -2 -2 -2 -2 -2 -1 -1 -1 -1 -1 -1 -1 -1 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 -2 -2 -2 -1 -1 -1 0 0 0 1 1 1 2 2 2 -2 -2 -1 -1 0 0 1 1 2 2 -2 -2 -2 -1 -1 -1 0 0 0 1 1 1 2 2 2

25. -2 -2 -2 -2 -1 -1 -1 -1 0 0 0 0 1 1 1 1 2 2 -2 -1 1 0 2 2 2 2 2 2 -2 -2 -2 -1 -1 -1 1 1 1 0 0 0 Uncertainty and perseveration

26. reaction times reaction times reaction times 550 440 420 420 400 500 400 380 450 380 milliseconds milliseconds milliseconds 360 360 400 340 340 350 320 320 300 300 300 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 cue onset (sec) cue onset (sec) cue onset (sec) Motor cortex Motor cortex Motor cortex Premotor cortex Premotor cortex Premotor cortex SN/VTA SN/VTA SN/VTA Superior colliculus Superior colliculus Superior colliculus salience proprioception affordance Low DA Low DA High DA High DA High DA Low DA

27. Motor cortex Motor cortex Premotor cortex Premotor cortex SN/VTA SN/VTA Superior colliculus Superior colliculus Uncertainly, delusions and confusion perseveration confusion X X X

28. Thank you And thanks to collaborators: Rick Adams Harriet Brown Jean Daunizeau Lee Harrison Stefan Kiebel James Kilner Jérémie Mattout Klaas Stephan And colleagues: Peter Dayan Jörn Diedrichsen Paul Verschure Florentin Wörgötter And many others