Computational models for imaging analyses

1. Computational models for imaging analyses Zurich SPM Course February 14, 2014 ChristophMathys

2. What the brain is about • What do our imaging methods measure? • Brain activity. • But when does the brain become active? • When predictions have to be adjusted. • So where do the brain’s predictions come from? • From a model. Model-based imaging, Zurich SPM Course, Christoph Mathys

3. What does this mean for neuroimaging? If brain activity reflects model updating, we need to understand what model is updated in what way to make sense of brain activity. Model-based imaging, Zurich SPM Course, Christoph Mathys

4. The Bayesian brain and predictive coding • Model-based prediction updating is described by Bayes’ theorem. • the Bayesian brain • This is implemented by predictive coding. Hermann von Helmholtz Model-based imaging, Zurich SPM Course, Christoph Mathys

5. Advantages of model-based imaging • Model-based imaging permits us • to infer the computational (predictive) mechanisms underlying neuronal activity. • to localize such mechanisms. • to compare different models. Model-based imaging, Zurich SPM Course, Christoph Mathys

6. How to build a model Fundamental ingredients: Prediction Sensory input Hidden states Inference based on prediction errors Model-based imaging, Zurich SPM Course, Christoph Mathys

7. Example of a simple learning model • Rescorla-Wagner learning: • Learning rate • Previous value (prediction) • Prediction error () • New input • Inferred value of Model-based imaging, Zurich SPM Course, Christoph Mathys

8. From perception to action Agent World Sensory input Generative process Inversion of perceptual generative model Inferred hidden states True hidden states Action Decision model Model-based imaging, Zurich SPM Course, Christoph Mathys

9. From perception to action Agent World • In behavioral tasks, we observe actions (). • How do we use them to infer beliefs ()? • We introduce a decision model. Sensory input True hidden states Inferred hidden states Action Model-based imaging, Zurich SPM Course, Christoph Mathys

10. Example of a simple decision model • Say 3 options A, B, and C have values , , and . • Then we can translate these values into action probabilities via a «softmax» function: • The parameter determines the sensitivity to value differences Model-based imaging, Zurich SPM Course, Christoph Mathys

11. All the necessary ingredients • Perceptual model (updates based on prediction errors) • Value function (inferred state -> action value) • Decision model (value -> action probability) Model-based imaging, Zurich SPM Course, Christoph Mathys

12. Reinforcement learning example (O’Doherty et al., 2003) O’Doherty et al. (2003), Gläscher et al. (2010) Model-based imaging, Zurich SPM Course, Christoph Mathys

13. Reinforcement learning example Significant effects of prediction error with fixed learning rate O’Doherty et al. (2003) Model-based imaging, Zurich SPM Course, Christoph Mathys

14. Bayesian models for the Bayesian brain Agent World Sensory input True hidden states Inferred hidden states • Includes uncertainty about hidden states. • I.e., beliefs have precisions. • But how can we make them computationally tractable? Action Model-based imaging, Zurich SPM Course, Christoph Mathys

15. The hierarchical Gaussian filter (HGF): a computationally tractable model for individual learning under uncertainty Model-based imaging, Zurich SPM Course, Christoph Mathys

16. HGF: variational inversion and update equations • Inversion proceeds by introducing a mean field approximation and fitting quadratic approximations to the resulting variational energies (Mathys et al., 2011). • This leads to simple one-step update equations for the sufficient statistics (mean and precision) of the approximate Gaussian posteriors of the states . • The updates of the means have the same structure as value updates in Rescorla-Wagner learning: • Furthermore, the updates are precision-weighted prediction errors. • Prediction error • Precisions determine learning rate Model-based imaging, Zurich SPM Course, Christoph Mathys

17. HGF: adaptive learning rate • Simulation: Model-based imaging, Zurich SPM Course, Christoph Mathys

18. Individual model-based regressors • Uncertainty-weighted prediction error Model-based imaging, Zurich SPM Course, Christoph Mathys

19. Example: Iglesias et al. (2013) Model-based imaging, Zurich SPM Course, Christoph Mathys

20. Example: Iglesias et al. (2013) • Model comparison: Model-based imaging, Zurich SPM Course, Christoph Mathys

21. Example: Iglesias et al. (2013) Model-based imaging, Zurich SPM Course, Christoph Mathys

22. Example: Iglesias et al. (2013) Model-based imaging, Zurich SPM Course, Christoph Mathys

23. Example: Iglesias et al. (2013) Model-based imaging, Zurich SPM Course, Christoph Mathys

24. How to estimate and compare models:the HGF Toolbox • Available at http://www.tranlsationalneuromodeling.org/tapas • Start with README and tutorial there • Modular, extensible • Matlab-based Model-based imaging, Zurich SPM Course, Christoph Mathys

25. How it’s done in SPM Model-based imaging, Zurich SPM Course, Christoph Mathys

26. How it’s done in SPM Model-based imaging, Zurich SPM Course, Christoph Mathys

27. How it’s done in SPM Model-based imaging, Zurich SPM Course, Christoph Mathys

28. How it’s done in SPM Model-based imaging, Zurich SPM Course, Christoph Mathys

29. How it’s done in SPM Model-based imaging, Zurich SPM Course, Christoph Mathys

30. Take home Agent World • The brain is an organ whose job is prediction. • To make its predictions, it needs a model. • Model-based imaging infers the model at work in the brain. • It enables inference on mechanisms,localization of mechanisms, and model comparison. Sensory input True hidden states Inferred hidden states Action Model-based imaging, Zurich SPM Course, Christoph Mathys

31. Thank you Model-based imaging, Zurich SPM Course, Christoph Mathys