Variational Filtering and DEM Symposium Workshop on Computational Neuroscience

1. Variational filtering and DEM EPSRC Symposium Workshop on Computational Neuroscience Monday 8 – Thursday 11, December 2008 Abstract This presentation reviews variational treatments of dynamic models that furnish time-dependent conditional densities on the path or trajectory of a system's states and the time-independent densities of its parameters. These obtain by maximizing a variational action with respect to conditional densities. The action or path-integral of free-energy represents a lower-bound on the model’s log-evidence or marginal likelihood required for model selection and averaging. This approach rests on formulating the optimization in generalized coordinates of motion. The resulting scheme can be used for online Bayesian inversion of nonlinear hierarchical dynamic causal models and is shown to outperform existing approaches, such as Kalman and particle filtering. Furthermore, it provides for multiple inference on a models states, parameters and hyperparameters using exactly the same principles. Free-form (Variational filtering) and fixed form (Dynamic Expectation Maximization) variants of the scheme will be demonstrated using simulated (bird-song) and real data (from hemodynamic systems studied in neuroimaging).

2. Overview Hierarchical dynamic models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Examples (Hemodynamics and Bird songs)

3. Generalised coordinates Likelihood (Dynamical) prior

4. Energies and generalised precisions Instantaneous energy General and Gaussian forms Precision matrices in generalised coordinates and time

5. Hierarchical dynamic models

6. A simple energy function of prediction error Hierarchal forms and empirical priors Dynamical priors (empirical) Structural priors (empirical) Priors (full)

7. Overview Hierarchical dynamic models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Examples

8. Variational learning Aim: To optimise the path-integral (Action) of a free-energy bound on model evidence w.r.t. a recognition densityq Free-energy: Expected energy: Entropy: When optimised, the recognition density approximates the true conditional density and Action becomes a bound approximation to the integrated log-evidence; these can then be used for inference on parameters and model space respectively

9. Lemma : The free energy is maximised with respect to when Where and are the prior energies and the instantaneous energy is specified by a generative model Mean-field approximation Variational energy and actions Recognition density We now seek recognition densities that maximise action

10. Lemma: is the stationary solution, in a moving frame of reference, for an ensemble of particles, whose equations of motion and ensemble dynamics are Ensemble learning Variational filtering Proof: Substituting the recognition density gives This describes a stationary density under a moving frame of reference, with velocity as seen using the co-ordinate transform

11. 5 0 -5 2 0 -2 A toy example 5 4 3 2 1 0 -1 -2 0 20 40 60 80 100 120

12. Overview Hierarchical dynamic models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Examples (hemodynamics and bird songs)

13. Optimizing free-energy under the Laplace approximation Mean-field approximation: Laplace approximation: The Laplace approximation enables us the specify the sufficient statistics of the recognition density very simply Conditional modes Conditional precisions Under these approximations, all we need to do is optimise the conditional modes

14. Approximating the mode … a gradient ascent in moving coordinates Taking the expectation of the ensemble dynamics, we get: Here, can be regarded as a gradient ascent in a frame of reference that moves along the trajectory encoded in generalised coordinates. The stationary solution, in this moving frame of reference, maximises variational action. by the Fundamental lemma; c.f., Hamilton's principle of stationary action.

15. Dynamic expectation maximization D-Step inference local linearisation (Ozaki 1992) E-Step learning M-Step uncertainty A dynamic recognition system that minimises prediction error

16. Overview Hierarchical dynamic models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Examples (hemodynamics and bird songs)

17. A linear convolution model Prediction error Generation Inversion

18. hidden states 1.5 1 0.5 0 -0.5 cause -1 5 10 15 20 25 30 cause 1.2 1 0.8 0.6 time 0.4 0.2 0 -0.2 -0.4 5 10 15 20 25 30 time {bins} Variational filtering on states and causes

19. Linear deconvolution with variational filtering (SDE) – free form Linear deconvolution with Dynamic expectation maximisation (ODE) – fixed form

20. 6 5 Accuracy and embedding (n) 4 sum squared error (causal states) 3 2 2 2 1.5 1.5 1 1 1 0 0.5 0.5 1 3 5 7 9 11 13 0 0 -0.5 -0.5 -1 -1 -1.5 -1.5 0 10 20 30 40 0 5 10 15 20 25 30 35 time time The order of generalised motion Precision in generalised coordinates

21. 1 DEM(0) DEM(4) 0.5 EKF 0 true DEM(0) -0.5 EKF -1 -1.5 0 5 10 15 20 25 30 35 time 0.9 hidden states 0.8 1 0.7 0.5 0.6 0 0.5 0.4 -0.5 0.3 -1 0 10 20 30 40 0.2 time 0.1 EKF DEM(0) DEM(4) hidden states DEM and extended Kalman filtering With convergence when sum of squared error (hidden states)

22. level A nonlinear convolution model This system has a slow sinusoidal input or cause that excites increases in a single hidden state. The response is a quadratic function of the hidden states (c.f., Arulampalam et al 2002).

23. Comparative performance Sum of squared error DEM and particle filtering

24. Inference on states Triple estimation (DEM) Learning parameters

25. Overview Hierarchical dynamic models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics and Bird songs

26. An fMRI study of attention Stimuli 250 radially moving dots at 4.7 degrees/s Pre-Scanning 5 x 30s trials with 5 speed changes (reducing to 1%) Task: detect change in radial velocity Scanning (no speed changes) 4 x 100 scan sessions; each comprising 10 scans of 4 different conditions F A F N F A F N S ................. A – dots, motion and attention (detect changes) N – dots and motion S – dots F – fixation V5 (motion sensitive area) Buchel et al 1999

27. A hemodynamic model Visual input Motion Attention convolution kernel state equations output equation Output: a mixture of intra- and extravascular signal

28. Inference on states Hemodynamic deconvolution (V5) Learning parameters

29. … and a closer look at the states

30. Overview Hierarchical dynamic models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics and Bird songs

31. Synthetic song-birds syrinx hierarchy of Lorenz attractors

32. Song recognition with DEM

33. … and broken birds

34. Summary Hierarchical dynamic models Generalised coordinates (dynamical priors) Hierarchal forms (structural priors) Variational filtering and action (free-form) Laplace approximation and DEM (fixed-form) Comparative evaluations Hemodynamics and Bird songs