1 / 41

Image Stabilization by Bayesian Dynamics

Image Stabilization by Bayesian Dynamics. Yoram Burak Sloan-Swartz annual meeting, July 2009. What does neural activity represent?. In Bayesian models: probabilities. Accumulated evidence in area LIP Shadlen and Newsome (2001). Direction of motion: single, static variable.

manchu
Download Presentation

Image Stabilization by Bayesian Dynamics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Image Stabilization by Bayesian Dynamics • Yoram Burak • Sloan-Swartz annual meeting, July 2009

  2. What does neural activity represent? • In Bayesian models:probabilities Accumulated evidence in area LIP Shadlen and Newsome (2001) Direction of motion: single, static variable

  3. What does neural activity represent? • In Bayesian models:probabilities Accumulated evidence in area LIP Shadlen and Newsome (2001) Direction of motion: single, static variable What about multi-dimensional, dynamic quantities?

  4. Foveal vision and fixational drift

  5. Foveal vision and fixational drift Fixational drift is large in the fovea: - between micro-saccades - ~20 receptive fields - between spikes (100 Hz) - ~2-4 receptive fields ! Image from: X. Pitkow cone separation: 0.5 arcmin

  6. Foveal vision and fixational drift Fixational drift is large in the fovea: - between micro-saccades - ~20 receptive fields - between spikes (100 Hz) - ~2-4 receptive fields ! Image from: X. Pitkow cone separation: 0.5 arcmin Downstream areas require knowledge of trajectory to interpret spikes

  7. Joint decoding of image and position Bayesian: vs. N x 2probabilities Discrimination task: X. Pitkow et al, Plos Biology (2007) # positions

  8. Joint decoding of image and position Bayesian: vs. N x 2probabilities Discrimination task: X. Pitkow et al, Plos Biology (2007) # positions 30 x 30 binary pixels Unconstrained image N x 2900probabilities

  9. Joint decoding of image and position Bayesian: vs. N x 2probabilities Discrimination task: X. Pitkow et al, Plos Biology (2007) # positions 30 x 30 binary pixels Unconstrained image N x 2900probabilities Can the brain apply a Bayesian approach to this problem?

  10. Can the brain apply a Bayesian approach to this problem? Decoding strategy Performance in parameter space What are the biological implications?

  11. Can the brain apply a Bayesian approach to this problem? Decoding strategy Performance in parameter space What are the biological implications?

  12. Decoding strategy Factorized representation: Discards information about correlations

  13. Decoding strategy Factorized representation: Discards information about correlations Update dynamics: minimize DKL evidence, diffusion Exact if trajectory is known.

  14. Decoding strategy Factorized representation: Discards information about correlations Update dynamics: minimize DKL evidence, diffusion Exact if trajectory is known. Retinal encoding model: evidence - Poisson spiking (rate λ1 for on pixels, λ0 for off) diffusion - Random walk (diffusion coefficient D)

  15. Decoding strategy Factorized representation: Discards information about correlations Neural Implementation - Two populations: where , what For 30 x 30 pixels: N × 2900 → N + 900quantities.

  16. Update rules Update of what neurons: Ganglion cells multiplicative gating What nonlinearity Where

  17. Update rules Update of what neurons: Ganglion cells multiplicative gating What nonlinearity Where Update of where neurons: Ganglion cells multiplicative gating Where + diffusion What

  18. Demo m x m binary pixels image retina 2d diffusion (D) Poisson spikes: 100 Hz (on), 10 Hz (off) Decoder

  19. Demo

  20. Can the brain apply a Bayesian approach to this problem? Decoding strategy Performance in parameter space What are the biological implications?

  21. Performance Performance degrades with larger D (and smaller λ) accuracy Convergence time [s] D D

  22. Performance Faster and more accurate for larger images m = 5, 10, 30, 50, 100 accuracy Convergence time [s] D D

  23. Demo

  24. Performance Faster and more accurate for larger images m = 5, 10, 30, 50, 100 accuracy Convergence time [s] D D

  25. Performance Faster and more accurate for larger images m = 5, 10, 30, 50, 100 accuracy Convergence time [s] D D

  26. Performance Faster and more accurate for larger images m = 5, 10, 30, 50, 100 accuracy Convergence time [s] D D

  27. Performance scales with linear image size m accuracy Convergence time [s] D/m D/m m x m pixels

  28. Performance scales with linear image size m D* accuracy Convergence time [s] D/m D/m m x m pixels Analytical scaling:

  29. Performance Performance improves with image size. Success for images 10 x 10 or larger Prediction for psychophysics: Degradation in high acuity tasks when visual scene contains little background detail.

  30. Temporal response of Ganglion cells Common view: fixational motion important to activate cells, due to biphasic response f(t) 50 ms t Temporal response makes decoding much more difficult. Non-Markovian: Need history

  31. Temporal response of Ganglion cells Approach: Choose decoder that is Bayes optimal if the trajectory is known. Ganglion accuracy Convergence time [s] Where What D D history dependent decoder / naive decoder “filtered trajectory”

  32. Temporal response of Ganglion cells Is fixational motion beneficial? Known trajectory , perfect inhibitory balance Convergence time [s] D Optimal D - order of magnitude smaller than biological value

  33. Can the brain apply a Bayesian approach to this problem? Decoding strategy Performance in parameter space What are the biological implications?

  34. Network architecture Each ganglion cell innervates multiple what & where cells (spread: ~10 arcmin) Reciprocal, multiplicative gating Ganglion What Where

  35. Activity: What neurons Slow dynamics, evidence accumulation Where neurons Fewer. Highly dynamic activity Tonic, sparse in retinal stabilization conditions.

  36. Activity: What neurons Slow dynamics, evidence accumulation Where neurons Fewer. Highly dynamic activity Tonic, sparse in retinal stabilization conditions. Where in the brain? Monocular If so, suggests LGN or V1 LGN? Modulatory inputs to relay cells (gating?) V1? Lateral connectivity in where network, Increase in number of neurons.

  37. Summary Strategy for stabilization of foveal vision Factorized Bayesian approach to multi-dimensional inference

  38. Summary Strategy for stabilization of foveal vision Factorized Bayesian approach to multi-dimensional inference Explicit representation of stabilized image “What” and “where” populations

  39. Summary Strategy for stabilization of foveal vision Factorized Bayesian approach to multi-dimensional inference Explicit representation of stabilized image “What” and “where” populations Good performance at 1 arcmin resolution Problem is easier for large images, for coarser reconstruction

  40. Summary Strategy for stabilization of foveal vision Factorized Bayesian approach to multi-dimensional inference Explicit representation of stabilized image “What” and “where” populations Good performance at 1 arcmin resolution Problem is easier for large images, for coarser reconstruction Network architecture: Many-to-one inputs from retina, multiplicative gating (what/where)

  41. Acknowledgments Uri Rokni Haim Sompolinsky Markus Meister Special thanks - the Swartz foundation

More Related