1 / 69

BMI Principles

BMI Principles . Jose C. Principe University of Florida Adapted from Hayrettin Gürkök , U. of Twente , NL. Literature. Difficulties in Invasive BMIs. BCIs offer an easy entry to research Non invasiveness straight forward data collection Closer to cognition

zenda
Download Presentation

BMI Principles

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BMI Principles Jose C. Principe University of Florida Adapted from Hayrettin Gürkök, U. of Twente, NL

  2. Literature

  3. Difficulties in Invasive BMIs • BCIs offer an easy entry to research • Non invasiveness straight forward data collection • Closer to cognition • Conventional signal processing • BMIs research infrastructure is much harder • Work with animals (ethics) • Difficult instrumentation • Unclear signal processing

  4. Choice of Scale for Neuroprosthetics

  5. Electrode Arrays Utah array Brain Gate Michigan probes J. C. Sanchez, N. Alba, T. Nishida, C. Batich, and P. R. Carney, "Structural modifications in chronic microwire electrodes for cortical neuroprosthetics: a case study," IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2006

  6. Technical Issues with BMIs • An implantable BMI requires beyond of state of the art technology: • Ultra low power • Ultra miniaturized • Huge data bandwidth/power form factor • Packaging

  7. 50µm pitch Electrodes Modular Electrodes IC Electrode attachment sites Electrode Array Patterned Substrate Flexible substrate 12mm Flip-chip connection IF-IC 18 mm + Thru vias to Battery Thru vias to RX/Power Coil Battery RFIC 28mm TX antenna Coil Supporting screws 12.5 mm Supporting Substrate 15mm 3.5 mm Coil winding Coin Battery (10 x 2.5 mm) FWIRE: Florida Wireless Implantable Recording Electrodes Specifications: 16 flexible microelectrodes (40 dB, 20 KHz) Wireless (500 Kpulse/sec) 2mW of power (72-96 hours between charges)

  8. RatPack Low-Power, Wireless, Portable BMIs • Requirements • Total Weight: < 100g • Battery Powered: Run for 4 hours • Implantable • Biocompatible • Heat flux: < 50 mW/cm2 • Power dissipation should not exceed a few hundred milliwatts • Backpack • Small form factor • Speed vs. Low Power

  9. UF PICO System PICO system = DSP + Wireless Generation 3

  10. General Architecture BCI (BMI)bypasses the brain’s normal pathways of peripheral nerves (and muscles) J.R. Wolpaw et al. 2002

  11. BMIs: How to put it together? • NeoCortical Brain Areas Related to Movement Posterior Parietal (PP) – Visual to motor transformation Premotor (PM) and Dorsal Premotor (PMD) - Planning and guidance (visual inputs) Primary Motor (M1) – Initiates muscle contraction

  12. Motor Tasks Performed Data Task 1 • 2 Owl monkeys – Belle, Carmen • 2 Rhesus monkeys – Aurora, Ivy • 54-192 sorted cells • Cortices sampled: PP, M1, PMd, S1, SMA • Neuronal rate (100 Hz) and behavior is time synchronized and downsampled to 10Hz Task 2

  13. 100 msec Binned Counts Raster of 105 neurons (spike sorted)

  14. Ensemble Correlations – Local in Time – are Averaged with Global Models

  15. Computational Models of Neural Intent • Three different levels of neurophysiology realism • Black Box models – function relation between input - desired response (no realism!) • Generative Models –state space models using neuroscience elements (minimal realism). • White models – significant realism (wish list!)

  16. Optimal Linear Model • The Wiener (regression) solution • Normalized LMS with weight decay is a simple starting point. • Four multiplies, one divide and two adds per weight update • Ten tap embedding with 105 neurons • For 1-D topology contains 1,050 parameters (3,150) w0 w9 Z-1 delay of 1 sample S adder wi(n) parameter i at time n

  17. 3-D, 2-D Trajectory Modeling and Robot Control • Collaboration with Miguel Nicolelis, Duke University • Sponsored by DARPA

  18. Time-Delay Neural Network (TDNN) • The first layer is a bank of linear filters followed by a nonlinearity. • The number of delays to span I second • y(n)= Σwf(Σwx(n)) • Trained with backpropagation • Topology contains a ten tap embedding and five hidden PEs– 5,255 weights (1-D) Principe, UF

  19. Multiple Switching Local Models • Multiple adaptive filters that compete to win the modeling of a signal segment. • Structure is trained all together with normalized LMS/weight decay • Needs to be adapted for input-output modeling. • We selected 10 FIR experts of order 10 (105 input channels) d(n)

  20. Recurrent Multilayer Perceptron (RMLP) – Nonlinear “Black Box” • Spatially recurrent dynamical systems • Memory is created by feeding back the states of the hidden PEs. • Feedback allows for continuous representations on multiple timescales. • If unfolded into a TDNN it can be shown to be a universal mapper in Rn • Trained with backpropagation through time

  21. Generative Models for BMIs • Use partial information about the physiological system, normally in the form of states. • They can be either applied to binned data or to spike trains directly. • Here we will only cover the spike train implementations. Difficulty of spike train Analysis: Spike trains are point processes, i.e. all the information is contained in the timing of events, not in the amplitude of the signals!

  22. Particle Filters for Point Processes Neural Tuning function Kinematic State spike trains NonGaussian Prediction Instantaneous tuning model spikes kinematics Linear filter nonlinearity f Poisson model Updating P(state|observation)

  23. Generative Data Modeling ….. ….. Hidden Processes (Brain areas) ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. ….. Neural Channels Observable Processes (probed neurons) ….. ….. ….. Time

  24. BMI lessons learned BMIs are beyond the Proof of Concept stage, but…. Present systems are signal translators and will not be the blue print for clinical applications Current decoding methods use kinematic training signals - not available in the paralyzed I/O models cannot contend with new environments without retraining BMIs should not be simply a passive decoder – incorporate cognitive abilities of the user

  25. BMI lessons learned BMIs are beyond the Proof of Concept stage, but…. Present systems are signal translators and will not be the blue print for clinical applications Current decoding methods use kinematic training signals - not available in the paralyzed I/O models cannot contend with new environments without retraining BMIs should not be simply a passive decoder – incorporate cognitive abilities of the user

  26. A Paradigm Shift for BMIs! Neural Signal Processing DSP algorithm Desired response • During training the user actions create a desired response to the DSP algorithm. • During testing the DSP algorithm creates an approximation to the desired response.

  27. A Paradigm Shift for BMIs! Neural Signal Processing Control Algorithm Learning Algorithm X • The control algorithm learns through reinforcement to achieve common goals in the environment. • Shared control with user to enhance learning in multiple scenarios and acquire the net benefits of behavioral, computational, and physiological strategies

  28. Construction of a New FrameworkHow to capitalize on the perception-action cycle? EXTERNAL WORLD DOES ACTION MEET FUTURE REALITY? Causality line SENSORY STIMULUS LIMBIC SYSTEM PAST FUTURE Body line PREDICTIVE MODELING ORGANIZED PAST EXPERIENCE INTERNAL REPRESENTATION • The brain is embodied and the body is embedded • Need to quantify Brain State at different time resolutions • Intelligent behavior arises from the actions of an individual seeking to maximize received reward in a complex and changing world. • The BMI must engage and dialogue with the user: • Exploits better engineering knowledge • Utilizes cognitive states • Dissects behavior top-down • Exploits rewards • Learns with use • Propose Reinforcement Learning to train the BMI.

  29. Reward Learning Involves a Dialogue AGENT actions states rewards ENVIRONMENT Goal Start Relation between the agent and its environment. Environment: You are in state 14. You have 2 possible actions. Agent: I'll take action 2. Environment: You received a reinforcement of 17.8 units. You are now in state 13. You have 2 possible actions. Agent: I'll take action 1. repeat

  30. CABMI involves TWO intelligent agents in a cooperative dialogue!!! User’s neuromodulation sets the value function for the CA COMPUTER AGENT actions rewards states environment ROBOT Both the CA and the user have the same reward in 3D space RAT’S BRAIN RAT’S BRAIN

  31. Features of co-adaptive BMI 31 Enables intelligent system design in BMIs Both systems adapt in close loop in a very tight coupling between brain activity and computer agent ( CA states are specified by brain activity). User must incorporate the CA in its world (can a rat learn this?) CA must decode brain activity for its value function (can it model the signature of behavior?). In fact CABMI is a “symbiotic” biological-computer hybrid system.

  32. Experiment workspace [top view] The user learns first to associate levers with water reward in a training phase. In brain control, it progressively associates the blue guide LED of the robotic arm with the target lever LEDs. Only when the robot presses the target lever it will get reward.

  33. Experiment workspace [top view]

  34. Experimental CABMI Paradigm Incorrect Target Correct Target Starting Position Grid-space Map workspace to grid Robot Arm Rat Match LEDs • 27 discrete actions • 26 movements • 1 stationary Match LEDs Rat’s Perspective Water Reward Left Lever Right Lever

  35. Experimental CABMI Paradigm 35 CA rewards are defined in 3D at the target lever positions. RL is used to train the CA in brain control (tabula rasa, i.e. no a priori information). During brain control, shaping of the reward field increases the level of difficulty across multiple sections with an adjustable threshold target.

  36. Neuromodulation defines the States Hall, Brain Research (1974) Sampling rate 24.4 kHz Bilateral Premotor/motor Areas 32 channels Spike sorted data

  37. Performance metrics Performance metrics: Percentage of trials earning reward Average control time required to reach a target 4 sessions were simulated using random action selection to estimate chance performance for the CABMI in increasing difficulty tasks.

  38. % trials earning rewardtime to achieve reward Performance in 4 tasks of increasing difficulty

  39. Closed-Loop RLBMI Robot workspace in rat visual field of view. BLUE – Robot GREEN - Lever Functional levers Non-functional levers Top-view of the rat behavioral cage.

  40. Event Related Desynchronization (ERD) and synchronization (ERS) • It is well established that preparation, execution, and also imagination of movement produce an event-related desynchronization (ERD) over the sensorimotor areas, with maxima in the alpha band (mu rhythm, 10 Hz) and beta band (20 Hz). • The mu ERD is most prominent over the contralateralsensorimotor areas during motor preparation and extends bilaterally with movement initiation • ERD during hand motor imagery is very similar to the pre-movement ERD, i.e., it is locally restricted to the contralateralsensorimotor areas

  41. Event Related Desynchronization (ERD) and synchronization (ERS) • During movement preparation and execution, an increase of synchronization (ERS) in the 10-Hz band normally appears over areas not engaged in the task (idling) • ERS can also be observed after the movement, over the same areas that had displayed ERD earlier

  42. Beta rebound following movement and somatosensory stimulation • The general finding is that beta oscillations are desynchronized during preparation, execution, and imagination of a motor act • After movement offset, the beta band activity recovers very fast (<1 s) and short-lasting beta bursts appear. • The occurrence of a beta rebound related to mental motor imagery implies that this activity does not necessarily depend on motor cortex output. • A number of experiments have also shown beta oscillations to be sensitive to somatosensory stimulation

  43. ERS (Blue) and ERD (Red) ERD ERS 12.0 Hz +/- 1.0 10.9 Hz +/- 0.9 Pfurtscheller

  44. ERS (Blue) and ERD (Red) Pfurtscheller

  45. Beta ERS Pfurtscheller

  46. Alpha and Beta ERS Pfurtscheller

  47. Signal Processing for ERD/ERS • Bandpass filtering between 9-13 Hz will emphasize this component. • Estimate the power • Place a statistical threshold for detection. • Alternatively use PSD and threshold the appropriate frequency band.

  48. Paradigm 1 (http://www.dcs.gla.ac.uk/~rod/Videos.html)

  49. Paradigm 2

  50. Event Related Potentials • ERPs are a signature of cognition. They signal a massive communication amongst brain areas (kind of the brain’s impulse response to an internal stimulus). • This is very good, but the problem is that it is normally much smaller than the ongoing EEG activity (i.e. the SNR is negative).

More Related