Treating Epilepsy via Adaptive Neurostimulation

1. Treating Epilepsy via Adaptive Neurostimulation Joelle Pineau, PhD School of Computer Science, McGill University Congress of the Canadian Neurological Sciences Foundation June 9, 2010

2. Learning objectives • Review the neurostimulation hypothesis for treating epilepsy. • Understand the basic principles of adaptive neurostimulation. • Study a mathematical framework for optimizing the choice of neurostimulation parameters. • Observe results from applying adaptive neurostimulation in vitro. Adaptive Neurostimulation2 Joelle Pineau

3. Disclosure statement This research was supported by the Natural Sciences and Engineering Research Council of Canada and the Canadian Institutes of Health Research. This is joint work with Massimo Avoli, MD, PhD; Keith Bush, PhD; Arthur Guez; Gabriella Panuccio, MD, PhD; and Robert Vincent. Adaptive Neurostimulation3 Joelle Pineau

4. Epilepsy • Epilepsy is a neurological disorder marked by spontaneous seizures. • Affects ~1% of world's population. • Up to 20-25% of those do not benefit from standard treatments (anti-convulsants, surgery). • Causes are varied (pre-disposition, head trauma, fever, tumor, etc.) • What is a seizure? • Abnormal electrical activity in the brain, may produce physical convulsions, or other symptoms. www.chse.louisville.edu/graphics/epilepsy07.jpg Adaptive Neurostimulation4 Joelle Pineau

5. Anatomy of a seizure (in vitro) Adaptive Neurostimulation5 Joelle Pineau

6. Neurostimulation hypothesis External perturbation of an epileptic neural system can alter dynamics away from excitability. Adaptive Neurostimulation6 Joelle Pineau

7. Deep brain stimulation (DBS) • Implanted electrodes electrically stimulate brain tissue. • Recent clinical trials of DBS: • Stimulation of the Anterior Nucleus of the Thalamus in Epilepsy (SANTÉ) • 157 people; 17 sites in US; 2003-2008; sponsored by MedtronicNeuro • Randomized Controlled Trial of Hippocampal Stimulation for Temporal Lobe Epilepsy (METTLE) • 90 people; 1 site in Canada; 2008-2011; sponsored by U. of Calgary • RNSTM System Long-Term Treatment Clinical Investigation: • 280 people; 28 sites in US; 2006-2013; sponsored by NeuroPace • Closed-loop stimulation “detect-then-stimulate” Many parameters to control: stimulation site, stimulation frequency/intensity, stimulation pattern, … www.medgadget.com www.onemedplace.com Adaptive Neurostimulation7 Joelle Pineau

8. Electrophysiology results from in vitro model The literature repeatedly shows 1Hz, 10-200A, fixed stimulation successfully suppresses seizures in vitro. [D’Arcangelo et al., Neurob of Disease. 2005]. Adaptive Neurostimulation8 Joelle Pineau

9. A few interesting open questions • What parameter settings (stimulation site, frequency, intensity, pattern) achieve maximal suppression? • Can we reduce the number and/or intensity of stimulations, while maintaining suppression efficacy? • How can we customize parameters for different subjects? Adaptive Neurostimulation9 Joelle Pineau

10. Adaptive neurostimulation paradigm Objective: create a stimulation device which is • Optimal: maximize seizure reduction + minimize stimulation. • Responsive: strategy evolves as a function of the observation. • Automatic: stimulation strategy learned from data. Adaptive Neurostimulation10 Joelle Pineau

11. Adaptive neurostimulation example Adaptive Neurostimulation11 Joelle Pineau

12. Methods: Data collection in vitro • Electrophysiological recording in the Entorhinal Cortex (B), with stimulation at fixed frequencies in the Subiculum (A). Adaptive Neurostimulation12 Joelle Pineau

13. Methods: Data collection and labeling Experimental protocol: • Control (min. 3 seizures) • Periodic pacing at 0.2Hz (min. 20 minutes). • Recovery (until interval between ictal events stabilizes). • Etc. with 0.5Hz, 1.0Hz, 2.0Hz. Then, manually identify: • Seizure occurrences • Neurostimulation parameters: {0Hz, 0.2Hz, 0.5Hz, 1.0Hz, 2.0Hz} Adaptive Neurostimulation13 Joelle Pineau

14. Methods: Signal processing • Select decision window duration: 1 sec. • Select observation window duration: 13 sec. • Extract observation features using signal processing techniques: • Range, energy, multi-scale Fourier transform Adaptive Neurostimulation14 Joelle Pineau

15. Methods: Training data • Form an input vector, xt, for each decision window, t : xt = {zt, at, ct, zt+1} where zt = observation features at t at = neurostimulation parameters at t ct = cost function at t • The cost function depends on the occurrence of seizures and stimulation delivered: ct = ctseizure +  ctstim wherectseizure = {1 if seizure occurred at time t, 0 otherwise} ctstim = {1 if stimulation occurred at time t, 0 otherwise}  is a free parameter. Adaptive Neurostimulation15 Joelle Pineau

16. Methods: Minimizing the cost function • The objective is to select actions such as to minimize the expected cumulative cost: E [ ct + ct+1 + ct+2 + … + cT | zt ] • Use regression analysis to estimate the cost for different action choices from the training data: Qk(zt, at) = ct + maxaA Qk-1(zt+1, a) • Select the action which minimizes the expected cost: at := argmaxaA Qk(zt, a) Adaptive Neurostimulation16 Joelle Pineau

17. Experimental protocol for validation • Control period (min. 3 seizures). • Periodic pacing at 1.0 Hz (min. 20 minutes). • Recovery period. • Adaptive stimulation strategy (min. 20 minutes). • Recovery period, no stimulation. • Periodic pacing at effective frequencyf = ns/T where ns=# stimulations during adaptive protocol T=duration of adaptive protocol Adaptive Neurostimulation17 Joelle Pineau

18. Proportion of time spent in seizure • Proportion of time spent in seizure, averaged over N=11 slices. * = statistically significant at p=0.05 Adaptive Neurostimulation18 Joelle Pineau

19. Effective frequency of the adaptive protocol Adaptive Neurostimulation19 Joelle Pineau

20. Suppression efficacy for slices with eff > 1Hz * N=11 N=4 * = statistically significant at p=0.1 Adaptive Neurostimulation20 Joelle Pineau

21. Adaptive protocol example #1 (a) Adaptive controller suppresses a seizure by increasing the frequency of stimulation. Adaptive Neurostimulation21 Joelle Pineau

22. Adaptive protocol example #2 (a) Adaptive controller suppresses a seizure by increasing the frequency of stimulation. (b) A short seizure develops, stimulation is applied to shorten its duration. Adaptive Neurostimulation22 Joelle Pineau

23. Adaptive protocol example #3 (a) Adaptive controller suppresses a seizure by increasing the frequency of stimulation. (b) A short seizure develops, stimulation is applied to shorten its duration. (c) Adaptive controller increases frequency to suppress seizure, then decreases frequency. Adaptive Neurostimulation23 Joelle Pineau

24. Limited slice-to-slice variation. Short lifespan. In vitro model has known periodic pacing strategy. Restricted parameter space. Larger variance between subjects. Longer lifetime; disease can evolve over time. No known open-loop strategies. Higher-dimensional action space (more electrodes, intensity settings, etc.) In vivo: Challenges In vitro model In vivo model Adaptive Neurostimulation24 Joelle Pineau

25. Discussion • Animal models of epilepsy provide a rich framework for investigating adaptive neurostimulation strategies. • Most adaptive neurostimulation approaches adopt a “detect-then-stimulate” paradigm. • Our work leverages techniques from the control literature. • Goal is to directly minimize a cost function. • Explicit seizure prediction (or detection) is not required. • Results show good suppression in vitro, in some cases using significantly less stimulation than periodic pacing. • Preliminary evidence suggests that neurostimulation can be used to probe the excitability of the system. Adaptive Neurostimulation25 Joelle Pineau

26. References • J. Pineau, A. Guez, R. Vincent, G. Panuccio & M. Avoli, Treating epilepsy via adaptive neurostimulation: A reinforcement learning approach. Int. J. of Neural Systems.19(4). 2009. • M. Avoli, M. D’Antuono, J. Louvel, R. Kohling, G. Biagini, R. Pumain, G. D’Arcangelo, & V. Tancredi, Network and pharmacological mechanisms leading to epileptiform synchronization in the limbic systems in vitro. Prog. Neurobiol. 68(3). 2002. • G. D’Arcangelo, G. Panuccio, B. Tancredi, M. Avoli. Repetitive low-frequency stimulation reduces epileptiform synchronization in limbic neuronal networks. Neurobiology of Disease. 19(1-2). 2005. • A. Guez, R. Vincent, M. Avoli, J. Pineau. Adaptive treatment of epilepsy via batch-mode reinforcement learning. Innovative applications of Artificial Intelligence. 2008. Adaptive Neurostimulation26 Joelle Pineau

27. Contrasting approaches to adaptive neurostimulation • Use observation window to detect seizures; when a seizure is detected, start neurostimulation. 2. Use observation window to characterize state of the system; apply neurostimulation as necessary to minimize seizure occurrence. Adaptive Neurostimulation27 Joelle Pineau

28. Results: Proportion of decision windows with seizures Adaptive Neurostimulation28 Joelle Pineau

29. Results: Proportion of decision windows with stimulation Adaptive Neurostimulation29 Joelle Pineau

30. Adaptive neurostimulation deployed in vitro Adaptive Neurostimulation30 Joelle Pineau