Lunch Talk on Brain-Computer Interfacing Artificial Intelligence, University of Groningen

1. Lunch Talk onBrain-Computer InterfacingArtificial Intelligence, University of Groningen Pieter Laurens Baljon December 14, 2006 12:30-13:00

2. Overview • What is a BCI? • EEG-based BCI • Preprocess, extract features, classify • Functional correlates of features • Our BCI Setup • Online, offline and simulation • Clinical- or theoretical relevance (or both?)

3. What is a BCI • Interface between the brain and computer • Normally: hands and arms, voice • Could be deficient through stroke or ALS • A BCI: • “must not depend on the brain’s normal output pathways of peripheral nerves and muscles”1 • Prosthesis connected to nerveendings is not a BCI

4. What is a BCI Adapted from Carmena et al. 2003, in PLoS Biology 1(2)

5. What is a BCI(Spelling example) YouTube: http://www.youtube.com/watch?v=yhR076duc8M

6. What is a BCI(Pong example) YouTube: http://www.youtube.com/watch?v=qCSSBEXBCbY

7. What is a BCI • Brain signal can come from • Invasive electrodes • Non-invasive measurements • EEG, fMRI, etc. • Underlying assumption • Intentions have discernible counterpart in brain signal

8. EEG-based BCI • Sub fields of EEG-based BCI: • Signal processing on the EEG • Cognitive task for the subject (psychology) • Designing computer application (HMS) • Typical pattern-recognition pipeline • Preprocessing • Feature extraction • Classification (not considered here)

9. The EEG: Preprocessing • Preprocessing • Recombining electrodes can improve SNR 1. Spatial Filtering • Laplacian filters • Subtract surrounding electrodes • Vary distance to surrounding electrodes 2. Statistical recombination • Independent-Component Analysis • Common-Spatial Patterns

10. The EEG: Feature Extraction • Signal is recorded in 2 or more conditions • Features should correlate with condition. • They must be detectable in single trial • Two principal approaches: • Brute force machine learning • Combine all imaginable features • Features with a functional correlate • Potential shifts: Bereitschafts potential • Rhythms: Alpha, mu, beta, etc. • P300: Particular waveform

11. The EEG: Sensorimotor Rhythm (SMR) • Function of periodical brain activity • The predominance of a function • Expressed by spectral power • Many rhythms are ‘idling-rhythms’. • Alpha rhythm over occipetal lobe (~10Hz) • Mu rhythm over motor cortex (~10 Hz)

12. The EEG: Sensorimotor Rhythm (SMR) University college, London & TU Graz VR application, controlling a wheelchair

13. The EEG: (SCP) & P300 • Slow cortical potentials: • Low-pass filtered signal • E.g. Bereitschafts potential • Ability to self regulate • Also used for neurofeedback • To treat ADHD • P300 is ‘evoked potential’ • Less training • Indicate attended target Tetraplegic operating a speller application Outline of a P300 speller application. When target row/column is highlighted, it evokes a P300.

14. Training • Subject: biofeedback • learning to control physiological ‘parameters’ • E.g. Heartrate, EEG-components • System: any Pattern Recognition method • BCI competition: Different sorts of data • Complexity of classifier • Reduces ‘meaningfulnes’ of transformation?

15. Training • No ‘continuous mutual learning’. • Mostly epoch based • Update the system in between sessions • Danger of oscillations in feedback loop. • There is no between-subjects design yet • Due to large inter-subject variability (?) • Could elucidate • Effect of non-linear vs. linear feedback on EEG

16. Our BCI Setup (online) • General purpose framework: BCI2000 • Modular setup for • Amplifier driver • Signal processing • Application • Open-source Borland C++ • Large community: over 100 labs • Initial problems running BCI experiments

17. Our BCI Setup (offline) • Offline analysis in MatLab • Framework to test pattern recognition • Setup similar to BCI2000 • Simple addition of new features, thus far: • Preprocessing: ICA, CSP • Features: Spectral power, Hjorth • Classification: HMM, kNN, LDA, SVM

18. Our BCI Setup (simulation) • Addition to BCI2000. • Signal source can model SMR changes • Collaboration with developers of BCI2000 • Simulation in order to: • reverse engineer inner workings of BCI2000 • pretest settings for adaptivity

19. Clinical- & Theoretical relevance • Most of the research is on healthy subjects • Clinical research poses problems: • Proper operation requires extensive training • ALS Patients are only to learn control if they had it before the injury. • Small body of potential subjects • Birbaumer reports a “significant increase in quality of life” They normally cannot communicate at all.

20. References • [1] J. R. Wolpaw, N. Birbaumer, W. J. Heetderks, D. J. McFarland, P. H. Peckham, G. Schalk, E. Donchin, L. A. Quatrano, C. J. Robinson and T. M. Vaughan, “Brain-computer interface technology: A review of the first international meeting,” IEEE Transactions on rehabilitation engineering, vol. 8, pp. 164–173, 2000. • Slide 1. Cover of the book Mathilda, about a telekinetic girl. Illustration: Quentin Blake • Slide 3. PL Baljon (author) operating a BCI. Private collection. Photo: Mark Span. • Slide 5, 6. Movies from youtube, filmed at CeBIT from Fraunhofer BCI, Berlin BCI. • Slide 7. “Hans-Peter Salzmann gelang es 1996 erst nach monatelangem Training mit dem Thought Translation Device, den Cursor zu steuern.” Source : University of Tübingen • Slide 12. “Controlling a wheelchair in a VR application” Source: University college, London & TU Graz. • Slide 13. Tetraplegic operating a speller device: Source: NIBIB, http://www.nibib.nih.gov/NewsEvents/Calendar/ExhibitBoothLetter grid is taken from the BCI2000 manual. It is an excerpt from a trial with a P300 speller application.