Brain-Computer Interfaces for Communication in Paralysis: A Clinical Experimental Approach

1. Brain-Computer Interfaces for Communication in Paralysis: A Clinical Experimental Approach By Adil Mehmood Khan

2. Contents ● Introduction ● BCI for communication in paralysis ● BCI software ● Self-regulation of SCPs and training ● Spelling through brain-cmputer communication

3. Brain-Computer-Interface (BCI) ? “A system for controlling a device e.g. computer, wheelchair or a neuroprothesis by human intention which does not depend on the brain’s normal output pathways of peripheral nerves and muscles” [Wolpaw et al., 2002]. HCI – Human Computer Interface DBI – Direct Brain Interface (University of Michigan) TTD – Thought Translation Device (University of Tübingen)

4. BCI_ Some examples of BCI applications BCI Leeb et al., Computational Intelligence and Neuroscience, 2007 (doi:10.1155/2007/79642)

5. Brain Computer Interfaces ● Allow patients to control a computer by concious changes of brain activity ● Provide a means of communication to completeley paralysed patients: amyotrophic lateral sclerosis (ALS), cerebral palsy, locked in syndrome ● Can be used to control a cursor, select symbols, control external devices like orthesis / prothesis (depending on type of BCI) ● Have a very low data rate, typical a few bit per second or less ● First results in the 1970ies (Vidal, visual evoked potentials, VEP-BCI)

6. Brain Computer Interfaces Principles of operation:

7. Brain Computer Interfaces – Electrophysilogical Activities used ● SCP Slow Cortical Potentials ● Mu Movement Imagination ● P300, SSVEP ERP-Analysis ● cortical neurons, direct brain interfaces The control information is extracted from the real time EEG-recording http://www.wired.com/news/images/full/thoughtlock1_f.jpg

8. Brain Computer Interfaces – SSVEP ● Steady State Visual Evoked Potentials derived from the visual (occipital) cortex ● focussing attention to visual stimuli of different frequency shows up in the EEG freqeuncy bands ● relibable and high transfer rate, but some prerequisites (eyes) http://www.iua.upf.es/activitats/ semirec/semi-Reilly/

9. Brain Computer Interfaces – SCP BCIs ● detection of slow cortical potentials (SCPs) ● needs DC EEG Amplifiers ● first successful device end 1990‘s: Niels Birbaumers Thought translation device intensive training was necessary to gain control over the SCP waves SCPs: DC-shifts, slow negativation of cortical areas Preparation of movement and cognitive tasks, Several hundert milliseconds before the task Patinet using TTD to write a letterhttp://www.heise.de/ct/06/18/088/bild1.jpg

10. Brain Computer Interfaces - μ-rhythm BCIs ● μ–rhythm is the idle-rhythm of the motor cortex ● frequencies around 10 and 18 Hz. ● ERD / ERS – event related desynchronisation / synchronisation movements or imagination of movements inhibit the μ–rhythm Berlin-BCI, http://www.fraunhofer.de/

11. Brain Computer Interfaces - P300 BCIs ● P300 wave – posivite component in the event related potential, 300ms after a stimulus ● natural response to events considered as important ● selection of a symbol: count the flashes, algorithm averages trails and finds a P300 P300 runtime user interface

12. Brain Computer Interfaces - μ / P300 comparison μ - BCIs P300 BCIs Require training do not require training 2d-control possible 1D control only movement imagination concentration / decision affected by movement affected by distraction

13. BCI for Communication in Paralysis • An overview of different approaches to BCIs developed at Institute of Medical Psychology and Behavioral Neurobiology. • Thought-Translation Device (TTD). • Brain-Controlled Web Browser. • Visual and Auditory feedback modes. • Oscillatory Features based Classification.

14. SCPs: A Brief History • An initial application: Epileptic Seizures • Down regulation of brain potentials towards a positive amplitude. • BCI for “locked-in syndrome” • Communication through self-regulation of SCPs • Also known as TTD

15. BCI Software • Thought Translation Device: Components • EEG Amplifier • EEG8 system • G.tec amplifiers • BrainAmp system • Two Monitors • One for operator (supervise the training) • One for patient (feedback) • Sampling frequency: 256Hz • Digitized with 16 bits/sample • Amplitude range: +(-) 1mV • Low frequency cutoff: 0.01 Hz • High frequency cutoff:: 40-70 Hz

16. TTD Feedback and Communication System • The current version of TTD software is derived from BCI2000 standard

17. TTD Feedback and Communication System

18. TTD Software Data acquisition and storage Online signal processing Classification Feedback and application interface

19. Brain Computer Interfaces - BCI2000 ● Research Platform for BCI Systems ●Written by Gerwin Schalk, Wadsworth Center, Albany (NY) ●Modular structure: Signal Aquisition, Signal Processing and User Application communicatie via TCP/IP ●Operator module used for configuration of the other modules ● various user tasks availbale: 1D/2D cursor, Speller, P300, SCP ● free for academic use ● driver for OpenEEG available http://www.bci2000.org/

20. BCI2000: Components • Filters • Spatial, temporal, and spectral • Online artifact detection and correction • Classification • Linear Discriminant Analysis (LDA) • Simple Threshold Classification • Support Vector Machine (SVM) • MATLAB interfaces.

21. Self-Regulation of SCPs • ● Slow event-related direct-current shifts of the EEG. • ●They last from 0.3 seconds up to several seconds. • ●Occur as a result of external or internal events. • ● Negative shift is related to excitability of neurons. • ● Positive shifts are measured during the execution • of cognitive tasks • ●Healthy subjects as well as patients can learn to • produce positive or negative SCPs • ●Training requires feedback • Visual • Auditory

22. Self-Regulation of SCPs ● Recording site for feedback signal is usually Cz. ●EEG is usually recorded from 3-7 Ag/AgCI-electrodes placed at Cz, C3, C4, Fz, and Pz. ● vEOG is recorded using a bipolar channel for online and offline artifact correction. ●A fixed percentage of vEOG signal is subtracted from the SCP signal at Cz for EOG correction. ● SCPs are calculated by applying a 500ms moving average to EEG signal.

23. Self-Regulation of SCPs • ● With the visual feedback modality: • Subjects viewed the course of their SCPs as the vertical movement of feedback cursor. • Vertical cursor movement corresponded to the SCP amplitude. • Task was to move the cursor towards the modality indicated by a red rectangle.

24. Self-Regulation of SCPs: Training Process

25. Self-Regulation of SCPs: Training Process ● First 2-4 sec of the trial. ● Target is illuminated in red. ● Allows the subject to prepare for the corresponding SCP regulation.

26. Self-Regulation of SCPs: Training Process ● Feedback is provided by the vertical position of the steady horizontally moving cursor. ● Cortical negativity moved the cursor up. ● Positivity moved the cursor down. ● Center of the screen corresponded to the baseline. ● Task is to move the cursor to the red area.

27. Self-Regulation of SCPs: Training Process • ● A response is classified as correct if: • Average potential carried the correct polarity. • Or is inside the target boundaries of the required goal. • ● Classification methods, such as LDA or SVM can be • used for improvement of the correct response rate.

28. Self-Regulation of SCPs: Performance • ● Performance • Percentage of correct responses in valid trials. • After a rate of 75% correct responses, subjects were trained to select letters and write messages • ● Subjects typically reach this level after 1 to 5 months • of training, with 1 to 2 training days per week. • ● A training day comprises 7 to 12 runs, and a run • comprises 70 to 100 trials.

29. Applications • Spelling by Brain-Computer Communication: • A program driven by “yes” or “no” responses, which serve as “select” or “reject” commands. • Requires three intervals in one trial. • Allows user to select letters from a language alphabet and to • combine letters into words and sentences.

30. Applications • Spelling by Brain-Computer Communication: • A program driven by “yes” or “no” responses, which serve as “select” or “reject” commands. • Requires three intervals in one trial. • Allows user to select letters from a language alphabet and to • combine letters into words and sentences. • Presentation of the letter set. • Displayed in target rectangles on the screen.

31. Applications • Spelling by Brain-Computer Communication: • A program driven by “yes” or “no” responses, which serve as “select” or “reject” commands. • Requires three intervals in one trial. • Allows user to select letters from a language alphabet and to • combine letters into words and sentences. • Feedback is provided. • Self regulation of SCP amplitudes is used • to select or reject the letter set.

32. Applications • Spelling by Brain-Computer Communication: • A program driven by “yes” or “no” responses, which serve as “select” or “reject” commands. • Requires three intervals in one trial. • Allows user to select letters from a language alphabet and to • combine letters into words and sentences. • Response interval indicating to user the • result of the selection. • Error correction is done using a “go-back” • option.

33. Spelling by Brain-Computer Communication

34. Spelling by Brain-Computer Communication • Performance: • Writing the most conveniently situated letter , “E,” takes 5 trials. • Writing the most remote sign requires 9 trials, i.e. 36 – 45 sec. • Improvement • A simple personal dictionary to make free spelling less time consuming. • Contains words that are frequently used by patients. • A complete word is suggested after at least two letters have been written. • This word can then be chosen with a single selection response.