NeuroPhone : Brain-Mobile Phone Interface using a Wireless EEG Headset

1. NeuroPhone: Brain-Mobile Phone Interface using a Wireless EEG Headset Andrew T. Campbell, TanzeemChoudhury, ShaohanHu, Hong Lu, Matthew K. Mukerjee!, Mashfiqui Rabbi, and Rajeev D. S. Raizada Dartmouth College, Hanover, NH, USA

2. Motivation • Mobile phones and neural signals are present are accessible to many people. • Recent advances in technology has led to the development in low-cost EEG headsets. • Smart phones are now powerful enough to run sophisticated machine learning algorithms. • It is thus easy to interface neural signals with mobile computing paradigms.

3. Introduction • This group proposed to used neural signals to control a mobile phone. • They developed the NeuroPhone system that translates and decodes neural signals to drive a mobile app using off-the-shelf wireless EEG headsets. • This paper demonstrates their brain-controlled address app: • An application that uses the brain signals to select address contacts to call.

4. Introduction • They implement their mobile app using two different paradigms: P300 dialing and “Wink”-triggered dialing. • P300 signals are positive transient deflections in EEG that are elicited in response to a rare or novel stimulus • The eye “Wink” is a type of EMG signal that is generated in response to the contraction of skeletal muscle contraction.

5. Challenges • Research Grade EEG headsets • Expensive (Often costing tens of thousands of dollars) • Offer very robust and reliable EEG signals • Off-the-shelf EEG headsets • More affordable ($100-$500) • Electrode design and amplification are not as robust • Results in noisy, low-quality signals. • Require more sophisticated processing techniques to classify neural events. • Most Off-the-shelf headsets are wireless and thus encrypt the EEG signals. • They are designed for synchronization with a computer (using wireless dongle). • They complicate the process of developing a clean brain-mobile interface.

6. Challenges • There is an energy cost for brain-mobile interfacing: • Continuously streaming raw brain-signals wirelessly • Running classifiers on the phone introduces heavy processor loads. • Brain-mobile phones could likely be used in applications such as: walking, riding in a car or bicycle, shopping, etc. • Many of these cases present significant noise artifacts in the EEG signals. • These signals will need to be filtered out to improve the brain-mobile interface

7. NeuroPhone • The NeuroPhone system uses the iPhone to display pictures of contacts in the phone’s address book. • The pictures are displayed and flashed in random order. • For the EEG mode, the user concentrates on a picture of the person they wish to call. • For the wink mode, the person winks with the left or right eye to make the intended phone call

8. P300 • Whenever the user concentrates on a target stimulus among a pool of non-target stimulus, the target stimulus (flash) will elicit a positive peak in the EEG at around 300ms after stimulus onset (P-300). • The P300 signal can be found on most EEG channels • Common on central and parietal channels

9. NeuroPhone - P300 Paradigm • In This case, there are 6 total stimuli on the screen (5 non-target and 1 target). The user visually attends to one of the photos while each photo is flashed in a random order. Whenever the target photo flashes, a P300 should be generated.

10. Wireless EEG Headset • Emotiv EPOC headset • 14 data electrodes (2 reference electrodes) • Transmits encrypted data wirelessly to a windows-based machine. (802.11) 2.4GHz • Low SNR • Contains build in gyroscope • ~$300

11. Pre-Processing • Signals were band-passed filtered to keep only the relevant information within the P300 range. • Signal averaging was performed to increase the SNR • This improves the quality of the signal while simultaneously adding lag to the system

12. Classification • To reduce complexity, only a subset of relevant channels are used for classification. • Wink Mode • Multivariate, naive Bayesian classifier. • P300 Mode • Decision stump classifer

13. Implementation • Laptop relay is used for decoding of the encrypted Emotiv signals • Encrypted EEG signals are sent from the phone to a laptop for decryption (via WiFi). • Decrypted EEG signals are sent back to the phone. • Signals are sampled at 128 samples per second and transferred to the phone at 4kbps per channel.

14. Wink Mode Classification • Emotiv head-set was put on backwards to place two electrodes directly above the eyes. • Data was collected by having the subject wink multiple times. • Data were labeled as “wink” or “non-wink” • A Bayesian classifier was trained by calculating the mean and variance of each wink and non-wink and building respective Gaussian models. • As can be seen, the two models do not overlap leading to good classification

15. P300 Classification • The Gaussian distributions overlap too much and therefore cannot be classified with a Bayesian classifier. • Signals from each of the six stimuli were band-passed filtered between 0-9Hz. • The highest signal segment at around 300ms after stimulus onset is extracted. • For classification, a decision stump is used where the threshold is set to the maximum value of the extracted segment.

16. Results (Wink-Mode) • Multiple sessions were collected on three subjects. • Subjects performed the test while sitting and while walking • The classifier was trained on five sessions from a single subject and then tested on the remaining subjects. (I think). • Results are shown in table 1 • Precision: % of classified winks that are actual winks • Recall: % of actual winks that are classified as winks. • Accuracy: % of total events that are classified correctly

17. Results (P300 mode) • Data was collected with same set of subjects while sitting, with loud background music and while standing up.

18. Discussion • Although data was classified using the P300 mode, large amounts of averaging is needed to get decent classification accuracies. • This “unresponsiveness” of the system proves to be very frustrating for the end user. • i.e. it can take 100 seconds to initiate a phone call with only 89% chance of dialing the right person (with six to choose from). • This System is currently not in any form to be used by subjects on a regular basis. • Looking into single trial classification techniques to speed up the system.

19. Phone Loading Statistics • The CPU usage when running the application: • 3.3% for the iPhone (iphone 3g?). • Total memory usage: • 9.40MB memory used • (9.14MB are for GUI elements). • Continuous streaming raw EEG channels to the phone, and processing signals lead to battery drain (no quantitative measure given) • Looking into duty cycling to solve this phone.