1 / 71

Using a Low-Cost Electroencephalograph for Task Classification in HCI Research

Using a Low-Cost Electroencephalograph for Task Classification in HCI Research. Johnny C. Lee Carnegie Mellon University. Desney S. Tan Microsoft Research. UIST 2006, Montreux Switzerland. NY Times Magazine, October 16, 2005. National Geographic, March 2005. Brain-Computer Interfaces (BCI).

casey
Download Presentation

Using a Low-Cost Electroencephalograph for Task Classification in HCI Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using a Low-Cost Electroencephalograph for Task Classification in HCI Research Johnny C. Lee Carnegie Mellon University Desney S. Tan Microsoft Research UIST 2006, Montreux Switzerland

  2. NY Times Magazine, October 16, 2005 National Geographic, March 2005

  3. Brain-Computer Interfaces (BCI) A direct technological interface between a brain and computer not requiring any motor output from the user Example Conferences/Journals with BCI interests: Neural Information Processing Systems (NIPS) IEEE Transactions on Biomedical Engineering IEEE Transactions on Neural Systems and Rehabilitation Engineering

  4. Why is this relevant to UIST or HCI? BCI research traditionally focuses on exploratory neuroscience and rehabilitation engineering. Brain sensing could provide valuable data about: - engagement - cognitive work load - surprise - satisfaction - frustration Potentially helpful to Context Sensitive or Evaluation Systems

  5. Values of HCI Values of BCI To use any means necessary to demonstrate that brain-computer interaction is possible. To use reasonable means to achieve a practical benefit to many users. We’d like to: It is okay to: • use equipment costing $100K to +$1 million USD • use highly invasive surgical procedures • require hours or days of operant conditioning • remove data from poor performing subjects use fairly affordable and accessible equipment be safe for repeated and extended use be usable without requiring significant user training use data from all subjects to evaluate its performance VS

  6. Where do we start?

  7. Brain Sensing/Imaging Technologies MRI CT ECoG SPECT PET MEG fMRI EROS/fNIR EEG Currently Impractical for HCI - Safe, easy, no medical expertise

  8. EEG – Electroencephalograph the neurophysiological measurement of the electrical activity of the brain by recording from electrodes placed on the scalp (skipping the lower level neurophysiology) • Measures the voltage difference between two locations on the scalp • Only picks up gross, macroscopic, coordinated, and synchronized firing of neurons near the surface of the brain with perpendicular orientation to the scalp. (thus majority of activity is hidden) Analogous to holding a thermometer up to the side of a PC case

  9. EEG Devices Manufacturer: BioSemi Channels: 64-128 Cost: ~$30K USD Manufacturer: EGI Systems Channels: 128-512 Cost: $100K-$250K USD

  10. The Brainmaster • Lowest cost FDA approved device • Designed for home and small clinical use. • Only $1500 USD • Specs: • 2-channels • 8-bit at 4µV resolution • 256 samples/sec • Has yet to be validated for BCI research work. • If it works, it lowers the entry bar for BCI research.

  11. Validating the Device (and ourselves) 1.Validate the device Can we get useful data from such a low-end device? 2. Validate ourselves To explore this space, we must be able to collect our own data.

  12. Validating the Device (and ourselves) Keirn, Z., “A New Mode of Communication Between Man and His Surroundings”, IEEE Transactions on Biomedical Engineering, Vol. 37, No. 12, 1990. • Data is available for download • Data has not been reproduced in the past 15 years • Some computational BCI researchers have just used this data. • State of the art does is not a great deal better.

  13. Reproducing the Keirn Data We adapted procedure from Keirn to better control potential confounds. 3 tasks: • Rest (Baseline): Relaxation and clearing of mind • Math: Mental arithmetic, prompted with “7 times 3 8 5” • Rotation: Mentally rotate an object, prompted with “peacock” Tasks from the original paper were designed to elicit hemispheric differences.

  14. Experimental Procedure User is instructed to keep eyes closed, minimize body movement, and not to vocalize part of the tasks. For each task, a computer driven cue is given: Rest, Math, Rotate Following Math and Rotate, the experimenter says either the math problem or object

  15. Experimental Procedure Block design adapted from Kiern task (14 seconds) trial Rot Math Rest Math Rest Rot Rest Rot Math Rot Rest Math Rest Math Rot Math Rot Rest session

  16. Experimental Procedure Rot Math Rest Rot Math Rest Rot Math Rest Math Rest Rot Math Rest Rot Math Rest Rot Rest Rot Math Rest Rot Math Rest Rot Math Rot Rest Math Rot Rest Math Rot Rest Math Rest Math Rot Rest Math Rot Rest Math Rot Math Rot Rest Math Rot Rest Math Rot Rest 3 sessions per subject Many short tasks prevent correlation with EEG drift

  17. Experimental Procedure Rot Math Rest Rot Math Rest Rot Math Rest Math Rest Rot Math Rest Rot Math Rest Rot Rest Rot Math Rest Rot Math Rest Rot Math Rot Rest Math Rot Rest Math Rot Rest Math Rest Math Rot Rest Math Rot Rest Math Rot Math Rot Rest Math Rot Rest Math Rot Rest Subjects: 8 subjects (3 female) 29-58 years of age All were cognitively and neurologically healthy All right handed

  18. EEG Setup International 10-20 EEG electrode placement system Two channels placed on P3 and P4 with both references tied to Cz. Electrodes are held in place using conductive paste. 5-10 minute preparation.

  19. Processing the Data

  20. Data Processing task (14 seconds) Rot 14 secs

  21. Data Processing task (14 seconds) Rot 14 secs Task Cue

  22. Data Processing task (14 seconds) Rot 14 secs Experimenter Prompt

  23. Data Processing task (14 seconds) Rot 14 secs Task Onset

  24. Data Processing task (14 seconds) Rot 14 secs Performing Task

  25. Data Processing task (14 seconds) Rot 14 secs ~4 secs Performing Task

  26. Data Processing task (14 seconds) Rot 10 secs Performing Task

  27. Removing time for machine learning Most machine learning algorithms don’t handle time series data very well. 10 seconds

  28. Removing time for machine learning • Divide the 10 seconds into 2 sec windows that overlap by 1 sec • Perform signal processing on each of the 9 windows to get our “time less” feature set 2 secs

  29. Removing time for machine learning • Divide the 10 seconds into 2 sec windows that overlap by 1 sec • Perform signal processing on each of the 9 windows to get our “time less” feature set 2 secs

  30. Removing time for machine learning • Divide the 10 seconds into 2 sec windows that overlap by 1 sec • Perform signal processing on each of the 9 windows to get our “time less” feature set 2 secs

  31. Removing time for machine learning • Divide the 10 seconds into 2 sec windows that overlap by 1 sec • Perform signal processing on each of the 9 windows to get our “time less” feature set 2 secs

  32. Removing time for machine learning • Divide the 10 seconds into 2 sec windows that overlap by 1 sec • Perform signal processing on each of the 9 windows to get our “time less” feature set This provides 486 windows per participant 2 secs

  33. Signal features for each window Generic signal features such as mean power, peak frequency, peak frequency amplitude, etc. Features frequently used in EEG signal analysis.

  34. Common EEG Features Raw EEG Spectral Power Theta (4Hz-8Hz) Beta Low (12Hz-20Hz) Beta High (20Hz-30Hz) Gamma (30Hz-50Hz) Delta (1Hz-4Hz) Alpha (8Hz-12Hz)

  35. Feature Processing and Selection The 39 base features from each window are mathematically combined to create 1521 total features. We used a feature preparation and selection process similar to [Fogarty CHI’05] to reduce the number of features: 23 features for 3-task classification (486 examples) 16.4 features for pair-wise classification (324 examples)

  36. Baseline Results – 3 cognitive tasks BayesNet classifier Chance:33.3% 50% 50% 50%

  37. 2 secs 86.5% 68.3% 82.9% 83.8%

  38. We can do better… ???

  39. Throwing time back in… We can average over temporally adjacent windows to improve classification accuracy “Math”

  40. Averaging with Task Transitions Task transitions result in conflicting data in averaging window. High density of transitions will result in lower accuracy.

  41. Averaging with Task Transitions Fewer task transitions will yield better classification accuracy.

  42. Averaging with Task Transitions No transitions and averaging over all data will be the even better.

  43. Classification Accuracy with Averaging +5.1 to +15.7% for 3-tasks Error bars represent standard deviation

  44. So, can we really read minds? Possibly not, we might be really detecting subtle motor movements…. Error bars represent standard deviation

  45. Cognitive/Motor “Fabric” Tasks of varying cognitive difficultly are involuntarily coupled with physiological responses, such as minute imperceptible motor activity. [Kramer ’91] Therefore, it is impossible to completely isolate cognitive activity neurologically intact individuals. Does this matter to neuroscience? Yes Does this matter to HCI? Maybe not

  46. Cognitive/Motor “Fabric” If motor artifacts are reliably correlated with different types of tasks or engagement, why not use those to help the classifier? Requiring users to not move is also very impractical. Non-Cognitive Artifacts detected by EEG: • Blinking • Eye movement • Head movement • Scalpal GSR • Jaw and facial EMG • Gross limb movements • Sensory Response Potentials

  47. Experiment 2 – Game Task To explore this idea of using non-cognitive artifacts to classify tasks using EEG, we chose a PC-based video game task. Halo, a PC-based first person shooter game produced by Microsoft Game Studios. Navigate a 3D environment in an effort to shoot opponents using various weapons. Relatively high degree of interaction with mouse and keyboard input controls.

  48. Game Tasks • Rest – baseline rest task, relax, fixate eyes on cross hairs on center of screen, do not interact with controls. Game elements do not interact with participant. • Solo – navigate environment, interact with elements in the scene, and collect ammunition. Opponent controlled by expert did not interact with participant. • Play – navigate environment and engage opponent controlled by expert. Expert instructed to play at a level just slightly above skill of participant to optimally challenge them.

  49. Game Experimental Procedure • Setup, design, and procedure was similar to first study. • Participants had tutorial and practice time with game controls. • 3 tasks repeated 6 times (counterbalanced) • Tasks were 24 seconds to allow navigation time. • Only 2 sessions were run for each participant • Same 8 participants from first study were run in this study. • Same data preparation and machine learning procedure.

  50. Results – Game Tasks 93.1% Error bars represent standard deviation

More Related