1 / 31

Toward Exploiting EEG Input in a Reading Tutor

Toward Exploiting EEG Input in a Reading Tutor. Jack Mostow, Kai-min Chang, and Jessica Nelson Project LISTEN www.cs.cmu.edu/~listen Carnegie Mellon University.

quinlan-barton
Download Presentation

Toward Exploiting EEG Input in a Reading Tutor

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Toward Exploiting EEG Input in a Reading Tutor Jack Mostow, Kai-min Chang, and Jessica Nelson Project LISTEN www.cs.cmu.edu/~listen Carnegie Mellon University This work was supported by the Institute of Education Sciences, U.S. Department of Education, through Grants R305A080157 and R305A080628 to Carnegie Mellon University. The opinions expressed are those of the authors and do not necessarily represent the views of the Institute or U.S. Department of Education.

  2. Motivation: peer into student’s mind • Identify mental states • Cognitive: effort, recognition, understanding, learning, … • Affective: attention, engagement, frustration, … • Motor: speech, motion, expression,… • Decide accordingly • What to teach next • How to teach it • …

  3. Peering Into Minds: EEG (electroencephalography)

  4. EEG in the Lab Pros: • Controlled experimental conditions • Temporally fine-grained data • Reflects widespread brain activity • Detects relevant mental states: attention (Marosi et al. ’02), engagement (Lutsyuk et al. ’06), frustration (Berka et al. ’06) Cons: • Impractical for schools! • Wet electrodes (gel or saline) • Expertapplication and monitoring • Money for equipment & personnel • Magnetically-shielded room

  5. New Portable EEG Devices Pros: • Feasible to use in schools • Inexpensive • Dry electrodes • No expert needed • Headphones and microphone Cons: • 1-2 electrodes; not whole brain • What can such devices detect?

  6. Detect brain states related to learning? • Can EEG detect when reading is difficult? • Can EEG detect differences between words? • Which EEG features detect differences best?

  7. Carnegie Mellon 1. Detect when reading is difficult? Easy vs Hard (Grade K-1) (GRE level) We need water, land, and air to live. Earth has all these things. Water covers much of Earth. Most of this water is not safe to drink. Many people are running out of fresh drinking water. In regard to propaganda the early advocates of universal literacy and a free press envisaged only two possibilities: the propaganda might be true, or it might be false. They did not foresee what in fact has happened…

  8. 1. Detect when reading is difficult? • For adults / children • 10 adults in ProjectLISTEN lab • 11children ages 9-10, at school • Reading connected text / isolated words • Aloud / silently

  9. Carnegie Mellon Pilot study setup Reader Child reader Adult reader Project LISTEN’s Reading Tutor

  10. Collecting EEG and Reading Tutor Data Reading Tutor Log Combined Log MindSet (EEG) Log Time Student Text 07-12-2011-09:51:00.10 Kai-min We … 07-12-2011-09:51:00.20 Kai-min need … 07-12-2011-09:51:00.35 Kai-min need … 07-12-2011-09:51:00.55 Kai-min water… 07-12-2011-09:51:01.01 Kai-min land … 07-12-2011-09:51:01.50 Kai-min air … Time Student Text Raw 07-12-2011:09:51:00.10 Kai-min We -33 … 07-12-2011:09:51:00.20 Kai-min need 351 … 07-12-2011:09:51:00.35 Kai-min need 117 … 07-12-2011:09:51:00.55 Kai-min water 661 … 07-12-2011:09:51:00.10 Kai-min land -451 … 07-12-2011-09:51:01.50 Kai-min air 43 ,,, Time Student Raw Attention 07-12-2011-09:51:00.01 Kai-min -33 11 … 07-12-2011-09:51:00.02 Kai-min 351 17 … 07-12-2011-09:51:00.03 Kai-min 117 20… 07-12-2011-09:51:00.04 Kai-min 66 1 14 … 07-12-2011-09:51:00.05 Kai-min -451 26… 07-12-2011-09:51:00.06 Kai-min -3 43… We need water, land, and air to live. Earth has all these things. Water covers much of Earth.

  11. MindSet (EEG) Features Raw EEG signal, reported at 512 Hz Filtered EEG signal, 512 Hz Proprietary “attention” measure, 1 Hz Proprietary “meditation” measure, 1 Hz Proprietary signal quality measure, 1Hz Delta band (1-3 Hz), 8Hz Theta band (4-7 Hz), 8Hz Alpha band (8-11 Hz), 8Hz Beta band (12-29 Hz), 8Hz Gamma band (30-100 Hz), 8Hz Gamma+ band (101-256 Hz), 8Hz

  12. Machine Learning Approach • Train classifiers to detect mental states associated with stimuli • f = Binary Logistic Regression Classifier • X = MindSet (EEG) Features (averaged over stimulus interval) • Y = Easy or hard sentences 1 F 1 1 Y = f( X ) N N

  13. Reader-Specific ClassifiersReader-Independent Classifiers Test on: Train on: Trial 1 Trial 2 Trial 3 Trial 4 Trial 1 Trial 2 Trial 3 Trial 4 Trial 1 Trial 2 Trial 3 Trial 4 Trial 1 Trial 2 Trial 3 Trial 4 Test on: Train on:

  14. Class Size Imbalance • More easy sentences than hard ones • Made the two sets equal in size using 3 approaches: • Random Oversampling (with replacement)

  15. Class Size Imbalance • More easy sentences than hard ones • Made the two sets equal in size using 3 approaches: • Random Oversampling (with replacement) • Random Undersampling

  16. Class Size Imbalance • More easy sentences than hard ones • Made the two sets equal in size using 3 approaches: • Random Oversampling (with replacement) • Random Undersampling • Directed Undersampling (Truncating)

  17. Detect when reading is difficult:Reader-specific classifier accuracy

  18. Detect when reading is difficult:Reader-independent classifier accuracy chance p < .05

  19. 2. Detect differences between words? Hard Words Cologne Chassis Brocade Easy Words Bedroom Chicken Station Non-Words KOF CUN WAF Illegal String FFS GHT NKL

  20. Machine Learning Classifiers • Multinomial logistic regression classifiers • Measure classifier’s rank accuracy (Mitchell et al. ’04) • Use classifier to rank-orderpossible class labels • Rank accuracy = percentile rank of correct label; 0.5 = chance • More sensitive than % correct 1.0 rank accuracy = 0.67 True label  0.5 0.0

  21. Detect differences between words: Reader-specific rank accuracy chance p < .05

  22. Detect differences between words: Reader-independent rank accuracy chance p < .05

  23. 3. Which features detect text difficulty best? • Train classifier using each feature in isolation • Average accuracy across subjects, CV folds • Higher = better

  24. 3. Which features detect word differences best? • Train classifier using each feature in isolation • Average rank accuracy across subjects, CV folds • Higher = better

  25. Which EEG features are sensitive to which lexical properties? • Do EEG spectra reflect natural lexical variance among sentences? • If so, those bands may carry different information for a tutor. • Correlate with MRC Psycholinguistic word properties (Coltheart 81) • Blank = not statistically significant; bold = passes False Discovery Rate test

  26. 4. Conclusions and Future Work • 1-electrode EEG tells easy from hard better than chance. • Frequency bands tap different properties a tutor may use. • Detect mental states such as attention or frustration • Use longitudinal EEG in schools to: • Instrument authentic behavior • Label data based on normal tutor use, not artificial experiments • Detect longer-term learning, not just recency effects • Combat EEG noise with “big data” • Inform tutor redesign and make student-specific models

  27. Questions?

  28. 3. Which features detect word differences best? • Train classifier using each feature in isolation • Average rank accuracy across subjects, CV folds • Higher = better

  29. Which features predict word differences best?

More Related