Automated Web-Based Behavioral Test for Early Detection of Alzheimer’s Disease

1. Automated Web-Based Behavioral Test for Early Detection of Alzheimer’s Disease Eugene Agichtein*, Elizabeth Buffalo, Dmitry Lagun, Allan Levey, Cecelia Manzanares, JongHo Shin, Stuart Zola • Emory University Intelligent Information Access Lab

2. Emory IR Lab: Research Directions • Modelingcollaborativecontent creation for information organization and indexing. • Miningsearchbehavior data to improve information finding. • Medical applications of Search, NLP, behavior modeling.

3. Mild Cognitive Impairment (MCI) and Alzheimer’s Disease • Alzheimer’s disease (AD) affects more than 5M Americans, expected to grow in the coming decade • Memory impairment (aMCI) indicates onset of AD (affects hippocampus first) • Visual Paired Comparison (VPC) task: promising for early diagnosis of both MCI and AD before it is detectableby other means

4. VPC: Familiarization Phase

5. VPC: Delay Phase Delay

6. VPC: Test Phase

7. VPC Task: Eye Tracking Equipment

8. Subjects with Normal Visual Recognition Memory > 66% of time on Novel Images

9. VPC:Low Performance Indicates Increased Risk for Alzheimer’s Disease Eugene Agichtein, Emory University 1. Detects onset earlier than ever before possible 2. Sets stage for intervention

10. Behavioral Performance on the VPC test is a Predictorof Cognitive Decline Eugene Agichtein, Emory University [Zola et al., AAIC 2012] Scores on the VPC task accurately predicted, up to three years prior to a change in clinical diagnosis, MCI patients who would progress to AD, and Normal subjects who would progress to MCI

11. VPC: Gaze Movement Analysis Lagun et al., Journal of Neuroscience Methods, 2011 Visual examination behavior in the VPC test phase. In this representative example, the familiar image is on the left (A), and the novel image is on the right (B), for a normal control subject. The detected gaze positions are indicated by blue circles, with the connecting lines indicating the ordering of the gaze positions.

12. Technical Contribution: Eye Movement Analysis Lagun et al., Journal of Neuroscience Methods, 2011

13. Significant Performance Improvements Lagun et al., Journal of Neuroscience Methods, 2011

14. Our Big Idea: Web-based VPC task (VPW)with E. Buffalo, D. Lagun, S. Zola • Web-based version of VPC without an eye tracker • Can be administered anywhere in the worldon any modern computer. • Can adapt classification algorithms to automatically interpret the viewing data collected with VPW

15. VPC-W Architecture

16. VPC-W: basic prototype demo ViewPortposition Familiarization (identical images) Delay Test (novel image on left)

17. Experiment Overview • Step 1: Optimize VPC-W on (presumably) Normal Control (NC) subjects • Step 2: Analyze VPC-W subject behavior with both gaze tracking and viewport tracking simultaneously • Step 3: Validate VPC-W prediction on discriminating Impaired (MCI/AD) vs. NC

18. VPC-W: Novelty Preference Preserved Self-reported elderly NC subjects tested with VPC-W over the internet exhibit similar novelty preference to that of VPC. Single-factor ANOVA reveals no significant difference between VPC and VPC-W subjects

19. VPC vs. VPC-W: Similar Areas of Interest Areas of attention: heat map for VPW (viewport-based) is concentrated in similar areas to VPC (unrestricted eye-tracking) . VPC VPC-W Quantifying viewing similarity: Coarse measure: divide into 9 regions (3x3), rank by VPC and VPW viewing time. The Spearman rank correlation varies between 0.56 and 0.72 for different stimuli. VPC ranking VPC-Wranking

20. Actual Gaze vs. Viewport Position Attention w.r.t. ViewPort

21. Eye-Cursor Time Lag Analysis XY: minimum at -75.00 ms 199.8578X:minimum at -90.00 ms 161.8480Y:minimum at -35.00 ms 116.3665

22. Viewport Movement ~ Eye Movement Normal elderly subject (NP=88%, novel image is on left). Impaired elderly subject (NP=49%, novel image is on left).

23. Exploiting Viewport Movement Data Novelty Preference + fixation duration distribution

24. VPC-W Results: Detecting MCI 21 Subjects (11 NC, 10 aMCI), recruited @Emory ADRC: Accuracy on the pilot data comparabletobest reported values for manually administered cognitive assessment test (MC-FAQ, reported accuracy, specificity, and sensitivity of 0.83, 0.9, and 0.89 respectively) (Steenland et al., 2009). Accuracy, Sensitivity, Specificity, and AUC (area under the ROC curve) for automatically classifying patients tested with VPC-W using 5-fold, 10-fold, and leave-one-out cross validation.

25. Current Work • Analysis: • Applying deep learning and “motif” analysis for more accurate analysis of trajectory • Incorporating visual saliency signals • Data collection: • Longitudinal tracking of subjects • “Test/Retest”: effects of repeated testing • Sensitivity analysis: for possible use in drug trials • Wide range of “normative” data using Mturk worker pool

26. Future Directions and Collaboration Possibilities • Can we apply similar or the same techniques for cost-effective and accessible detection of: • Autism (previous work on difference in gaze patterns) • ADHD • Stroke/Brain trauma • Other possibilities? • What can we learn about the searcher from their natural search and browsing behavior? • Image search and examination preferences (anorexia) • Correlate behavior with biomarkers (Health 101 cohort)

27. VPC-W Summary • VPC-W, administered over the internet, elicits viewing behavior in normal elderly subjects similar to eye tracking-based VPC task in the clinic. • Preliminary results show automatic identification of amnesticMCI subjects with accuracy comparable to best manually administered tests. • VPC-W and associated classification algorithms could facilitate cost-effective and widely accessible screening for memory loss with a simple log on to a computer. • Other potential applications: online detection and monitoring of ADD, ADHD, Autism and other neurological disorders. • This project has the potential to dramatically enhance the current practice of Alzheimer’s clinical and translational research.

28. Eye Tracking for InterpretingSearch Behavior • Eye tracking gives information about searcher interests: • Eye position • Pupil diameter • Saccades and fixations Camera Reading Search

29. We Will Put an Eye Tracker on Every Table! - E. Agichtein, 2010 • Problem: eye tracking equipment is expensive and not widely available. • Solution: infer searcher gaze position from searcher interactions.

30. InferringGaze from Mouse Movements Guo & Agichtein, CHI WIP 2010 Predicted Actual Eye-Mouse Coordination No Coordination (35%) Bookmarking (30%) Eye follows mouse (35%)

31. Post-click Page Examination Patterns • Two basic patterns: “Reading” and “Scanning” • “Reading”: consuming or verifying when (seemingly) relevant information is found • “Scanning”: not yet found the relevant information, still in the process of visually searching

32. Cursor Heapmaps (Reading vs. Scanning)[Task: “verizon helpline number”] Relevant (dwell time: 30s) Not Relevant (dwell time: 30s) Move cursor horizontally  “reading” Passively move cursor  “scanning”

33. Typical Viewing Behavior (Complex Patterns) [Task: “number of dead pixels to replace a Mac”] Relevant (dwell time: 70s) Not Relevant (dwell time: 80s) Keep the cursor still and scroll  “scanning” dominant Actively move the cursor with pauses  “reading” dominant