Overview Gerald Witt & Harry Zhang August 12, 2003

1. SAfety VEhicles using adaptive Interface Technology Phase 1 Research ProgramQuarterly Program Review Overview Gerald Witt & Harry Zhang August 12, 2003

2. SAVE - IT Phase 1 Program Overview • Program Team • Mission and Objectives • Program Plan Summary • Technical Approach • Phase 1 Research model • Team coordination • Schedule • Human Factors research summary

3. Program Team A comprehensive program team has been assembled bringing a unique blend of expertise and complimentary capabilities. Program Manager Gerry Witt DDE Human Factors Team Leader Dr. Harry Zhang DDE Technology Team Leader Greg Scharenbroch DDE Principal Investigator Greg Scharenbroch Seeing Machines Inc. Ford Evaluation Principal Investigator Jeff Greenberg U of Iowa Principal Investigator Dr. John Lee UMTRI Principal Investigator Dr. David Eby DDE Principal Investigator Dr. Harry Zhang GM Evaluation Principal Investigator Scott Geisler • Task Members • Tim Newman • Dr. Branislav Kisacanin • Nancy Edenborough • Michelle Wilkes • Development Focus • Technology • Development • System • Integration Eye Tracking Technology • Task Leader • Jeff Greenberg • Research Focus • Evaluation • Task Leader • Scott Geisler • Research Focus • Evaluation • Task Leaders • Dr. John Lee • Dr. Dan McGehee • Dr. Tim Brown • Research Focus • Distraction Mitigation • Cognitive Distraction • Telematics Demand • Guidelines and • Standards • Evaluation • Task Leaders • Dr. David Eby • Dr. Paul Green • Dr. Bary Kantowitz • Dr. Dave LeBlanc • Research Focus • Scenario ID • Driving Task Demand • Performance • Telematics Demand • Evaluations - on road • Task Leaders • Dr. Zhang • Dr. Smith • Research Focus • Visual Distraction • Intent • Safety Warning • Countermeasures • Data fusion • Benefits Analysis

4. SAVE - IT Mission and Objectives Develop system operational performance requirements and guidelines for adaptive interface conventions Conduct comprehensive human factors research to derive distraction and workload measures for use adaptive interfaces. Identify scalable system concepts and sensing technologies for further research to follow the SAVE-IT program Phase 1 E C T J I B V E O Mission To demonstrate a viable proof of concept that is capable of reducing distraction related crashes and enhancing collision warning effectiveness S Phase 1 Phase 1 Develop and apply evaluation procedures for assessment of safety benefits Phase 2 Provide the public with documentation on human factors research findings for performance and standardization development Phase 1 Advance the deployment of adaptive interface technology countermeasures for distraction related crashes Enhance collision warning effectiveness by optimizing alarm onset based on driver’s workload or distraction

5. IOWA UMTRI Program Plan Summary Phase II Data Fusion, System Integration and Evaluation Phase I Research and Concept Development Evaluation 14A Iowa 14B Ford 14C UMTRI 14D GM 14 Scenario Identification Data Fusion 11ADistraction Mitigation 11BSafety Warning Countermeasures Subcontractors: 11 System Integration Vehicle build Demo. 13 Crash statistics analysis 1 Driving Task Demand Literature review 2A Identify diagnostic measures 2B Develop and validate algorithms 2C Literature review 3A Identify diagnostic measures 3B Develop and validate algorithms 3C Performance Distraction Mitigation Literature review Cognitive distraction Visual distraction 4A Identify countermeasures Cognitive distraction Visual distraction 4B Validate countermeasures Cognitive distraction Visual distraction 4C Cognitive Distraction Literature review 5A Identify diagnostic measures 5B Develop and validate algorithms 5C Telematics Demand Literature review 6A Identify demand levels 6B Validate demand levels 6C Iowa Visual Distraction Literature review 7A Identify diagnostic measures 7B Develop and validate algorithms 7C UMTRI Literature review 8A Identify diagnostic measures 8B Develop and validate algorithms 8C Intent Program Summary and Benefit Evaluation 15 Literature review 9A Identify countermeasures 9B Safety Warning Countermeasures Establish Guidelines & Standards 12 Technology / architecture concept identification 10A Technology / architecture concept car 10B Technology Development 2004 - 2005 2003 DELPHI

6. SAVE-IT Phase 1 Research Model Technology/Architecture Concept Development Human Factors Research Concept Demonstration Adaptive Safety Warning And Distraction Mitigation System Architecture Concepts Scenario Identification Diagnostic Research Driving Performance Real Time Distraction Sensing Requirements Cognitive Distraction Visual Distraction Phase 2 Recommendations and development plan Distraction Assessment Data Fusion Concepts Telematics Demand Driving Task Demand Situational Threat Assessment Concepts Intent CountermeasureTechnology Identification and HMI Concepts Safety Warning Countermeasures Distraction Mitigation

7. Preliminary SAVE-IT Model • Provides real time assessment of driver distraction • Provides global situational threat assessment • Provides adaptive countermeasures. • Provides a watchful eye when your not • Increases safety guard band when required Target Steering Assessment Throttle Pedestrian Detect HMI data fusion Processor Brakes FW Long Range FW Short range Phone Side Detect Vehicle Control • Longitudinal MMM N • Lateral R Long Range Response Req’d Climate Situational R. Short Range Threat Y 3D Audio Assessment Adaptive HMI Environment • Stimulate/Suppress HUD • Warning sensitivity Driver State Monitor MMM • Information priority Haptic FB Phone Displays IP Controls Flashers Warnings Steering CHMSL Throttle • Substantially reduces perceived • false alarm conditions and minimizes • driver disregard. Bio Signs Brakes Eye Tracking/ Oculometrics •

8. Team Coordination • Close team coordination is required to maintain consistency within research, experimental design and conclusions • A bi- weekly Human Factors team meeting is held via conference call. • Schedule review • Design Reviews • Issue discussions and resolutions • Commonization strategy discussion • Common dependant variables • Age groups, etc. • Common development process • Literature review • Report • Design/data collection • Team design reviews • IRB approval • Data collection • Findings and recommendations • Phase 2 planning • Algorithm development and validation plan • Preliminary Phase 2 research plan

9. Phase 1 Schedule Milestones • Literature reviews complete, report submitted to NHTSA/Volpe 9/10/2003 • Final Reports and Recommendations to Delphi 12/31/2003 • Phase 2 planning documentation to Delphi 12/31/2003 • Final report and recommendations to NHTSA/Volpe 3/4/2004 • Phase 2 concept vehicle demonstration to NHTSA/Volpe 3/4/2004 • Phase 2 planning documentation to NHTSA/Volpe 3/4/2004

10. Comparison of Approaches Demand Approach COMUNICAR (Europe) Non-Adaptive- Interface Approach GIDS (Europe) Arousal Approach SAVE-IT: Real-time Adaptive interface Common scenarios among tasks Driving & non-driving demands Driver state (distraction, intent, physiological measures) Safety warning systems CAMP Workload (U.S., Japan) De Waard Comprehensive Safety Management Systems

11. Philosophy of Comprehensive Safety Management Systems • Driver impairment reduces overall attentional capacity. • Driver distraction increases the attention allocated to non-driving tasks and reduces the attention allocated to driving tasks. • Safe driving requires commensurate attention paid to driving tasks. • Required attention to driving tasks varies with driving task demand. • Objectives: To assess distraction, impairment, and driving task demand in order to ensure sufficient attention is paid to driving tasks.

12. Task 1: Crash Statistics Analysis • Objectives • Identify crash scenarios (e.g., rear-end crashes) that the SAVE-IT program should be designed to prevent. • Major Findings • 20-50% of crashes involve some form of driver distraction and inattention. • CDS appears to be best suited for the task. FARS, GES, and HSIS are not appropriate for this task. • Prior research indicated that single-vehicle-run-off-the-road and rear-end crashes are most common scenarios in which driver distraction is a causative factor. • Prior research indicated distracting events include interior (e.g., radio, cell phones, passengers) and exterior (e.g., scenery) objects and events. • Current Status(55% completed) • Literature review report (to Delphi) Completed • Crash data analysis plan In progress • Crash data analysis Sept.-Nov. ’03 • Expert panel meeting Nov. ‘03

13. Task 2: Driving Task Demand • Objectives • Determine the level of attentional demand imposed by the driving environment that represents the required level of attention allocated to the driving tasks. • Major Findings • Analysis of crash data is key because crash rates can be assumed to indicate environmental unpredictability and the amount of attention demanded by the environment. • HSIS is the best suited crash database because it contains information about environmental conditions (e.g., weather, traffic volume, road surface conditions) at the time of crashes. • In laboratories, driving demand can be approximated by visual demand (% of time needed to look at the road to drive safely) as measured with the visual occlusion method. • Current Status (2A 50% completed; 2B 60% completed) • Literature review report (to Delphi) (2A) Completed • HSIS database preparation, review of analysis plan (2A) Completed • Crash data analysis (2A) Aug.-Sept. ’03 • Review of test plan, simulator preparation, pilot testing (2B) Completed • “Visual occlusion” simulator experiment (data collection) (2B) In progress • Data analysis and algorithm development (2B) Sept-Nov. ’03

14. Task 3: Performance • Objectives • Determine performance measures/variables that are diagnostic of driver distraction. • Major Findings • Literature review indicated that there exists very limited data (e.g., distribution data, eye glance data) comparing driving with and without various in-vehicle devices (e.g., radio, phone, navigation device). • NHTSA’s 100-car naturalistic driving study currently conducted at Virginia Tech should be very useful. • Normative data on drugs and driving can be useful. • Current Status (3A 15% completed; 3B 55% completed) • Identification of research needs/gaps (3A) Completed • Literature review report (to Delphi) (3A) In progress • Instrumented car preparation, pilot testing (3B) Completed • Review of test plan (3B) Completed • On-road experiment (data collection) (3B) Summer ’03 • Data analysis and algorithm development (3B) Sept-Dec. ’03

15. Task 4: Distraction Mitigation • Objectives • Develop appropriate countermeasures that can mitigate against excessive levels of distraction. • Major Findings • Research on computer etiquette, negotiated access, and automation challenges is useful. • Used four focus groups (24 participants at Iowa City & Seattle) to determine what activities drivers find distracting (driver/system initiated, technology/non-technology oriented) and what mitigation strategies they prefer. Potential technology for mitigating distraction is viewed positively by some and negatively by others. • Potential countermeasures such as warning, informing, advising, demand minimizing, prioritizing/filtering, locking, etc. can be summarized by a model-based taxonomy of mitigation strategies with degree of intervention and locus of control (driver vs. IVIS) as the dimensions. • Current Status (45% completed) • Literature review report (to Delphi) (4A) In progress • Focus group study (data collection) (4B) Completed • Focus group data analysis and draft report (4B) Completed • Revision of mitigation taxonomy based on focus group input (4B) Completed • Cognitive task analysis (4B) Aug.-Dec. ‘03 • “Driver acceptance” simulator experiment (4B) Aug.-Dec. ‘03

16. Task 5: Cognitive Distraction • Objectives • Determine which measures (performance, driver state, and vehicle state variables) are diagnostic of cognitive distraction. • Develop an algorithm relating diagnostic measures to performance (including RT). • Major Findings • Cognitive distraction may be manifested in terms of driver state (eye movements, scan patterns, ocular responses, psycho-physiological measures), driving performance, and vehicle system state. • Theories and models such as multiple resource theory, malleable resource theory, strategic task management (switching), and ACT-R can be useful. • Hidden Markov Models (representing stochastic sequences where states are not directly observed but are associated with a probability density function) and Support Vector Machines (determining optimal hyperplane separating two classes) will be very useful in predicting driver distraction. • Current Status (40% completed) • Literature review report (to Delphi) (5A) In progress • Experimental design (5B) In progress • Simulator experiment (5B) Aug.-Dec. ‘03

17. Task 6: Telematics Demand • Objectives • Determine distraction potential and prioritization for commonly-used telematics functions. • Major Findings • Distraction potential may be measured in terms of task completion time, number of errors, number of glances, mean glance duration, reaction time, etc. • Current guidelines (e.g., Alliance’s Statement of Principles, SAE J2364) and IVIS Demand model for key task characteristics (visual, auditory, cognitive, manual) can be very useful. • Current Status (6A 60% completed, 6B 2% completed) • Summary of prior research (6A) Nearly completed • Literature review report (to Delphi) (6A) In progress • Preliminary test plan (6B) Completed • Simulator experiment (6B) Sept.-Dec. ‘03

18. Task 7: Visual Distraction • Objectives • Identify eye glance measures that are diagnostic of visual distraction and that can be used in real-time, adaptive interface technology systems. • Determine performance (including RT) effects of visual distraction. For example, RT = f(glance duration, glance frequency, etc.). • Major Findings • Prior research indicated that visual distraction (off-road glances) degrades driving performance (e.g., SDLP, lane departures, RT) and increases the likelihood of crashes. • Prior research rarely used automatic eye tracking systems to measure visual behaviors in real time and focused on task-based (e.g., radio tuning) rather than time-based visual behaviors. • Real-time measurement of time-based visual behaviors is critical to SAVE-IT. • Current experiments measured visual behaviors in real time using Seeing Machines eye tracking system and indicated that visual distraction degraded performance (e.g., RT). • Current Status (7A completed; 7B 40% completed) • Literature review report (to Delphi) (7A) Completed • Experiment 1 data collection (7B) Completed • Review of Experiment 2 test plan, facility preparation (7B) Completed • Experiment 2 data collection (7B) In Progress • Data analysis, algorithm development (7B) Sept-Dec. ’03

19. Task 8: Intent • Objectives • Identify a list of Intents that are measurable and potentially useful for distraction mitigation and safety warning countermeasures. • Determine diagnostic measures for reliable detection of those intents. • Major Findings • Information about driver intent can improve system effectiveness and acceptance and reduce nuisance alerts. For example, FCW warnings may be delayed/suppressed if drivers intend to brake, and blind spot warnings may be issued when drivers intend to change lane. • Intent detection variables can be classified into affordance (e.g., exit ramp), motive (e.g., navigation info), kinematics (e.g., raw, heading), control (e.g., turn signal), and eye glances. • Current Status(50% completed) • Literature review report (to Delphi) (8A) Completed • Framework for intent determination (8B) Completed • Acquisition and preliminary examination of ACAS FOT pilot data (8B) Completed • Naturalistic Lane Change Data • Markov matrix analysis of eye movements (8B) Completed • Analysis of kinematic & control variables (8B) In progress

20. Task 9: Safety Warning Countermeasures • Objectives • Improve the effectiveness and acceptability of safety warning systems by designing these systems to adaptively respond to intent, distraction, and demand information. • Major Findings • Although adaptive systems can fail because of poor user acceptance (e.g., not understanding the system, perceived system inconsistency or unpredictability, drivers feeling no longer in control) and poor design (e.g., oscillations), they have shown some promise and can be acceptable to drivers. • Excessive nuisance alerts pose major problems for FCW and other warning systems. Nuisance alerts may be reduced by adjusting warning criteria using driver state information (distraction and intent). • FCW warnings and lane drift warnings may be delayed/suppressed if drivers are attentive or intending to engage in particular maneuvers. • Blind spot warnings may be issued when drivers intend to change lane. • Current Status (45% completed) • Literature review report (to Delphi) (9A) Completed • Identification and definition of adaptive enhancement issues (9B) Completed • Selection of FCW algorithm and DVI (9B) Completed • Preliminary design of experiments (9B) Completed • 2 simulator experiments (9B) Sept.-Dec. ‘03

21. Common Issues and Solutions • Regular team meetings convened to • Discuss common issues and solutions. • Commonize design and variables across experiments as much as possible. • Dependent Variables • Four common dependent variables used across all experiments • Reaction time for initial brake application • Reaction time for foot off accelerator • Steering entropy • Steering reaction time and direction • Many other variables (e.g., SDLP, lane departures, head and eye movements) will be used, although they may vary in different experiments. • Subject Ages • Subject ages divided into 3 age groups: 18-25; 35-55; and 65-75 years old. • All experiments must include the group of 35-55 years old. • Due to time and budgetary constraints, there is no requirement to use the younger (18-25 years old) or older subjects (65-75 years old). If multiple age groups are used, however, the additional group(s) may be 18-25 years old, or 65-75 years old, or both. • The choice of subject ages is made to balance the age effect and age representation (e.g., inclusion of subjects of 45 years of age or older). 35-55 years old are also the likely initial adopters of SAVE-IT technologies.

22. Common Issues and Solutions • All simulator experiments (at Delphi, Iowa, and UMTRI) use GlobalSim Simulator. • All visual glance behaviors measured with Seeing Machines eye tracking system. • If applicable, the driving scenario is the “lead vehicle following” scenario. • The same vehicle (white passenger car) used as the lead vehicle in all experiments. • For all experiments, a “rubber-band” control (Lee et al., 2002) is used to set time headway at 1.8 s at the moment of lead vehicle braking. • Except for Task 9, lead vehicle braking is non-imminent (e.g., -0.2 g). • Lead vehicle braking is unpredictable and infrequent (e.g., less than 1 per minute). • If practical, both rural roads and freeways are used. • Target speed = 45 mph for rural roads and 65 mph for freeways. • Some experiments use one type of roads only because the use of both types of roads may result in excessive long sessions or the use of too many subjects. • Per Volpe’s request, the Hidden Markov Model will be considered as a method in Tasks 3 (performance), 5 (cognitive distraction), and 8 (intent). • The minimum number of subjects per condition is 8. Many more subjects are used in many experiments.