A Study of Dialogue Management Principles Corresponding to the Driver's Workload

1. T. Nishimoto (Univ. Tokyo), M. Shioya (Hitachi, Ltd.)J. Takahashi, H. Daigo (JARI) A Study of Dialogue Management PrinciplesCorresponding to the Driver's Workload Mind-free concept (reducing mind distraction) is important in voice telematics systems. An experiment concerning a network-distributed voice-activated telematics system was conducted. Reference model, basic principles and requirements are also discussed. Standardizations of dialogue control, voice command and effect sound are effective. A standard could be developed by complementing the dialogue principles of ISO 15005.

2. Background and Objective Autonomous or Dedicated Telematics Car Navigation Systems ( Access Closed-Specced Information Services ) To meet ‘ Visual Distraction ’ - hands-free ( hands-on-the-wheel ), - eyes-free (eyes-on-the-road ) Network-Distributed Telematics Systems ( Access Open-Specced Information Services ) To meet ‘ Mind Distraction ’ - mind-free ( mind-on-the-driving ) HMI Reference Model for Telematics Systems - Priority Classes for Information - Prohibit to Provide Info. Not for Driving - Situation-Aware Control - Introduce HMI Classes Network-Distributed Telematics System to meet Safety & Situation ( Priority, Emergency, Preference )

3. Network-Distributed Voice-Activated Telematics Service System ASP: Application Service Provider Web Server ASP Reservation Center Web Server Restaurant Info. Center Web Server Voice Engine Voice Engine Select Voice Engine Depending on Request Internet Web Server ASP Voice / Recognized Data URL / Voice Weather Info. Center Web Server In-Vehicle System Voice Browser Vehicle Info. Voice Engine Voice Engine Voice Engine Voice Location Info. Web Server Voice-Activated Center System Text-To-Speech Data as Reply to Request

4. Voice Telematics HMI System Architecture Driver’s Conditions Driving & Environmental Conditions Networks Driver Communications Vehicle Center System In-Vehicle System SR HMI Server - Sensing - Control - Share HMI Server - Distraction Sensing - Dialogue Control considering Info. Priority & Driver’s Skill - Load Share btw Center / In-Vehicle Request Request Event-driven Multimodal Interface (SALT-like) SR Voice/data VoIP <=> Voice VoIP<=>Voice TTS Contents Info. Tags Contents VoiceXML Documents / Interpreter TTS VoiceXML Documents / Interpreter

5. An Example of Dialogue Control for Driver Driver B’s Skill Driver A’s Skill Mixing Load Passing Right Turn pause dialogue Left Turn pause dialogue Driving Ability Remained for B Driving Ability Remained for A Telematics Load Constant Cruise : Driving Load + Telematics Load : Driving Load TIME

6. Dialogue Control in Case of Right Turn Resume Dialogue at Winker OFF Vehicle Right Turn May Resume Dialogue by Command when Signal is Red and Vehicle Speed = 0 Pause Dialogue at Winker ON

7. Evaluations of the Test (1) Control of Dialogues: - Most drivers agree that it feels much safe due to the control of dialogues. - Standardization of dialogues is also requested. (2) Pause of Dialogues: - Most drivers agree the necessity of the standardization of interruption and resumption method of dialogues. (3) Voice Commands: - Most drivers agree the necessity of preparing standard commands. (4) Effect Sounds and Announcements: - Most drivers agree the necessity and the effectiveness of the effect sounds and the announcements. - The standardization is also requested.

8. View Points of the Reference Model Scope - Dialogues between a driver and telematics service systemswhile a vehicle is in motion. - Use mainly voices as inputs by the driver, recognized mainly by voice recognition techniques. - Use mainly voices or sounds as outputs by the systems. - Treat services not only traffic information, POI information, news, and weather, but also various services, popular in the Internet world ( e.g. e-mail, electronic commerce, information retrieval, and entertainment ). - The reference model aims to present basic principles, requirements and recommendations, for accessing the telematics services without mind distraction.

9. Basic Principle of the Reference Model - In the dialogues for the telematics services, the important basic principle is that the skill for processing the driving load is always secured by the driver. - That is, the driver should keep his 'mind-on-the-driving' until the driving load processing is completed, namely, should let his 'mind-free' from others but the driving. - The dialogues for the telematics services should be executed within the driver’s remained skill besides for the driving load processing.

10. Basic Requirements of the Reference Model (1) - The dialogues for the telematics services are to be managed or controlled so that the skill for processing the driving load is secured first. - Then, within the driver’s remained skill the telematics load is processed. - The driving load and the telematics load are to be pre-estimated or measured real-time. - The driver’s skill or capability is to be pre-estimated or measured real-time.

11. Basic Requirements of the Reference Model (2) - The management or control is to be done considering driver’s condition and driving situations ( e.g. road, traffic, vehicle and travel conditions ) of that time. - The management or control is to be done considering the priorities of the services. - The telematics service systems' state is to be informed to the driver properly. - The final initiative of the dialogues is to be taken by the driver.

12. Typical HMI Related ISO Standards ISO 14813-1 ITS (TICS) Fundamental Services: 32 Services in 8 Categories 16951 Priority of On-Board Messages: 2 Methods for Determining Priority of ITS and Non-ITS Messages ISO 15005Dialogue Management : Ergonomic Design Principles for Dialogues between Driver and ITS Systems Considering Driver’s Workload for Safe Vehicle Control 15007 Measurement of Driver’s Visual Behaviour: Key Terms and Parameters for Analysis 15006 In-Vehicle Auditory Presentation: Specifications for Presentation through Speech or Sounds to Meet Auditory or Mental Overload 15008 In-Vehicle Visual Presentation: Specifications for Image Quality of Displays for ITS Systems used while Vehicle is in Motion 17287 Suitability Assessment: Procedure for Assessing whether Specific ITS Systems are Suitable for Use by Drivers while Driving

13. Complementing the ISO Standards

14. A citation from ISO 15005: 5.2.4.3.4 Recommendation TICS should provide timely visual information to the driver. An example of a supplemental recommendation: 5.2.4.3.4-1 Recommendation Voice-activated telematics systems will provide the driver with timely audible information. Example 1: In situations where the driving workload increases such as when turning right or left at an intersection, the driver will be advised sufficiently in advance of the maneuver so that the voice-based telematics service dialogue can be performed safely. Figure [Dialogue Control in Case of Right Turn] shows an example of dialogue control during the execution of a right turn, which is one situation involving a heavy driving workload. The dialogue is interrupted and resumed based on the On/Off status of the turn signal lever, respectively. "Interrupt" and "resume" commands are also provided so as to give precedence to the driver's judgment.

15. Future Works - Consensus for - Reference model of voice-activated telematcs systems - Workload measurement methods (Visual + Auditory) - Evaluations of proposing requirements/recommendations - Pause of dialog (driving situation & drivers' skill) - Standardization of voice commands - Standardization of effect sounds / announements Contacts - Takuya Nishimotonishi@hil.t.u-tokyo.ac.jphttp://hil.t.u-tokyo.ac.jp/~nishi/