Dialogue systems in cars face two major challenges Speech recognition errors

1. The Effect of Cognitive Load on a Statistical Dialogue System Milica Gašić, PirrosTsiakoulis, Matthew Henderson, Blaise Thomson, Kai Yu, Eli Tzirkel* and Steve Young Cambridge University Engineering Department, *General Motors Driving Perfomance Dialogue as a Secondary Task • We measured differences in speed and related statistics per subject • We examined which is larger for Talking&Driving: • Dialogue systems in cars face two major challenges • Speech recognition errors • Increased cognitive load on the user • Statistical dialogue modelling deals with speech recognition errors • Substantial research concerns safety while talking to a dialogue system in a car • We examine how humans speak when under cognitive load • We find dis-fluencies in communication and preference towards certain system questions • Driving is more erratic when the subjects talk to the system at the same time Dialogue Performance • When talking subjects were given specific dialogue tasks to complete • We measured both the objective task completion and the perceived (subjective) task completion Experimental Set-up • Bayesian Update of Dialogue State dialogue manager provides robustness to speech recognition errors: • It models dialogue via a Bayesian network with hidden concepts • It maintains a distribution over the hidden concepts • Domain: TopTable restaurant domain for Cambridge (150 venues, 8 slots) • Car Simulator: seat, steering weal, pedals and large projector • 30 subjects drove along a motorway in three scenarios • Driving for 10 minutes (without talking) • Talking to the system for 7 dialogues • Talking&driving at the same time (7 dialogues) • Although not statistically significant, the performance is worse when driving at the same time. Conversational Patterns User obedience to system’s questions: tem • Users prefer confirmations to request when they are driving Analysis of measures related to speaking which increase for Talking&Driving compared to Talking: Results Cognitive Load • Cognitively loaded user speech is more dis-fluent and louder • Subjects were able to notice differences in cognitive load: Conclusions • Dialogues with cognitively loaded users tend to be less successful • Cognitively loaded users tend to answer some system questions more than others • Users tend to use barge-ins and filler significantly more often when cognitively loaded • Incremental dialogue and adaptation techniques are needed to better model dialogue as a secondary task Acknowledgements We would like to thank Prof. Peter Robinson and Ian Davies for their help with the simulated car experiments.