1 / 9

Automatic generation of multimodal interaction models from behavioural data

Marie-Luce Bourguet Computer Science Dept. Queen Mary, University of London. Automatic generation of multimodal interaction models from behavioural data. Multimodal Interaction Models. Interaction Model. Application. Multimodal Engine. …. Speech recognition. Gesture recognition.

adele-harding
Download Presentation

Automatic generation of multimodal interaction models from behavioural data

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Marie-Luce Bourguet Computer Science Dept. Queen Mary, University of London Automatic generation of multimodal interaction models from behavioural data

  2. Multimodal Interaction Models Interaction Model Application Multimodal Engine … Speech recognition Gesture recognition

  3. pen move pen move pen down pen up gesture delete reset speech remove Modeling Multimodal Interaction “remove”

  4. pen move pen down pen up pen move gesture delete pen move pen move reset pen down speech remove speech remove pen move pen move pen up pen move pen down reset speech remove pen up pen move gesture delete gesture delete reset Modeling Multimodal Interaction: Synchronisation patterns “remove”

  5. pen move pen down pen up pen move gesture delete reset speech remove pen move pen down speech move pen up pen move reset Modeling Multimodal Interaction: Mutual disambiguation

  6. Design Tool

  7. Simulation Tool

  8. Automatic Generation of Interaction Models Interaction Model Training Module … Speech recognition Gesture recognition

  9. Conclusion Building interaction models from user interaction data presents numerous advantages: • It bypasses the difficult task of designing multimodal interaction models • The resulting interaction models are highly personalised • They should enable natural and robust interaction The FSM based formalism is well adapted because very simple analysis of the data should permit the automatic generation of interaction models.

More Related