1 / 23

The Entertainment Industry Meets Simulation: The Challenge of the Holodeck

The Entertainment Industry Meets Simulation: The Challenge of the Holodeck. Bill Swartout Institute for Creative Technologies University of Southern California. The ICT is intended to serve as a nexus between the simulation and entertainment communities. Institute For Creative Technologies.

paul2
Download Presentation

The Entertainment Industry Meets Simulation: The Challenge of the Holodeck

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Entertainment Industry Meets Simulation:The Challenge of the Holodeck Bill Swartout Institute for Creative TechnologiesUniversity of Southern California

  2. The ICT is intended to serve as a nexus between the simulation and entertainment communities

  3. Institute For Creative Technologies Tech Show Biz

  4. How can we best simulate reality? • Two approaches • Military simulation • Entertainment industry

  5. Simulating Reality:Two Approaches Military Simulation • Literal: try to include as much reality in simulation as possible • Homogeneous Entertainment Industry • Goal-directed: create effect of real experience by selectively focusing on elements (real and unreal) • Hybrid Approach: combine disparate elements

  6. Movie clip

  7. To the Holodeck and beyond…

  8. Mission Rehearsal Exercise Project: New Operations

  9. Mission Rehearsal Exercise Project: ICT VR Theater

  10. What we don’t want… Movie clip

  11. New Research Areas • Speech Recognition • Natural Language Understanding • Dialogue Management • Natural Language Generation • Speech Synthesis • Gesture Generation • Perception • Personality Variation • Sound Spatialization • Interactive Story Structure

  12. Movie clip

  13. Integration Raises New Challenges: Natural LanguageProject Leaders: Ed Hovy (ISI), Kevin Knight (ISI), David Traum (ISI), Shri Narayanan (IMSC) • Speech recognition: Noisy environment • Multimodal: Face To Face (speech+gesture), Radio • Multiple Interactors (Sgt, medic, squads,…) • Messages tailored for multiple addressees/overhearers • Emotion

  14. “Squad Leaders listen up!” Challenge: Speech SynthesisProject Leaders: Lewis Johnson (ISI), Shri Narayanan (IMSC) • Most speech synthesis sounds unnatural • Even the best have limited expression (AT&T NextGen) • No command voice • Use hybrid approach: • Multiple synthesizers withdifferent “voices” • Command voice Text toSpeechSynthesis • Use pre-recorded library for highly emotional speech

  15. Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI)

  16. Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI) Movie clip

  17. Challenge: Modeling EmotionsProject Leaders: Jon Gratch (ICT), Stacy Marsella (ISI) • Problem addressed: • Virtual humans need emotions to be believable & realistic • Approach: • Characters have beliefs, goals, utilities • Construct plans • Events evaluated against plans • If goals thwarted (or enabled) characters react

  18. Emotional Reaction Movie clip

  19. Modeling Emotions: Issues • Integrating gesture, natural language & voice to project realistic behavior • Managing unintended interpretations • Hollywood vs. Psychology • Movie emotions quite different from natural emotions • Recognizing human emotions

  20. Challenge:Story is Critical • More than an event list • Successful stories: • Juxtapose contrasting elements • Surprise (plot twists) • Dilemmas • Things go wrong • Build to climax (and ebb)

  21. Structuring Stories for Intelligent Agents: Issues • Traditional story telling tricks require linear structure • Immersed participant needs freewill • Need “bumpers” • Possibility: director agent controls environment & characters to limit choice

  22. Reasons for Optimism • Hybrid approach • Use all the tools in the toolbox • Entertainment industry as source of knowledge and techniques • Story context constrains possibilities • Synergy from graphics, sound & emotional engagement • The mind is a willing collaborator

More Related