1 / 11

By Eunice Ma Supervisor: Prof. Paul Mc Kevitt School of Computing and Intelligent Systems

Intelligent MultiMedia Storytelling System (IMSS) - Automatic Generation of Animation From Natural Language Input. By Eunice Ma Supervisor: Prof. Paul Mc Kevitt School of Computing and Intelligent Systems Faculty of Informatics University of Ulster, Magee. Project Goals.

grant
Download Presentation

By Eunice Ma Supervisor: Prof. Paul Mc Kevitt School of Computing and Intelligent Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Intelligent MultiMedia Storytelling System (IMSS)- Automatic Generation of AnimationFrom Natural Language Input By Eunice Ma Supervisor: Prof. Paul Mc Kevitt School of Computing and Intelligent Systems Faculty of Informatics University of Ulster, Magee

  2. Project Goals • To interpret natural language story and movie (drama) script input and to extract concepts from the input • To generate 3D animation and virtual world automatically, with speech and non-speech audio • To integrate the above components to form an intelligent multimedia storytelling system for presenting multimodal stories

  3. Previous Research • Schank’s Conceptual Dependency (CD) Theory • Primitives • SAM • Automatic Text-to-Graphics Systems • WIP/COMET/APT • WordsEye • ‘Micons’ and CD-based language animation • O Nuallain and Smith’s Spoken Image (SI) system

  4. Previous Research (continued) • Multimedia Storytelling • Larsen’s Interactive Storytelling • AesopWorld • Multimedia Interactive Systems • AIMI • AlFresco • KidsRoom • Animated Intelligent Agents • Cassell’s REA and SAM

  5. Software Analysis • Natural language processing tools • Gate • CPK NLP Suite • WordNet • 3D Animation Tools • Poser 4.0 • 3D Studio Max • VRML 2.0/97 • Microsoft DirectX • The Actors – Using Conversational Agents • COLLAGEN • Microsoft Agent • BEAT

  6. Context Diagram Story in natural language Speech (dialogue) Movie/drama script animation IMSS Tailored menu for script input non-speech audio I/O Media of IMSS

  7. General Architecture Natural language story Script writer Script parser Prefabricated objects Basic vocabulary Natural Language Parser Text To Speech Sound effects Concept database 3D authoring tools mapping Animation generation Primitives library Code combination 3D world with audio

  8. Natural language parser Data Flow Diagram Primitives library Animation director labelled script VRML without sound nodes Scene&Actor descriptions Media coordination Synthesized animation TTS dialogues Script parser script Non-speech audio Sound effect driver script Script writer story Music library

  9. DFD of Animation Director Primitives library Actor managers marked script Script distributor World builder VRML without sound nodes Scene generator VRML codes Props provider coordinator

  10. Project Schedule

  11. Conclusions • The objectives of IMSS meet the challenging problems in language animation. • Mapping language primitives with visual primitives • Sophisticated spatial relations between spatial cognition and prepositions in English • Visualization requires a gigantic ‘common senses’ knowledge base • Representing stories by temporal multimedia requires high coordination • Prospective practical applications • Children education • Multimedia presentation • Movie/drama production • Script writing • Computer games, Virtual Reality

More Related