580 likes | 694 Views
Story and Discourse Using Planning and Natural Language Models to Create Engaging and Interactive Stories in Computer Games. R. Michael Young Liquid Narrative Group Department of Computer Science North Carolina State University. liquid narrative project goals.
E N D
Story and DiscourseUsing Planning and Natural Language Models to Create Engaging and Interactive Stories in Computer Games R. Michael Young Liquid Narrative Group Department of Computer Science North Carolina State University
liquid narrative project goals • Increase our understanding of the comprehension process involved in human interaction within virtual worlds • Incorporate this understanding in systems that automatically create and manage users’ activities inside these worlds • Exploit the undertstanding to create novel and effective experiences for applications ranging from entertainment to education, training and others
specific focus • Develop computational models of interaction within narrative-oriented virtual worlds, using theories from • Cognitive psychology • Narrative theory • Computational linguistics • Artificial Intelligence
cognitive and computational models • Systems that • model the representation and reasoning methods of their users • target desired application-specific cognitive states of their users • act to move their users into those target states
narrative model The story world
narrative model The story world
narrative model The story The story world
narrative model The story The story world
narrative model The narrative discourse The story The story world
narrative model The narrative discourse The story The story world
narrative model The narrative discourse The story The story world
narrative model The narrative discourse The story The story world
narrative production The narrative discourse The story The story world
narrative production The narrative discourse The story The story world
narrative production The narrative discourse The story The story world
narrative production The narrative discourse The story The story world
narrative production The narrative discourse The story The story world
generating story world actions The narrative discourse The story The story world
creating parts of the story world • Operate from a given story world state with fixed story goals • Automatically select characters, create their plans, conflicts and all activity within the story world • Exploit algorithms for automatic plan generation from artificial intelligence
ai planning algorithms • Used to create action sequences for real-world settings • Take as input • Library of available actions • Description of current state • Set of goals to be achieved • Search for all possible action sequences that will achieve the goals when carried out from the initial state • Applicable to contexts where pre-scripting or hard-coding actions is infeasible • Young, R. Michael, Martha E. Pollack and Johanna D. Moore, Decomposition and causality in partial-order planning, in Proceedings of the Second International Conference on AI and Planning Systems, Chicago, IL, pages 188-193, July, 1994.
ai plans • Planning algorithms produce complex data structures representing actions and their constraints
ai plans • Planning algorithms produce complex data structures representing actions and their constraints Unlock(Door3,key1)
ai plans • Planning algorithms produce complex data structures representing actions and their constraints Unlock(Door3,key1) Grab(key1,desk) Open(Door3) Move-To(Door3)
ai plans • Planning algorithms produce complex data structures representing actions and their constraints Unlock(Door3,key1) Grab(key1,desk) {Grab < Move-to, Grab < Open Grab < Unlock, Move-to < Unlock, Move-to < Open Unlock < Open} {Grab < Move-to, Grab < Open Grab < Unlock, Move-to < Unlock, Move-to < Open Unlock < Open} Open(Door3) Move-To(Door3)
Unlock(Door3,key1) Grab(key1,desk) Open(Door3) Move-To(Door3) ai plans • Planning algorithms produce complex data structures representing actions and their constraints holding(key1) Unlock(Door3,key1) Grab(key1,desk) unlocked(door3) at(desk) {Grab < Move-to, Grab < Open Grab < Unlock, Move-to < Unlock, Move-to < Open Unlock < Open} {Grab < Move-to, Grab < Open Grab < Unlock, Move-to < Unlock, Move-to < Open Unlock < Open} at(door3) Open(Door3) Move-To(Door3) at(door3)
ai plans • Planning algorithms produce complex data structures representing actions and their constraints
ai plans • Planning algorithms produce complex data structures representing actions and their constraints
ai plans • Planning algorithms produce complex data structures representing actions and their constraints
ai plans • Planning algorithms produce complex data structures representing actions and their constraints
plan data structures and cognitive models • Plan structures are • provably sound means used to control a virtual world • produced by processes that mirror those of human planning suggested by experimental results • Hayes-Roth, B. and F. Hayes-Roth, A Cognitive Model of Planning. Cognitive Science, 1979. • Rattermann, M. J., et al, Partial and total-order planning: evidence from normal and prefrontally damaged populations, in Cognitive Science, 2002.
plan data structures and cognitive models • Plan structures are • similar to many of the constructs employed by psychological models • indicated in preliminary experiments to be accurate predictors of models used by human subjects • Graesser, A.C., et al, Question answering in the context of stories. Journal of Experimental Psychology: General, 1991. • Christian, D. and Young, R. M., Comparing Cognitive and Computational of Narrative Structure, Technical Report03-001, Computer Science Department, NC State University, 2003
selecting actions to include in the story The narrative discourse The story The story world
selecting actions to include in the story The narrative discourse The story The story world
moving from story world to story • Plan structures contain a high degree of detail • Story structures, while complex, leave out much of the essential action in a plot • Problem: How can we determine what action from the story world plan to include in the story itself?
moving from story world to story • Cooperative plan identification (CPI) originally used to create effective instructional texts from complex task plans Generate Candidate Subset Simulate Comprehension Process Create Task Plan • Young, R. Michael.Using Grice's Maxim Of Quantity To Select The Content Of Plan Descriptions. Artificial Intelligence, no. 115, 215-256, 1999.
generating candidate plan subsets • Sequence of candidate plans determines the type of descriptions that will be produced • Use heuristics drawn from work on narrative comprehension • Relate causal links to causal centrality • Create sequence of candidates by successively removing elements that are least central • Relationship to narrative comprehension suggests the technique’s applicability in this context
validating CPI techniques • Task efficacy evaluation • Use the plan subsets to create instructional texts • Have human subjects carry out tasks based on those written instructions • Use measure of specific plan-based errors as indication of technique’s effectiveness • Young, R. Michael.Cooperative Plan Identification: Constructing Concise and Effective Plan Descriptions. Proceedings of the National Conference of the American Association for Artificial Intelligence, pages 597-604. Orlando, FL, 1999.
generating narrative discourse The narrative discourse The story The story world
generating narrative discourse The narrative discourse The story The story world
moving from story to discourse • Once we’ve created a story’s plot • the plan can be used to drive the action within a 3D virtual world • how can we use resources within the medium to effectively communicate the story as it unfolds? • The 3D camera • Background Music • Narration and voiceover • Looking just at camera control: • Geometric constraint-solving • Shot composition • Bares, William and Lester, James. Intelligent Multi-Shot Visualization Interfaces for Dynamic 3D Worlds. Proceedings of the International Conference on Intelligent User Interfaces, 1999
camera control and its relationship to textual discourse generation • Discourse generation involves the determination of the content of a discourse and it’s organization • Most approaches use a speech act model in which communication is viewed as planned intentional action • Higher level discourse planning creates the rhetorical organization of a text • Lower level processing translates speech acts into text taking into account the constraints imposed by a grammar and a lexicon • Young, R. Michael, Johanna D. Moore and Martha E. Pollack, Towards a principled representation of discourse plans. In Proceedings of the Sixteenth Annual Meeting of the Cognitive Science Society, 1994.
camera control and its relationship to textual discourse generation • Shot composition involves the specification the shots in a sequence and the sequence’s organization • This approach uses a speech act model in which the use of the camera is viewed as planned intentional action • Higher level planning creates the rhetorical organization of a shot sequence • Lower level processing translates primitive camera acts into shot specifications, taking into account the constraints imposed by the scene geometry, an inclusion/exclusion list and a collection of shot types
generating cinematic 3D camera control • Use discourse planning techniques to create camera plans • Camera plans capture • Film idioms identified by cinematographers • Effects of individual shots on the mental state of the viewer • Camera planning • Determining shots and shot sequences • the temporal organization of the shots relative to the actions that they’re filming • Dan Amerson and Shaun Kime, Real-Time Cinematic Camera Control for Interactive Narratives in The Working Notes of the AAAI Spring Symposium on Artificial Intelligence and Interactive Entertainment, 2001.
Virtual Environment the mimesis system architecture Story Planner Discourse Planner Execution Manager Additional Components Natural Language Generator Database
mimesis worlds Monterey Bay Aquarium Virtual Tour
mimesis worlds • A visitor’s route is dynamically planned based on • Her expressed preferences • The information resources available at various locations • Character behavior is integrated into this plan to demonstrate key concepts • Camera control is planned so that the right things are observed at the right time Monterey Bay Aquarium Virtual Tour
mimesis worlds Monterey Bay Aquarium Virtual Tour