1 / 14

Storied Navigation

Storied Navigation. Edward Shen Hector Beltran Donyun Kim 12/15/06. Story Analogy. Idea. Build system that allows users to tell different stories from sequences of video clips with meaningful relationships between clips (using analogies to other stories). Original System Design.

wainwright
Download Presentation

Storied Navigation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Storied Navigation Edward Shen Hector Beltran Donyun Kim 12/15/06

  2. Story Analogy

  3. Idea • Build system that allows users to tell different stories from sequences of video clips with meaningful relationships between clips (using analogies to other stories)

  4. Original System Design What’s the story about? Presidential election Video Corpus • How do you feel about him? • What has he done in the past? • Who were the candidates? • When was the election? • What was the outcome of the election? • What were the issues?

  5. Challenges for Q/A scenario • How to derive context-relative questions based on provided information • How to derive plausible annotations for a video sequence based on the context of using it • How to design representation for video sequence such that the above can be conducted more easily

  6. Generating Questions • Assume video sequences annotated with Roger Schank’s form of indexing stories (theme,goal,plan,result,lesson) • Questions come from: • Generic question templates • Common sense reasoning • Story analogies

  7. Curiosity • Ask questions people tend to be more curious about • Introduce new pieces of information that produce effect of new direction of navigation

  8. Curiosity • Relationship between people • Why do people do what they do? • External forces • Values/Beliefs • Internal desire

  9. New System Design • Different users • Users that know video corpus very well • Users that are seeing videos for first time • Help new users create stories by using information first set of users have already entered (system developers)

  10. New system design System Developer Q7 Q7 Q4 3 3 Q1 5 5 5 5 Q8 Q8 Q5 2 1 Q2 Q9 Q9 Q3 Q5 Q6 Using semantic similarity and common sense 4 4 4 4 End User Why is Mary sad? 2 View suggested questions

  11. System Developer 2. Sentence/Sentences attached to clips 1. System Developer types in story Gustave is one of the dancers. He likes to suggest ideas for the dance. He gets very frustrated that he can only do what he is told. 1 2 3 3. Question set developed for each sequence 4. Generate Q/A sets between video sequences Q1 1 Q2 Q3 Q1 2 1 Q2 Q1 2 Q2 Q3 Q4 Q3 Q1 3 Q2 4 Q3

  12. Steps 1 – Segment Stories Gustave is one of the dancers. He likes to suggest ideas for the dance. 3/5 He gets very frustrated that he is only allowed to do what he is told. 4/5 At one point he was crying. This happened during the first week. 4/5 The second week was different. 3/5 Dancers were encouraged to design their own dance. 4/5 Gustave had a wonderful time. 1. Always include topic sentence 2. Try topic sentence with new sentence, if sentence does not return 4/5 same clips as previous sentence/segment, start new segment. Otherwise, add to previous sentence/segment

  13. Step 2 – Assign clips Gustave is one of the dancers. 1 He likes to suggest ideas for the dance. Gustave is one of the dancers. 2 He gets very frustrated that he is only allowed to do what he is told. At one point he was crying. This happened during the first week. The second week was different. Gustave is one of the dancers. 3 Dancers were encouraged to design their own dance. Gustave had a wonderful time.

  14. Step 3 – Question Generation Gustave is one of the dancers. 1 • General templates What,when,how,why He likes to suggest ideas for the dance. How does Gustave dance? What are Gustave’s ideas? • Emotions – why…? Ever feel the opposite? Common Sense 3. Conflict – ask for resolution 2 Gustave is one of the dancers. He gets very frustrated that he is only allowed to do what he is told. At one point he was crying. This happened during the first week. The second week was different. Why was Gustave frustrated? Is Gustave ever happy? 3 How was the conflict resolved? Gustave is one of the dancers. Dancers were encouraged to design their own dance. Gustave had a wonderful time. Why did Gustave have a wonderful time?

More Related