1 / 42

Workshop in Media Authoring Automatic Generation of Animation and Video

Workshop in Media Authoring Automatic Generation of Animation and Video. Contact Doron Friedman doronf@math.tau.ac.il Mobile: 054-461807. Agenda. Introduction Motivation and Background Course Material Logistics and Administration Description of Projects. Motivation – Media Authoring.

gauri
Download Presentation

Workshop in Media Authoring Automatic Generation of Animation and Video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workshop in Media Authoring Automatic Generation of Animation and Video Contact Doron Friedman doronf@math.tau.ac.il Mobile: 054-461807 Intelligent Tools for Media

  2. Agenda • Introduction • Motivation and Background • Course Material • Logistics and Administration • Description of Projects Intelligent Tools for Media

  3. Motivation – Media Authoring • 3D: • Professional animators – several years of study • Professional software • Low leve of abstaction • Very little reuse • Video • Professional editors • Professional software • Pixel based (rather than object based) Intelligent Tools for Media

  4. Smart Graphics • What this is NOT about • How to achieve efficient 3D rendering • What this IS about • How to quickly generate animation from high level input Intelligent Tools for Media

  5. Making movies (Simplified) Story/idea Script Shooting Script Animation File • Story- understanding • Storytelling • Directing • NLU • Reusable animation • Animation composition • Text-to-speech • Theory of space and time • Camera-control principles • Model of actions Intelligent Tools for Media

  6. Script -> Animation Formal Script Shooting Script Animation File • Reusable animation • Animation composition • Text-to-speech • Theory of space and time • Camera-control principles • Model of actions Intelligent Tools for Media

  7. Interactive Virtual Environments • Linear Animation • Authoring • Interactive Virtual Environments • Authoring • Agent • Real-time Director Intelligent Tools for Media

  8. Production Industry Rapid prototyping Storyboarding Gaming Industry Engine for interactive games A movie-maker game Toys (Lego/AIBO) Education Training Simulations Industrial Military Traffic Scientific Medical Applications Intelligent Tools for Media

  9. State of the trade today • Leading tools (Maya, RenderMan, ..) • Leading standards/trends (VRML, Java3D, …) • How authors work today Intelligent Tools for Media

  10. SMIL-1 - • Layout • Relative/absolute positioning • Z-index • Fit (meet, slice, scroll) • Time • Par • Seq • begin Intelligent Tools for Media

  11. SMIL-2 • 45 Modules / Language Profiles • Integration with other languages • Vector graphics • Excl (exclusive par) • Relative synchronization • Time manipulation (speed, reverse) • Animation • AudioLayout • Transitions Intelligent Tools for Media

  12. MPEG-4 as a Rich Media Representation Scheme • Several goals • Profile Approach Intelligent Tools for Media

  13. Mpeg-4 Relevant Features • TermCap node: • Frame rate • Memory/CPU load • BIFS switch node Intelligent Tools for Media

  14. MPEG-7 – the Bits about the Bits • D – Descriptors – representation of features (syntax + semantics) • DS – Description Schemes – structure and semantics of relationships between components • DDL – Description Defintion Language – to create new DS, and perhaps new D Intelligent Tools for Media

  15. Example – Multiple Description Types • Structural – right, left, moving towards • Physical – color, shape, motion • Conceptual – goalkeeper, player, ball Intelligent Tools for Media

  16. MPEG-21 • Define a multimedia framework to enable transparent and augmented use of multimedia resources across a wide range of networks and devices • 7 Items • Digital Item Declaration • Digital Item Identification and Description • Content Handling and Usage • Intellectual Property Management and Protection • Terminals and Networks • Content Representation • Event Reporting Intelligent Tools for Media

  17. Course Material • Background: OO Design and Programming (C++/Java, tbd) • 3D technologies: • VRML • HANIM • Java3D ? • MPEG-4/BIFS/XMT • Video/Mixed Technologies • MPEG-4 • JMF • SMIL-1/SMIL-2 Intelligent Tools for Media

  18. Course Logistics • Communication channels • Groups • Development methodology • Dates Intelligent Tools for Media

  19. Communication Channels • Mailing list: tba • Web-site: http://www.math.tau.ac.il/~doronf/workshop_2002.html Includes links to resources • My e-mail: doronf@post.tau.ac.il Intelligent Tools for Media

  20. Groups • 1-4 people in group (recommended: 2) • Group gets one grade • Inter-group integration – big bonus! Intelligent Tools for Media

  21. Development Methodology • Some thoughts on Software Engineering: • Writing programs is easy, writing software is impossible • % of code used • Legacy software Intelligent Tools for Media

  22. Development Process (waterfall) • MRD (Marketing Requirements Document) • HLD (High Level Design) • FRS (Functional Requirement Specification) • ATP (Acceptance Testing Procedures) • Code • Debug • Testing / Q.A. Intelligent Tools for Media

  23. Projects • Animation Library • Automatic Summaries • Video Template Authoring • Object-based Video • Film & TV School projects Intelligent Tools for Media

  24. Animation Library • Objects with animation • Humanoid • Car • Door • 4-legged animal • Elevator • Bottle, glass and drink • Cigarette and smoke • Weapons • … Intelligent Tools for Media

  25. Stand Walk (from, to), (path) Run Jump Sit (on) take (object) Put (object, location) Touch (object/organ) Wave Talk Eat/drink Climb (wall, stairs,..) Animation Library - Humanoid Intelligent Tools for Media

  26. Animation Library - Car • Drive (from, to), (path) • Open (door) • Close (door) • Traffic simulation Intelligent Tools for Media

  27. Animation Library - Speech • Text-to-speech • Lip animation (sync?) Intelligent Tools for Media

  28. Animation Library – Facial Expressions • Basic emotions (smile, angry, afraid) • Low level animation • Eyes (look-at, close, open, blink) • Eye-brows • Mouth • Face features (hair color, hair style, eye color, glasses) Intelligent Tools for Media

  29. Animation Library - Other • Door (open, close) • 4-legged (share animation with humanoid) • Elevator (open, close, go-to-floor) • Bottle, glass, drink (pour, person drinking, spill) • Gun (shoot, bullet/missile) • … Intelligent Tools for Media

  30. Animation Composition • Sequence (states) • Parallel • (walk, run, Jump) x (hold, wave) • Compound Animations (hierarchy) • E.g: eat = take (food), touch(mouth) • Inverse Kinematics Intelligent Tools for Media

  31. Smart Objects • Different Complexity Levels of Behavior • Default behaviors • Rules and constraints (trigger -> action) • Behavior model (perception, motivation, planning) • High level interface to expose capabilities • Naïve physics • Examples • Avatar • Tamaguchi/pet • Car • Elevator Intelligent Tools for Media

  32. Problem specification -example Smart lift in crowd simulation Intelligent Tools for Media

  33. Defining object behaviors - example • smart table in party simulation • graspable fruits • get_closest_ fruit (and eat it) • replace_fruits Intelligent Tools for Media

  34. Script Parsing • Define syntax • Define internal representation • Parse input, create representation, and write to file • Challenges • Action • Space (near, behind, path planning, obstacles) • Time (before, after, concurrent, fast) Intelligent Tools for Media

  35. Along Example Intelligent Tools for Media

  36. Animation Generation • Define internal representation • Read representation from file • Generate VRML • Challenges • Re-use VRML code • Performance and optimization • scalability Intelligent Tools for Media

  37. Automatic Editing • Formalize cinematic principles • Analyze screenplay • Hints (Annotations) in screenplay • Editing • Separate into shots • Decide shot type • Camera motion • Light • Sound • Dissolves • Effects Intelligent Tools for Media

  38. The Line External External Internal Apex Example - shots Camera placement Intelligent Tools for Media

  39. Examples • http://www.math.tau.ac.il/~doronf/se-anim_files/frame.htm Intelligent Tools for Media

  40. Template-based Video Editing Intelligent Tools for Media

  41. DOM Abstract Template XML XSL material 1…. XSL XSL material n SMIL-1 MPEG-4 caching DB caching Automated template-based generation • Wizard approach: user fills in the slots • Performance: static elements of presentation are pre-cached (unless very large) • Allows for dynamic, on-the-fly generation of presentations Intelligent Tools for Media

  42. Rich Media Transcoding Intelligent Tools for Media

More Related