1 / 67

Motion synthesis

This article explores motion synthesis techniques to generate human-like motions that look natural and can be controlled and interacted with. The evaluation of what "looks human" is discussed, as well as the challenges of describing human activities and considering nearby objects. Various methods for motion synthesis are examined, including animator-based, kinematic control, physical and biomechanical models, statistical models, and data-driven techniques. The concept of motion graphs and their use in motion search is also explored, along with the characteristics and properties of motion.

gertrudet
Download Presentation

Motion synthesis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Motion synthesis • Goals: • generate human motions that “look human” and “do what you want” • Synthesis • with control; with interaction • Evaluation • what “looks human?” • Features • Motion composes across the body and across time • so the number of available motions is huge • Multiple constraints on the appearance of motion • physics; • motor control system; • internal motion goals; • nearby objects;

  2. Key problems • What makes a motion look human? • can we tell good motions from bad? • How do we describe human activities? • with what vocabulary? at what time scales? • How do nearby objects affect our description • interactions and context

  3. Motion synthesis difficulties • People are good at spotting poor motion • and it sometimes matters • Motions can be very fast and very detailed • high accelerations, contacts create major issues • Authoring is mysterious • how does one specify constraints on activity usefully? • Complexity • interactions with objects, etc. create a need for families of motion • motion composes in nasty ways • motions should interact with objects, users, etc. • Control • character should be manageable • have some capability to cope on its own

  4. Motion synthesis, cont • Motion composes across the body and across time • so the number of available motions is huge • Multiple constraints on the appearance of motion • physics; • motor control system; • internal motion goals; • nearby objects;

  5. Motion synthesis • Methods • By animator • By kinematic control • profound difficulties with ambiguity • By combining observations • old tradition of move trees; also (Kovar et al 02, Lee et al 02, Arikan+Forsyth 02, Arikan et al 03,Gleicher et al 03) • By physical models • old tradition; (Witkin+Kass, 88; Witkin+Popovic 99; Funge et al 88; Fang+Pollard 03, 04) • By biomechanical models • old tradition; Liu+Popovic 02; Abe et al 04; Wu+Popovic 03; Liu+Popovic 02) • By statistical models • old tradition (e.g. Ramsey+Silverman 97); Li et al 02; Safanova et al 04; Mataric et al 99; Mataric 00; Jenkins+Mataric 04;

  6. Variational and Physical Methods

  7. Statistical models • Motion is a space-time curve • segment the curve • fit some form of parametric model to the pieces • parameters: • endpoints • time • etc Mataric et al,

  8. Mataric et al,

  9. Data-driven motion synthesis • Analogies • Text synthesis (Shannon) • Texture synthesis (Efros+Leung `99; many others since) “It means that in speaking with you, I am aware of how I think this is one of those questions that exposes a contradiction in our cultural cognitive disconnect the concept of authenticity exposes is, I believe, that we have inner and outer selves, and that the inner self is our real self. I personally find those ideas more misleading than helpful.”

  10. Motion graph • Take measured frames of motion as nodes • from motion capture, given us by our friends • Directed edge from frame to any that could succeed it • decide by dynamical similarity criterion • see also (Kovar et al 02; Lee et al 02) • A path is a motion • Search with constraints • root position+orientation • length of motion • occupy a frame at specified time • limb close to a point Motion Graph: Nodes = Frames Edges = Transition A path = A motion

  11. Search in a motion graph • Local • Kovar et al 02 • With some horizon • Lee et al 02; Ikemoto, Arikan+Forsyth 05 • Whole path • Arikan+Forsyth 02; Arikan et al 03 Motion Graph: Nodes = Frames Edges = Transition A path = A motion

  12. Local Search methods • Choose the next edge (Kovar, Gleicher, Pighin 02) • ensure that one can’t get stuck locally • but can’t guarantee a goal is available on longer scale

  13. On-line control of motion synthesis Reach target position Avoid patrolling enemy Dodge moving obstacles Don’t pass through stationary obstacles Agent travels along motion graph. When he reaches a decision point, he must choose which branch to take so he can best meet his objectives. Ikemoto+Arikan+Forsyth 05

  14. s1 a1 s a2 s2 a3 s3 Value of state s obtained by comparing to a set of example states, encoded using following weighted terms Distance to next waypoint on global path plan Local geometry Visible enemies Ikemoto+Forsyth+Arikan 05

  15. Reinforcement learning Sample control parameters (w) for a random state (s) Fix for the motion graph Generate motion Reward 1 w1 Fix for the motion graph Generate motion Reward 2 w2 … Fix for the motion graph Reward 3000 Generate motion w3000 Fix control parameters for state s to be the w that yielded maximum reward Ikemoto+Arikan+Forsyth 05

  16. Ikemoto+Arikan+Forsyth 05

  17. Ikemoto+Arikan+Forsyth 05

  18. Characteristic properties of motion • Characteristic features • most demands are radically underconstrained • motion is simultaneously • hugely ambiguous • “low entropy” • Suggests using “summaries” Arikan+Forsyth 02

  19. Summaries - 1 Make sequences into nodes

  20. Summaries - 2 • Graph as matrix • Edges form clumps • hierarchical clustering gives a hierarchy of graphs

  21. Search Maintain a pool of paths, inserting new “seed” paths on occasion Modify the motion path by: - removing a node - replacing two edges Text OR: - refine path by moving to higher resolution motion graph

  22. Arikan+Forsyth 02

  23. Arikan+Forsyth 02

  24. Arikan+Forsyth 02

  25. Arikan+Forsyth 02

  26. Arikan+Forsyth 02

  27. Limitations • Can’t synthesize motions one hasn’t seen • but see later • Long term structure of motion is strange • running backwards, etc. • No on-the-fly control of motion or interaction • but see later • Require more detailed control of “type” of motion • can deal with this

  28. Synthesis with off-line control • Annotate motions • using a classifier and on-line learning • efficient human-in-the loop training • Produce a sequence that meets annotation demands • a form of dynamic programming

  29. Annotation - desirable features • Composability • run and wave; • Comprehensive but not canonical vocabulary • because we don’t know a canonical vocabulary • Speed and efficiency • because we don’t know a canonical vocab. • Can do this with one classifier per vocabulary item • use an SVM applied to joint angles • form of on-line learning with human in the loop • works startlingly well (in practice 13 bits) P Walk classifier O Run classifier O Jump classifier P Stand classifier O Carry classifier Arikan+Forsyth+O’Brien 03

  30.    n - frames WalkPPPP Run Jump WavePPOO Carry Motion demand Synthesis by dynamic programming All frames in the database Arikan+Forsyth+O’Brien 03

  31. Dynamic programming practicalities • Scale • Too many frames to synthesize • Too many frames in motion graph • Obtain good summary path, refine • Form long blocks of motion, cluster • DP on stratified sample • split blocks on “best” path • find similar subblocks • DP on this lot • etc. to 1-frame blocks Arikan+Forsyth+O’Brien 03

  32. Arikan+Forsyth+O’Brien 03

  33. Arikan+Forsyth+O’Brien 03

  34. Still open • Local control of synthesis • Long term structure of motion is strange • running backwards, etc. • essential for interaction • Departing from data? • Can’t synthesize motions one hasn’t seen • essential for interaction

  35. Transplantation • Motions clearly have a compositional character • Why not cut limbs off some motions and attach to others? • we get some bad motions • build a classifier to tell good from bad • avoid foot slide by leaving lower body alone Ikemoto+Forsyth 04

  36. Loop { Randomly pick a synthesis rule D J dij Djikstra’s to find a path Kovar, Gleicher, Pighin 2002 If successful, output candidate motions } Ikemoto+Forsyth 04

  37. Ikemoto+Forsyth 04

  38. Ikemoto+Forsyth 04

  39. Ikemoto+Forsyth 04

  40. Ikemoto+Forsyth 04

  41. Ikemoto+Forsyth 04

  42. Ikemoto+Forsyth 04

  43. Evaluate in application Check Generate 22, 645 477, 362 340, 596 labelled “human” Classifier’s total error rate 13% false positive rate 12% But what does this mean in practice? Ikemoto+Forsyth 04

  44. Evaluate Original motion graph Unreal Tournament 2004 Synthesizer Motion Ai Motion demands Proxy Synthesizer Motion Bi • Position • Velocity • Rotation • Running/Falling + 40 hand-generated streams Enriched graph • Stand • Walk • Run • Jump Ikemoto+Forsyth 04

  45. Is the enriched graph better? Ikemoto+Forsyth 04

  46. Blending motion • Motion data can be blended successfully • known for a long time • can’t blend arbitrary motions • blending schedule doesn’t seem to matter all that much • multiple motions can be blended • time alignment matters • Idea: • search possible blends and score them • cache the best • at run-time, query cache

More Related