Motion Capture Animation & Procedural Animation

1. Motion Capture Animation & Procedural Animation Dr. Midori Kitagawa Arts and Technology Program University of Texas at Dallas Gif animations by Dax Norman

2. Three types of 3D computer animation • Keyframe animation • Motion capture animation • Procedural animation

3. Keyframe Animation

4. Keyframe animation • Keyframe is a drawing of a key moment in an animated sequence, where the motion is at its extreme. • Inbetweens fill the gaps between keyframes. • Every motion is created by animators.

5. Keyframe animation • To produce 3D keyframe animation, the animator builds the behavior of a 3D model by keying parameter values in key frames where the values are at their extremes.

6. Keyframe animation • 3D keyframe animation method is based on traditional hand-drawn 2D keyframe animation method. • Animation is manually produced from scratch. • Examples: Pixar animations

7. Motion capture animation

8. Motion capture animation • To produce a motion capture animation, the motion of a capture subject is recorded as 3D data and applied to a 3D model. • The animation is produced from the capture subject’s movement.

9. Motion capture animation • Although no 3D data is involved in rotoscoping, motion capture technology is sometimes referred as “devil’s rotoscoping.” • Examples: Avatar (2009), How to Train Your Dragon 2 (2014)

10. Procedural animation

11. Procedural animation • To produce a procedural animation, the animator defines a procedure or a set of operations to be performed. • Each operation can generate or alter data that passes through it and can be conditionally or non-conditionally executed.

12. Procedural animation • With procedural animation methods (e.g., particle systems, rigid dynamics and flexible dynamics), the user specifies a set of rules, initial conditions and parameter values and runs simulations. • Physics-based animation is a subset of procedural animation.

13. Procedural animation • Procedural methods can be used in any part of production pipeline to produce geometries, textures, lights, rigs, animation, composites, sounds and other elements. • Examples: visual effects (e.g., smokes, steam, fire, clouds, dust, water, and plants) in live action films and 2D/3D animations.

14. ATEC 4371 Topics in Animation: Procedural Methods for Computer Animation Fall 2015 Semester • Prerequisite: ATEC 2326 Computer Animation Process • Textbook: The Art of 3D Computer Animation and Effects, 4th Edition by Isaac Kerlow • Software: Houdini (Side Effects Inc. ) which is node-based.

15. More about motion capture animatoin…

16. What is Mocap? • Motion capture (mocap) is sampling and recording motion of humans, animals, and inanimate objects as 3D data. • The data can be used to study motion or to give an illusion of life to 3D computer models.

17. Applications • Entertainment • Medicine / Sports • Arts / Education • Science / Engineering

18. Entertainment: Live Action Films

19. Entertainment: 3D computer animations

20. Entertainment: Video Games

21. Medicine / Sports • Medicine (e.g., gait analysis, rehabilitation) • Sports (e.g. injury prevention, performance analyses, performance enhancement)

22. Arts / Education • Dance and theatrical performances • Archiving (e.g., Marcel Marceau) OSU/ACCAD

23. Science / Engineering • Engineering (e.g., biped robot developments) • Computer Science (e.g., human motion database, indexing, recognitions) • Design (e.g., ergonomic product design) • Military (e.g., field exercises, virtual instructors, and role-playing games)

24. Mocap animation • Motion capture animation is different from keyframe animation in terms of how motion is created. • Same principles of animations apply to mocap animation & keyframe animation! • Combination of motion capture animation, keyframe animation and procedural animatoin is often used.

25. Advantages of mocap animation • Faster to create (only if an established production pipeline exists.) • Secondary motions and all the subtle motions are captured. -> more realism • Physical interactions between performers and props can be captured. -> more realism

26. Disadvantages of mocap animation • Cost. • Manipulating mocap data is often difficult -> Re-capturing or key framing a shot with bad data is often easier. • Mapping mocap data of a performer to a 3D character with a different proportion may cause issues.

27. Popular types of mocap equipment • Magnetic systems • Mechanical systems • Optical systems

28. Magnetic systems • Utilize sensors placed on the body to measure the magnetic field generated by a transmitter source.

29. Magnetic systems •  Require no special lighting condition. •  Sensors are never occluded. • X Affected by electromagnetic force.

30. Mechanical systems • Exoskeleton with angle sensors.

31. Mechanical systems •  Measure joint angles (no marker ID problems). •  Sensors are never occluded. • X Breakable! • X Configuration of sensors is fixed. • X Constrains on joints.

32. Optical systems • The cameras are equipped with infrared LED's and filters. • The cameras see reflector markers.

33. Optical systems •  Higher sampling rate. •  Larger capture space. • X Markers are sometimes occluded -> marker ID problems. • X Provide only positional data -> joint angles need to be computed.

34. Mocap system at UTD • Vicon optical system • 8 high-speed MX 13 (up to 10,000 fps) and 8 high-resolution MX 40 (4 megapixels) cameras. • Capture up to 5 performers at once.

35. Mocap animation courses Graduate level: ATEC 6352 & UG level: ATEC 4345 • Group project based • Group of 3 to 5 members • Semester long project Pre-requisite: • ATEC 3317 Modeling and texturing OR • ATEC 3327 Lighting and composition Recommended: • ATEC 3328 Rigging I • ATEC 4337 Computer Animation

36. Mocap animation pipeline Preproduction Capture -> data cleaning -> retargeting Modeling -> rigging ----------------------------- mocap animation Keyframe animation Effect animation Sound design Texturing/lighting -> Rendering -> Composition Post-production * Tasks that everyone should be involved. * Tasks that can be designated to a member or members.

37. 2D Image -> 2D Camera Data -> 3D Markers Positions -> Trajectories -> Recon Skeleton -> Solver Skeleton System Calibration and Capturing and Processing Data FBX Reconstruction Triangulation Circle fitting Skeleton with subject’s size & proportion and motion data BLADE Subject Calibration VSK MOTION BUILDER VST(subject template) • Capture Range of Motion (ROM) • Reconstruct trajectories of ROM • Label markers Skeleton with subject’s size & proportion Mocap Pipeline Flow Chart Character Setup The processes that you go through for each character • Correlate : • Character • & • Character • (FBX) • Edit motion • Bake motion data to the skeleton FBX Character Setup • Create a skeleton • Bind skin to the skeleton • Rig the character Skeleton with 3D character’s size & proportion (and skin) MAYA The processes that you repeat for each shot Rendering FBX • Marge the rigged character (MB) and the skeleton with motion data (FBX) • Edit motion (IK/FK blend, Trax) • Render MB Skeleton with motion data & 3D character’s size & proportion Maya Scene File

38. UG Mocap & Procedural Animation course schedule Summer 2015: ATEC 4345 Motion Capture Animation • Prerequisite: ATEC 3317 Modeling and texturing OR • Prerequisite: ATEC 3327 Lighting and composition • Mondays 1- 5 pm Fall 2015: ATEC 4371 Topics: Procedural Methods • Prerequisite: ATEC 2326 Computer Animation Process • Mondays 1 – 3:45 pm Spring 2016: ATEC 4345 Motion Capture Animation

39. Dr. Midori Kitagawa midori@utdallas.edu ATEC 1.909