1 / 20

Facial Motion Capture for Games

Facial Motion Capture for Games. Adam Schwartz 12/01/2010. Introduction. State of the art: High end face animations: Cryengine Common / lowspec: Unreal Engine Regardless of tech, facial animation is now commonplace in games. Bones.

signa
Download Presentation

Facial Motion Capture for Games

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Facial Motion Capture for Games Adam Schwartz 12/01/2010

  2. Introduction • State of the art: • High end face animations: Cryengine • Common / lowspec: Unreal Engine • Regardless of tech, facial animation is now commonplace in games.

  3. Bones • Full Body Animation: Motion Capture is expressed on bones. • Unlike human bones, which provide structure for movement by muscles. • Animation bones: drive the movement of a structure composed of vertices; can be skinned (attached) to any kind of geometry.

  4. Bones • Simple construction • Easy characterization • Streamlined connection to motion capture

  5. Facial Animation: Motion capture is not usually done with bones, but with Blendshapes • Blendshapes: Predefined deformations of facial geometry. Unlike skeletons, these can’t be transferred to alternative geometry. Must have specific deformations for each of Motionbuilder’s Generic Expressions

  6. Blendshape Animation Example

  7. Blendshapes: Conclusions • Good for one character, not many. • Require significant artistic skill. • Alternative: Bones • Problem: Bone-based animation is standardized in Motionbuilder… but only for body animation.

  8. Skeleton as Clusters: An Alternative • Workaround: Cluster Groups • Bones ~ clusters • Bones can be reskinned regardless of geometry. • Weights can be adjusted faster than defining new blendshapes.

  9. Tools: Mocap • Motion Capture Lab in Department of Industrial Engineering • Vicon System • 4 megapixel cameras (120 Hz), SOA • 10 motion cameras, 2 video • cameras, passive reflective • markers • Four force plates to measure forces on ground • Facial expressions, electromagnetic device capture voltage in max 16 muscles

  10. Alternative Mocap • Active Marker systems (LED, etc) • Markerless Vision-Based • Non-Optical • Future: Mova Contour Reality Capture • 25 passive markers (for face) is sufficient.

  11. Tools: Software • Autodesk Maya 2010 • Autodesk Motionbuilder 2010 • Alternatives: Autodesk 3dsmax, Blender

  12. The Process • Import mocap data to Motionbuilder, connect to the “actor”.

  13. The Process • Express the actor on a “character” - a skeleton without bound skin geometry. • Characterize body as usual. • Characterize face as movements of the face’s bones instead of blendshape percentages.

  14. The Process • Plot all animations to the skeleton.

  15. The Process • An identical skeleton is resized and skinned to the character geometry in Maya, exported to Motionbuilder, and merged.

  16. The Process • The character is sent back to Maya, paint to adjust weights. • Group teeth, eyes, etc with relevant bones. • Animate Blinking. • Manually make final adjustments with keyframing.

  17. Character

  18. Side By Side

  19. Related Work • Standard Blendshape Method: Not transferable between characters, requires artistic skill. Very realistic results. (Autodesk, 2009) • Cluster group animation: Not transferable between characters. No significant artistic skill needed. Result quality is between bones and blendshapes. (3D Buzz, 2010) • Alternative bone-based rigs: Not standardized (yet)… • Updates to Maya to expand skeleton/mocap functionality. (Autodesk, 2010)

  20. References, Resources, Further Reading • Expressive Facial Gestures From Motion Capture Data by Eunjung Ju and Jehee Lee (EUROGRAPHICS 2008 / G. Drettakis and R. Scopigno • (Guest Editors) Volume 27 (2008), Number 2 • Joint Based Facial Rig by Mathew Smith www.jessieamadio.com/wordpress/?p=65 • Full-Body Character Riggingand Facial Motion Capture by Matthew Wynne http://www.mwynneconsulting.com • Maya/Motionbuilder Cluster Tutorials - http://www.3dbuzz.com • Tutorials and Forums at area.autodesk.com • Autodesk Maya 2010. San Rafael, California: Autodesk, Inc., 2009. Program documentation. • Motionbuilder Issue 4 http://www.3dbuzz.com/vbforum/sv_showvideo.php?v=1131, retrieved on 5 May 2010 • Autodesk HumanIK http://usa.autodesk.com/adsk/servlet/pc/index?id=9491249&siteID=123112 , retrieved on 1 Dec 2010

More Related