Modeling and Animating Eye Blinks

1. Modeling and Animating Eye Blinks • Laura Trutoiu, Liz Carter, Iain Matthews, Jessica Hodgins

2. What is an eye blink?

3. Eye blinks: Common wisdom • "... five frames is usually adequate for most situations, although four frames can make a character look more alert." • The Animator's Survival Kit by Richard Williams, Faber and Faber, NYC, 2001. Inter-eyelid distance Inter-eyelid distance Inter-eyelid distance Frame (24fps) Figure adapted from “Digital Character Animation”, Maestri,G . New Riders Publishing. 1996

4. Eye blinks: Common wisdom • "... five frames is usually adequate for most situations, although four frames can make a character look more alert." • The Animator's Survival Kit by Richard Williams, Faber and Faber, NYC, 2001. Inter-eyelid distance Frame (24fps) Figure adapted from “Digital Character Animation”, Maestri,G . New Riders Publishing. 1996

5. Inter-eyelid distance (pixels) 50 100 150 200 250 300 Frame (300 fps) Eye Blinks: What can we see? • Temporal asymmetry: • Fast closing, slow opening • Closing amplitude: • Eyes may not fully close • Variety: • Varying duration & closing amplitude • Lower eyelid motion

6. Eye blinks: Can you tell the difference? Video 1: forward Video 2: backward Eye blinks show temporal asymmetry and you can tell the difference.

7. Related work: Eyes in graphics • Eyes alive [Lee 2002] • eye gaze/saccades only • Eye motion with texture synthesis [Deng 2005] • no explicit eye blink model • requires training • Eyelid kinematics [Steptoe 2010] • parametric models for eyelid saccades and eye blinks using current position • 10 participants Eyes alive. LEE, S., BADLER, J., AND BADLER, N. ACM Trans. on Graphics 21, 3, 637–644. 2002.Automated eye motion using texture synthesis. DENG Z., LEWIS J. P., NEUMANN U. IEEE Computer Graphics and Applications 25, 2, 24– 30, 2005 Eyelid kinematics for virtual characters. STEPTOE, W., OYEKOYA, O., AND STEED, A. Comput. Animat. Virtual Worlds 21 (May), 161–171. 2010

8. How do we create more natural eye-blink animations? • Trackhigh speed video of eye blinks • Build agenerative modelthat closely resembles the tracked data • Validate the model throughperceptual experiments Why simple approximations are not good enough

9. Inter-eyelid distance (pixels) 1000 2500 3000 500 50 100 200 250 300 150 Inter-eyelid distance (pixels) Tracking eye blinks • Actor performance • Emotional vignettes and in-between pauses • High speed video • Casio Exilim F1 - 300 fps • Video tracking • Active Appearance Models (AAMs) [Matthews 2004] Active appearance models revisited. Matthews, I. and Baker, S. International Journal of Computer Vision 60, 2, 135–164. 2004

10. Tracking eye blinks

11. Building a generative model Normalized time series Eye blink time series Generated Data Re-timed data Principal Component Analysis • Video = unobtrusive way to measure eye blink dynamicsHigh speed video => temporal resolution

12. Perceptual experiments • Goal: • Can naive users differentiate between different eye blink dynamics? • Stimuli: • Animations at 30 fps • Role of appearance: • photorealistic and cartoon character • Naturalness rating (1-7) • Very natural clip (7) = “what you would expect to see in the real world”

13. Perceptual experiments

14. Perceptual experiments

15. Exp 1: Model vs. real blinks 1 2 Model, fully closed Model, naturally closed 3 4 Real, fully closed Real, naturally closed Model vs. real + Closing amplitude

16. p < 0.001 Naturalness Rating Real Model Exp 1: Model vs. real blinks * Real Model 300 fps n = 32, 2 (character) x 2 (profiles) x 2 (blur) x 2 (amplitude) repeated-measures ANOVA

17. Naturalness Rating Fully closed Naturally closed Exp 1: Model vs. real blinks Model fully closed Model naturally closed 300 fps n = 32, 2 (character) x 2 (profiles) x 2 (blur) x 2 (amplitude) repeated-measures ANOVA

18. Perceptual experiments Photorealistic > Cartoon Fully closed > Naturally closed

19. Perceptual experiments Photorealistic > Cartoon Fully closed > Naturally closed

20. Exp 2: Model vs. Approximations Model Linear Symmetric Linear Asymmetric Ease In/Ease OutSymmetric Ease In/Ease Out Asymmetric Hypothesis: Asymmetric > Symmetric, Non-linear > Linear

21. Exp 2: Model vs. approximations 1 Model 2 3 Asymmetric linear Asymmetric ease-in/ ease-out 4 5 Symmetric ease-in/ ease-out Symmetric linear

22. Exp 2: Model vs. approximations n = 43, 5 (profiles) x 4 (length) x 2 (character) repeated-measures ANOVA

23. Perceptual experiments Photorealistic > Cartoon Fully closed > Naturally closed Model > Approximations

24. Conclusions: Eye blinks are ... • Not symmetric: • More time to open than to close • Animation textbook - same number of frames for opening and closing • Not linear: • Fast closing (linear velocity) followed by slowly converging opening • Animation - linear or slowly accelerating, slowly decelerating • Eyes may not fully close: • At least 50% of the time • Animation - eyes always fully close • In our experiments, fully closed animations preferred ?

25. Future work • Sequences of eye blinks • inter-blink distance • variability may be useful • Synchronization with other facial motion • Blink sequence patterns

27. 1 2 Vertical only Horizontal only 3 4 Vertical + Horizontal No Lower eyelid motion Exp 3: Lower eyelid motion

28. Exp 3: Lower eyelid motion Horizontal+ Vertical No HorizontalNo Vertical Horizontal Vertical n = 32, 4 (lower eyelid motion) x 1 repeated-measures ANOVA