A Hierarchical Approach to Motion Analysis and Synthesis for Articulated Figures

1. A Hierarchical Approach toMotion Analysis and Synthesisfor Articulated Figures Jehee Lee Computer Science Department KAIST

2. Content • 1. Introduction • 2. Spatial filtering for orientation data • 3. Motion editing with spacetime constraints • 4. Multiresolution motion analysis and synthesis • 5. Conclusion and Future work

3. Character Animation • Realistic motion data • motion capture technology • commercial libraries • Produce animation from available motion clips • requires specialized tools • interactive editing, smoothing, enhancement, blending, … • provides a great variety in • character size, environment geometry, scenario, ...

4. Motion of Articulated Figures • Structure of motion data • bundle of motion signals • each signal represents either position or orientation • Difficulties in handling motion data • orientations yield complications • unit quaternions • mutual relationships among motion signals

5. Overview • Designing spatial filters for motion data • Motion editing with spacetime constraints • Multiresolution motion analysis and synthesis • Crafting animation from motion-captured data raw input signal convincing animation of arbitrary length Filtering Editing & Adaptation Analysis & Synthesis

6. Content • 1. Introduction • 2. Spatial filtering for orientation data • 3. Motion editing with spacetime constraints • 4. Multiresolution motion analysis and synthesis • 5. Conclusion and Future work

7. Spatial Filtering for Orientation Data • Linear shift-invariant (LSI) filter • filter mask : • vector-valued signal : • Not suitable for unit quaternion data • unit-length constraints

8. Previous Work • Brute-force normalization • Azuma and Bishop (‘94) • Employ exponential and logarithmic maps • Lee and Shin (‘96) • Fang et al (‘98) • Hsieh et al (‘98) • lack of crucial filter properties

9. Requirements • Avoid brute-force normalization • Satisfy desired properties • Independent of coordinate system • Independent of time • Independent of reflection

10. Basic Idea • Exploit correspondence in differential spaces • Linear motion : • Angular motion : Velocity Acceleration

11. Transformation • Transformation between linear and angular signals

12. Filter Design • Given: spatial filter F • Output: spatial filter H for orientation data • filter responses of unit length • locally supported • #support(H) = #support(F)

13. Filter Design • Given: spatial filter F • Output: spatial filter H for orientation data • filter responses of unit length • locally supported • #support(H) = #support(F)

14. Properties of Orientation Filters • Coordinate-invariance • Shift-invariance • Symmetry

15. Experimental Results (1) • Blurring by binomial masks Original Filtered Angular acceleration Original Filtered

16. Experimental Results (2) • Smoothing Original Filtered Angular acceleration Original Filtered

17. Experimental Results (3) • High-frequency boosting Original Filtered Angular acceleration Original Filtered

18. Video Smoothing and Sharpening

19. Summary (Motion Filtering) • A general scheme of constructing spatial filters for orientation data • satisfies desired properties • simple, efficient, and easy to implement • performs well for live-captured data

20. Content • 1. Introduction • 2. Spatial filtering for orientation data • 3. Motion editing with spacetime constraints • 4. Multiresolution motion analysis and synthesis • 5. Conclusion and Future work

21. Motion Editing • Reuse available motion clips • Interactive motion editing • direct manipulation through graphical interface • Motion adaptation • new characters • new environments • new scenario

22. Spacetime Formulation • Spacetime constraints • [Witkin and Kass 88] [Cohen 92] [Gleicher 98] • important features of the original motion • new features to be accomplished • To find a new motion • satisfying given constraints • preserving original characteristics

23. Previous Work • Geometric techniques • Bruderlin and Williams (‘95) • Popovic and Witkin (‘95) • lack of consideration on kinematic constraints • Spacetime optimization • Rose et al (‘96) • Gleicher (‘98) • yield very large optimization that is cumbersome to handle

24. Video Direct manipulation with spacetime constraints

25. Basic Idea • Structure of motion sequences • Intra-frame relationship • satisfying constraints • by inverse kinematics • Inter-frame relationship • avoiding jerkiness • by curve fitting

26. Motion Representation • Configuration of articulated figures • comprises both linear and angular components • rigid transformation

27. Motion Displacement Mapping

28. Adaptive Refinement • Flexibility in representation • spline curves over a uniform knot sequence • hard to determine knot density • adaptive refinement is needed • Multilevel or hierarchical B-splines • [Lee, Wolberg and Shin 97] [Forsey and Bartels 95] • sum of uniform B-spline functions • coarse-to-fine hierarchy of knot sequences

29. Multilevel B-spline Fitting Smooth initial approximation

30. Multilevel B-spline Fitting Refinement with finer functions

31. Multilevel B-spline Fitting

32. Hierarchical Motion Fitting • Hierarchy of successively refined motions • successively finer displacement maps • at each level in the hierarchy, • to compute displacements at constrained frames • to derive a displacement map by curve fitting

33. Knot Spacing • Direct manipulation • larger spacing yields wider range of deformation • Precision control • depend on density of finest knot sequence

34. Video Examples of motion adaptation

35. Summary (Motion Editing & Adaptation) • Hierarchical motion fitting • hierarchical representations for displacement maps • provides adaptive refinement • allows to edit motion at any level of detail • interactive performance • easy to implement

36. Content • 1. Introduction • 2. Spatial filtering for orientation data • 3. Motion editing with spacetime constraints • 4. Multiresolution motion analysis and synthesis • 5. Conclusion and Future work

37. Motion Analysis and Synthesis • Hierarchical representations for motion signals • facilitate a variety of signal processing tasks • smoothing, attenuation and enhancement • stitching and blending motion clips • Analysis and Synthesis • transform motion signals into MR representations • synthesize new motions from MR representations

38. Multiresolution Analysis • Represent motion at multiple resolutions • Hierarchy of successively smoother and coarser signals • Hierarchy of displacement maps

39. Previous Work • Image and signal processing • texture analysis and synthesis, image editing, curve and surface manipulation, data compression, and so on • Motion analysis and synthesis • Fourier analysis • Unuma, Anjyo and Takeuchi (‘95) • Multiresolution representation • Bruderlin and Williams (‘95)

40. Reduction Expansion Decomposition • Reduction : smoothing, followed by down-sampling • Expansion : up-sampling, followed by smoothing • Both of them can be realized by spatial masking

41. Representation and Reconstruction • Representation • Reconstruction

42. Video Motion analysis and its applications

43. Multiresolution Synthesis • Frequency-based motion editing • editing the global pattern of example motion • without explicit segmentation • stitching and blending example motions

44. Shuffling and Reconstruction Multiresolution representation of example motion

45. Shuffling and Reconstruction The base signal of new motion Shuffling

46. Shuffling and Reconstruction Reconstruct detail coefficients Multiresolution Sampling Shuffling

47. Shuffling and Reconstruction Multiresolution Sampling Shuffling

48. Multiresolution Sampling • Feature matching • example) the change of linear and angular velocities Matching

49. Multiresolution Sampling • Feature matching • example) the change of linear and angular velocities Reconstruct Matching

50. Multiresolution Sampling • Matching features at multiple resolutions Reconstruct Matching Matching