3D Skeleton-based Human Modeling with Metaballs

1. 3D Skeleton-based Human Modeling with Metaballs 18 April 2008 Donghun Kim Robot Vision Lab

2. Contents • Motivation • What is the Metaball? • How can we visualize Metaballs? • 3D Skeleton Human Model • Human model using Metaballs • Examples • Future Works

3. Motivation • Motion Analysis of Non-rigid Object • Motion Analysis of Articulated Object • Model based Pose Estimation and Tracking • 3D Mesh Data by Visual Hall • 3D Skeleton( Curve Skeleton by Thinning, DT, Geometric or General Fields) - on going • 3D Skeleton-based Motion tracking • Metaball Human Model for shape recovery • Motion Analysis by Skeleton-based Motion and Shape Info. (i.e. Manifold learning)

4. Implicit Surfaces • Implicit Surfaces: Surfaces which are contours(isosurface) through some scalar field in 3D • Different Field function: Metaball, Soft Objects, Blobbies

5. What are isosurfaces? • Consider a function f(x,y,z) which define a scalar field in 3D space. • Isosurface S is set of points for which f(x,y,z) = const. • It can be thought as an Implicit function relating x,y and z -> so called implicit surface sometimes

6. Metaballs x: a point in 3d space N: num. of metaballs pi : center position of metaballs ri : Metaball’s own density field • A particularly interesting case • Use implicit equation of the form • Gradient can be computed directly • Soft/blobby objects that blend into each other

7. Field Functions • Blobby Molecules • Metaballs • Soft Objects r is distance of a point in space to a particular control point

8. Comparison of Field Functions < Figure from [4]>

9. Examples < Images from [4]>

10. Metaballs are cool! 

11. Marching Cubes • Well-known Method for scalr field polygonizataion • Sample f(x,y,z) on a cubic lattice • For each cubic cell • Estimate where isosurface inetersects cell edges by linear interpolation • Tessellate depending on values of f() at cell vertices

12. Marching Cubes • Algorithm for creating a polygonal surface representation of an isosurface of a 3D scalar field • Combine simplicity with high speed because of using lookup tables • Application • Reconstruction of a surface from volumetric datasets • Creating a 3D contour of a mathematical scalar field

13. Marching Cubes • Each vertex can be either “inside” or “outside” • For each cube cell there are 256 ways for isosurface to intersect it can be simplified down to 15 unique

14. Marching Cubes • Example cubeindex = 0; if (grid.val[0] < isolevel) cubeindex |= 1; if (grid.val[1] < isolevel) cubeindex |= 2; if (grid.val[2] < isolevel) cubeindex |= 4; if (grid.val[3] < isolevel) cubeindex |= 8; if (grid.val[4] < isolevel) cubeindex |= 16; if (grid.val[5] < isolevel) cubeindex |= 32; if (grid.val[6] < isolevel) cubeindex |= 64; if (grid.val[7] < isolevel) cubeindex |= 128; < Images from [5]> P = P1 + (isovalue - V1) (P2 - P1) / (V2 - V1)

15. Marching Cubes • Sampling Grid Resolution • Smoothness and Processing time to display < Images from [5]>

16. Examples of Marching Cubes • More Information about Marching Cubes is in [3].

17. 7 3 2 6 11 8 15 12 1 0 4 5 9 10 13 14 3D Skeleton Human Model • Structure( 15 nodes, 14 metaballs)

18. Human Model using Metaballs • Head(1-2-7)

19. Human Model using Metaballs • Chest(0-1-2-3-6)

20. Human Model using Metaballs • Left Arm(1-2-3-8-12)

21. Human Model using Metaballs • Right Arm(1-2-5-11-15)

22. Human Model using Metaballs • Left Hip & Thigh(0-4-9)

23. Human Model using Metaballs • Right Hip & Thigh(0-5-10)

24. Human Model using Metaballs • Left Leg(4-9-13)

25. Human Model using Metaballs • Right leg(5-10-14)

26. Human Model using Metaballs • Result (Polygon Surface)

27. Human Model using Metaballs • Result (Filled surface)

28. Programming • 3D cloud data and mesh data from Visual Hall(using EPVH lib) as the input of 3D skeleton • 3D Human Model Tool based on Jonney’s 3D Human Model Frame (using Ellipsoid, Cylinder, Sphere) • OpenGL and FLTK GUI based tool • Functions • 3D reconstruction by Visual hall • Obtain Multiple Camera-view point images in OpenGL Environment • Calculate Camera matrix by transforming the OpenGL representation to Physical camera representation • Parameterized 3D Skeleton Human Model • 3D Skeleton-based Metaball Model • Simple model size option (Height considering body ratio, fat-thin option) • Image Saving with OpenCV in OpenGL

29. Modeling Tool Made by Dave Kim and Thanks to Jonney

30. Modeling Tool Made by Dave Kim and Thanks to Jonney

31. Data Comparison • Multi-View Silhouette Images of Different Human Model • Ellipsoid, Cylinder and Sphere vs. Metaballs • 3D cloud points and triangle meshes by Visual hall

32. Example (I)

33. Example (I)

34. Example (II) • Metaball Human Model

35. Example (II) • Result by Visual Hall

36. Future Work • 3D Curve skeleton Algorithm[2] • DT (fast) + General Field (robust) • 3D skeleton human model fitting to Curve Skeleton -> Pose Estimation • Shape recovery using the information from the skeletonization • Motion Analysis

37. Reference • Clement Menier, Bruno Raffin, “3D Skeleton-based Pose Recovery,” The 3rd international Symposimum on 3D Data Processing, Visualization and Transmission, 2006 • Nicu D. Cormea etc at al., “Curve-Skeleton Application,” IEEE Visualization 2005 • T. S. Newman, H. Yi, “A survey of the marching cubes algorithm,” Computers and Graphics, 2006 • Implicit Surfaces written by Paul Bourke, June 1997, http://local.wasp.uwa.edu.au/~pbourke/modelling_rendering/implicitsurf/ • Polygonising A Scalar Field written by Paul Bourke, May 1994, http://local.wasp.uwa.edu.au/~pbourke/geometry/polygonise