CSE 554 Lecture 9: Extrinsic Deformations

1. CSE 554Lecture 9: Extrinsic Deformations Fall 2012

2. Review Source • Non-rigid deformation • Intrinsic methods: deforming the boundary points • An optimization problem • Minimize shape distortion • Maximize fit • Example: Laplacian-based deformation Target Before After

3. Extrinsic Deformation • Computing deformation of each point in the plane or volume • Not just points on the boundary curve or surface Credits: Adams and Nistri, BMC Evolutionary Biology (2010)

4. Extrinsic Deformation • Applications • Registering contents between images and volumes • Interactive spatial deformation

5. Techniques • Thin-plate spline deformation • Free form deformation • Cage-based deformation

6. Thin-Plate Spline • Given corresponding source and target points • Computes a spatial deformation function for every point in the 2D plane or 3D volume Credits: Sprengel et al, EMBS (1996)

7. Thin-Plate Spline • A minimization problem • Minimizing distances between source and target points • Minimizing distortion of the space (as if bending a thin sheet of metal) • There is a closed-form solution • Solving a linear system of equations

8. Thin-Plate Spline • Input • Source points: p1,…,pn • Target points: q1,…,qn • Output • A deformation function f[p] for any point p pi qi p f[p]

9. Thin-Plate Spline • Minimization formulation • Ef: fitting term • Measures how close is the deformed source to the target • Ed: distortion term • Measures how much the space is warped • : weight • Controls how much non-rigid warping is allowed

10. Thin-Plate Spline • Fitting term • Minimizing sum of squared distances between deformed source points and target points

11. Thin-Plate Spline • Distortion term • Minimizing a physical bending energy on a metal sheet (2D): • The energy is zero when the deformation is affine • Translation, rotation, scaling, shearing

12. Thin-Plate Spline • Finding the minimizer for • Uniquely exists, and has a closed form: • M: an affine transformation matrix • vi: translation vectors (one per source point) • Both M and vi are determined by pi,qi, where

13. Thin-Plate Spline • Result • At higher , the deformation is closer to an affine transformation Credits: Sprengel et al, EMBS (1996)

14. Thin-Plate Spline • Application: image registration • Manual or automatic feature pair detection Source Target Deformed source Credits: Rohr et al, TMI (2001)

15. Thin-Plate Spline • Advantages • Smooth deformations, with physical analogy • Closed-form solution • Few free parameters (no tuning is required) • Disadvantages • Solving the equations still takes time (hence cannot perform “Interactive” deformation)

16. Free Form Deformation • Uses a control lattice that embeds the shape • Deforming the lattice points warps the embedded shape Credits: Sederberg and Parry, SIGGRAPH (1986)

17. Free Form Deformation • Warping the space by “blending” the deformation at the control points • Each deformed point is a weighted sum of deformed lattice points

18. Free Form Deformation • Input • Source lattice points: p1,…,pn • Target lattice points: q1,…,qn • Output • A deformation function f[p] for any point p in the lattice grid: • wi[p]: determined by relative position of p with respect to pi p f[p] qi pi

19. Free Form Deformation • Desirable properties of the weights wi[p] • Greater when p is closer to pi • So that the influence of each control point is local • Smoothly varies with location of p • So that the deformation is smooth • So that f[p] =  wi[p] qi is an affine combination of qi • So that f[p]=p if the lattice stays unchanged p f[p] qi pi

20. t s Free Form Deformation • Finding weights (2D) • Let the lattice points be pi,j for i=0,…,k and j=0,…,l • Compute p’s relative location in the grid (s,t) • Let (xmin,xmax), (ymin,ymax) be the range of grid p0,2 p1,2 p2,2 p3,2 p p0,1 p1,1 p2,1 p3,1 p0,0 p1,0 p2,0 p3,0

21. Free Form Deformation • Finding weights (2D) • Let the lattice points be pi,j for i=0,…,k and j=0,…,l • Compute p’s relative location in the grid (s,t) • The weight wi,j for lattice point pi,j is: • i,j: importance of pi,j • B: Bernstein basis function: p0,2 p1,2 p2,2 p3,2 p t p0,1 p1,1 p2,1 p3,1 p0,0 p1,0 p2,0 p3,0 s

22. Free Form Deformation • Finding weights (2D) • Weight distribution for one control point (max at that control point): p3,2 p3,1 p1,1 p0,2 p3,0 p0,1 p2,0 p1,0 p0,0

23. Free Form Deformation • A deformation example

24. Free Form Deformation • Registration • Embed the source in a lattice • Compute new lattice positions over the target • Fitting term • Distance between deformed source shape and the target shape • Agreement of the deformed image content and the target image • Distortion term • Thin-plate spline energy • Non-folding constraint • Local rigidity constraint (to prevent stretching)

25. Free Form Deformation • Registration example Deformed w/o rigidity Source Deformed with rigidity Target Credits: Loeckx et al, MICCAI (2004)

26. Free Form Deformation • Advantages • Smooth deformations • Easy to implement (no equation solving) • Efficient and localized controls for interactive editing • Can be coupled with different fitting or energy objectives • Disadvantages • Too many lattice points in 3D

27. Cage-based Deformation • Use a control mesh (“cage”) to embed the shape • Deforming the cage vertices warps the embedded shape Credits: Ju, Schaefer, and Warren, SIGGRAPH (2005)

28. Cage-based Deformation • Warping the space by “blending” the deformation at the cage vertices • wi[p]: determined by relative position of p with respect to pi pi p qi f[p]

29. Cage-based Deformation • Finding weights (2D) • Problem: given a closed polygon (cage) with vertices pi and an interior point p, find weights wi[p] such that: • 1) • 2) pi p

30. Cage-based Deformation • Finding weights (2D) • A simple case: the cage is a triangle • The weights are unique, and are the barycentric coordinates of p p1 p p3 p2

31. Cage-based Deformation • Finding weights (2D) • The harder case: the cage is an arbitrary (possibly concave) polygon • The weights are not unique • A good choice: Mean Value Coordinates (MVC) • Can be extended to 3D pi-1 pi αi αi+1 p pi+1

32. Cage-based Deformation • Finding weights (2D) • Weight distribution of one cage vertex in MVC: pi

33. Cage-based Deformation • Application: character animation • Using improved weights: Harmonic Coordinates

34. Cage-based Deformation • Registration • Embed source in a cage • Compute new locations of cage vertices over the target • Minimizing some fitting and energy objectives • Not seen in literature yet… future work!

35. Cage-based Deformation • Advantages over free form deformations • Much smaller number of control points • Flexible structure suitable for organic shapes • Disadvantages • Setting up the cage can be time-consuming • The cage needs to be a closed shape • Not as flexible as point handles

36. Further Readings • Thin-plate spline deformation • “Principal warps: thin-plate splines and the decomposition of deformations”, by Bookstein (1989) • “Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines”, by Rohr et al. (2001) • Free form deformation • “Free-Form Deformation of Solid Geometric Models”, by Sederberg and Parry (1986) • “Extended Free-Form Deformation: A sculpturing Tool for 3D Geometric Modeling”, by Coquillart (1990) • Cage-based deformation • “Mean value coordinates for closed triangular meshes”, by Ju et al. (2005) • “Harmonic coordinates for character animation”, by Joshi et al. (2007) • “Green coordinates”, by Lipman et al. (2008)