250 likes | 414 Views
CS4243 Project: Face Morphing 7. Caitlin, Michael, Wai Tsun, Wei Qi. Objectives. Achieve a seamless transition between faces regardless of face shape Features should be aligned while being warped. Requirements.
E N D
CS4243 Project: Face Morphing 7 Caitlin, Michael, Wai Tsun, Wei Qi
Objectives • Achieve a seamless transition between faces regardless of face shape • Features should be aligned while being warped Requirements • Landmark points should be marked rigorously (each important feature should have one) • Images can be frontal/non-frontal
Requirements • Landmark points should be marked rigorously (each important feature should have one) • Images can be frontal/non-frontal
Algorithm • 1. Landmark points/tesselation • 2. Piecewise warp two-way • 3. Color interpolation • 4. Remapping using linear interpolation
Landmark points • Marked manually • Tesselate using Delaunay triangulation • To facillate more control of transformation
Piecewise warp • For each triangle: • Interpolate src and dest points according to ratio t: in_between. Transform source image by doing a transformation from src to in_between. Transform source image by doing a transformation from in_between to dest. • Helps to align features compared to only warping one way
Color interpolation • Blend the two intermediate images according to ratio t. • Interpolation function is quadratic... so that it will stay at the source face for a longer period of time. • But the interpolation function can be adjusted for other effects
Remapping • Backward mapping using linear interpolation to determine final color • Because coordinates may not be integer after transform
Choice of transformation function • At first, tried Affine: features don’t align well • Polynomial: overfitting • Piecewise polynomial: couldn’t get it to work • Piecewise thin-plate spline: works well!
Thin-plate spline • Modeled after thin-metal plate • So should be a good model for face • Borrowed Python snippet and verified using Bookstein ’89 “Principal Warps: Thin-Plate Splines and the Decomposition of Deformations”
Sample models • Only compare two in the slides; more in video
Landmark points • Only on face; can be improved
Delaunay triangulation • Add corners so that whole image is transformed;
Global Affine • Features not aligned! Background warped
Global Polynomial • Features aligned, but strange deformation; Background warped • Quite good for global though
Global Thin-Plate Spline • Similar to global polynomial;
Piecewise Polynomial • Oops!
Piecewise Thin-plate Spline (Frontal) • Looks seamless! And background doesn’t warp • Didn’t put landmark points on garb…
Videos • More faces morphed in the videos: • Affine: http://www.youtube.com/watch?v=IxhHRjTqPpU • Poly: http://www.youtube.com/watch?v=SkBH9mDVC80 • TPS: http://www.youtube.com/watch?v=nC_6DDG9a3U • Piecewise TPS(non-frontal): http://www.youtube.com/watch?v=ZRmIsB-gpsc • Piecewise TPS (frontal): http://www.youtube.com/watch?v=SMbzd7FmAFY
Met Objectives? • Achieve a seamless transition between faces regardless of face shape • Features should be aligned while being warped; but blending of hair etc could be better Met Requirements • Landmark points should be marked rigorously (each important feature should have one); should mark non-face features as well • Images can be frontal/non-frontal
Credits • Faces: http://pics.psych.stir.ac.uk/2D_face_sets.htm • Landmark points marking: from lecturer • Transformation matrices: • Affine: using OpenCV library (Geometric Transforms) • Polynomial: self-implemented using numpy to multiply matrices, because the one from scikit-image didn’t work properly • Piecewise thin-plate spline: • Delaunay triangulation done using scipy.spatial library • Thin-plate spline adapted from snippet by Zachary Pincus, and verified from original paper by Bookstein ’89 “Principal Warps: Thin-Plate Splines and the Decomposition of Deformations” • Transformation: • remapping done using OpenCV library (Geometric Transforms) • Color interpolation: • OpenCV’s AddWeighted