1 / 22

CS4243 Project: Face Morphing 7

CS4243 Project: Face Morphing 7. Caitlin, Michael, Wai Tsun, Wei Qi. Objectives. Achieve a seamless transition between faces regardless of face shape Features should be aligned while being warped. Requirements.

victor-rich
Download Presentation

CS4243 Project: Face Morphing 7

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS4243 Project: Face Morphing 7 Caitlin, Michael, Wai Tsun, Wei Qi

  2. Objectives • Achieve a seamless transition between faces regardless of face shape • Features should be aligned while being warped Requirements • Landmark points should be marked rigorously (each important feature should have one) • Images can be frontal/non-frontal

  3. Requirements • Landmark points should be marked rigorously (each important feature should have one) • Images can be frontal/non-frontal

  4. Algorithm • 1. Landmark points/tesselation • 2. Piecewise warp two-way • 3. Color interpolation • 4. Remapping using linear interpolation

  5. Landmark points • Marked manually • Tesselate using Delaunay triangulation • To facillate more control of transformation

  6. Piecewise warp • For each triangle: • Interpolate src and dest points according to ratio t: in_between. Transform source image by doing a transformation from src to in_between. Transform source image by doing a transformation from in_between to dest. • Helps to align features compared to only warping one way

  7. Color interpolation • Blend the two intermediate images according to ratio t. • Interpolation function is quadratic... so that it will stay at the source face for a longer period of time. • But the interpolation function can be adjusted for other effects

  8. Remapping • Backward mapping using linear interpolation to determine final color • Because coordinates may not be integer after transform

  9. Choice of transformation function • At first, tried Affine: features don’t align well • Polynomial: overfitting • Piecewise polynomial: couldn’t get it to work • Piecewise thin-plate spline: works well!

  10. Thin-plate spline • Modeled after thin-metal plate • So should be a good model for face • Borrowed Python snippet and verified using Bookstein ’89 “Principal Warps: Thin-Plate Splines and the Decomposition of Deformations”

  11. Sample models • Only compare two in the slides; more in video

  12. Landmark points • Only on face; can be improved

  13. Delaunay triangulation • Add corners so that whole image is transformed;

  14. Global Affine • Features not aligned! Background warped

  15. Global Polynomial • Features aligned, but strange deformation; Background warped • Quite good for global though

  16. Global Thin-Plate Spline • Similar to global polynomial;

  17. Piecewise Polynomial • Oops!

  18. Piecewise Thin-plate Spline (Non-Frontal) • Breaks…

  19. Piecewise Thin-plate Spline (Frontal) • Looks seamless! And background doesn’t warp • Didn’t put landmark points on garb…

  20. Videos • More faces morphed in the videos: • Affine: http://www.youtube.com/watch?v=IxhHRjTqPpU • Poly: http://www.youtube.com/watch?v=SkBH9mDVC80 • TPS: http://www.youtube.com/watch?v=nC_6DDG9a3U • Piecewise TPS(non-frontal): http://www.youtube.com/watch?v=ZRmIsB-gpsc • Piecewise TPS (frontal): http://www.youtube.com/watch?v=SMbzd7FmAFY

  21. Met Objectives? • Achieve a seamless transition between faces regardless of face shape • Features should be aligned while being warped; but blending of hair etc could be better Met Requirements • Landmark points should be marked rigorously (each important feature should have one); should mark non-face features as well • Images can be frontal/non-frontal

  22. Credits • Faces: http://pics.psych.stir.ac.uk/2D_face_sets.htm • Landmark points marking: from lecturer • Transformation matrices: • Affine: using OpenCV library (Geometric Transforms) • Polynomial: self-implemented using numpy to multiply matrices, because the one from scikit-image didn’t work properly • Piecewise thin-plate spline: • Delaunay triangulation done using scipy.spatial library • Thin-plate spline adapted from snippet by Zachary Pincus, and verified from original paper by Bookstein ’89 “Principal Warps: Thin-Plate Splines and the Decomposition of Deformations” • Transformation: • remapping done using OpenCV library (Geometric Transforms) • Color interpolation: • OpenCV’s AddWeighted

More Related