1 / 17

Automated Face Replacement System: Digital Photographic Replacement

This project focuses on developing an automated system for replacing faces in photographs, revolutionizing the process currently done manually by graphic artists. Key components include face detection, head pose estimation, illumination extraction, facial expression synthesis, and merging/replacement algorithms. By considering light estimation, skin tone approximation, 3D model fitting, basis image generation, and color blending techniques, the system achieves realistic results. The system enables various applications in Hollywood special effects, personalized movies, and more. Further research areas include enhancing face detection and tracking for videos, improving expression synthesis, refining reflectance models, and advancing model-fitting accuracy.

jackiebennett
Download Presentation

Automated Face Replacement System: Digital Photographic Replacement

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Digital Face Replacement in Photographs CSC2530F Project Presentation By: Shahzad Malik January 28, 2003

  2. Face Replacement Motivation • Currently done manually by graphic artists using photo editing software • An automatic system has many potential uses: • Hollywood special effects • “Personalized” movies • Framing someone…

  3. Required Components • Need the following subsystems: • Face detection (and tracking for videos) • Head pose estimator • Illumination extractor (*) • Facial expression synthesis • Merging/replacement algorithm (*)

  4. Light Estimation • Assuming a Lambertian reflectance model: • Any image can then be represented by:

  5. Approximate Skin Tone • Cannot assume 3 basis images for arbitrary photographs • Use an approximate image to generate basis

  6. Fitting a Generic 3D Model • Need geometry to create basis images • Fit a generic 3D face mesh to images • “Lift” a texture using planar mapping

  7. Generate Basis Images • Set 3 linearly independent light positions • Relight skin tone model with each light

  8. Determining the Coefficients • Compute a least squares solution to: • Solve separately for each RGB channel

  9. Re-illuminating the Target Face • Set intensities of the 3 light sources to the coefficient values • Render the target face with these lights

  10. Flesh Pixel Detection • Match non-mesh skin pixels to new skin tone • Use a histogram-based skin classifier

  11. Histogram Matching • Generate histograms for newly lit face • Match the Gaussian distribution from original face to newly lit face • For each flesh pixel in original image, choose a new color with a similar location on the Gaussian bell curve

  12. Weighted Color Blending • Blend converted flesh pixels with face mesh pixels

  13. Results

  14. Results (continued)

  15. Results (continued)

  16. Results (continued)

  17. Summary • Presented a face replacement system • Takes lighting and merging into account • Future research areas: • Face detection and tracking (for videos) • Expression synthesis • More sophisticated reflectance model • Automatic and precise model-fitting

More Related