1 / 26

A Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model

A Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model. Ce Liu Heung-Yeung Shum Chang- Shui Zhang By 吴昊东. Author. Researcher Microsoft Research New England Research Affiliate Computer Science and Artificial Intelligence Laboratory (CSAIL)

deana
Download Presentation

A Two-Step Approach to Hallucinating Faces: Global Parametric Model and Local Nonparametric Model

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Two-Step Approach to Hallucinating Faces:Global Parametric Model and Local Nonparametric Model CeLiu Heung-Yeung Shum Chang-ShuiZhang By 吴昊东

  2. Author Researcher Microsoft Research New England Research Affiliate Computer Science and Artificial Intelligence Laboratory (CSAIL) Massachusetts Institute of Technology Adjunct Assistant Professor Computer Science Department, Boston University Ce Liu Homepage: http://research.microsoft.com/en-us/um/people/celiu/ (Microsoft) http:/people.csail.mit.edu/celiu/ (MIT) Workshops Chair of CVPR 2013

  3. Face Hallucination • Face super-resolution (FSR) • Face sketch-photo synthesis (FSPS) techniques A Comprehensive Survey to Face Hallucination, IJCV 2013

  4. Face super-resolution • Reconstruction-based • Input images alone • Learning-based • Learn by other HR & LR pairs of images (or only HR images)

  5. constraints • Sanity constraint. The result must be very close to the input image when smoothed and down-sampled. • Global constraint. The result must have common characteristicsof a human face, e.g. eyes, mouth and nose, symmetry, etc. • Local constraint. The result must have specific characteristics of this face image with photorealistic local features.

  6. constraints • Sanity constraint • Easily satisfied • Global constraint • Without -> noisy • Local constraint • Without -> too smooth, close to average face

  7. Two-Step Approach • Title: • A Two-Step Approach to Hallucinating Faces:Global Parametric Model and Local Nonparametric Model • Global + Local

  8. Bayesian Formulation • Smoothing and down-sampling • The same as averaging pooling* • If are vectors, • Reverse it!

  9. Bayesian Formulation • maximum a posteriori (MAP) criterion

  10. Global and Local Modeling of Face • Global + Local • Since is the low-frequency part of • Regard as soft constraint to

  11. Target

  12. The Method • Decouple high-resolution face image to two parts — high resolution face image — global face — local face • Two-step Bayesian inference = + 1. Inferring global face 2. Inferring local face Finally adding them together 1. Inferring global face 2. Inferring local face Finally adding them together ? ?

  13. Global Modeling • Apply PCA to training face images

  14. Global Modeling • is very close to human face with some smoothness • Calculate quickly using input

  15. Local Modeling • Patch-based nonparametric Markov network • Square patch size: & overlap • means • means (1,0) • means (0,1)

  16. Local Modeling • Assume that the above network is a Markov network • Suppose Gibbs distribution

  17. Local Modeling

  18. Local Modeling • Training pairs is the distance metric between two patches in • Laplacian image of

  19. Local Modeling • Total Energy

  20. Simulated Annealing

  21. Experimental Results • Database • FERET data set • AR data set • Other collections • Align the face image

  22. Experimental Results

  23. Experimental Results • Compete with other paper

  24. Conclusion • Combine global and local constrains • robustness and efficiency • Why not complex model • the error can be compensated in the local model • Patch-based nonparametric Markov network • Nonparametric -> accuracy • patch-based -> high efficiency • Simulated Annealing • Similar to Gibbs sampling but converges more quickly • Generalization -> Other super-resolution problem

  25. The End THANK YOU! Q&A

More Related