1 / 38

3D Face Modelling and Recognition Survey

3D Face Modelling and Recognition Survey. Timur Aksoy Biometrics Course Fall 2011 Sabanci University. Outline. Background Approaches Average Face Model Iterative Closest Point Algorithm Regional Models Hybrid 2D/3D Methods Conclusions. Background .

glenna
Download Presentation

3D Face Modelling and Recognition Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D FaceModellingandRecognitionSurvey Timur Aksoy BiometricsCourse Fall 2011 SabanciUniversity

  2. Outline • Background • Approaches • AverageFace Model • IterativeClosestPointAlgorithm • RegionalModels • Hybrid 2D/3D Methods • Conclusions

  3. Background • GalleryImagesare of personsenrolled in thesystem • Probesaretheimagestoidentifyorauthenticate. • Onetoonematchforveriification, onetomanyforidentification. • 3D model can be renderedeither as depthimage, shaded model or mesh of points. • 2D image can be thought of texturemap of 3D image. • Rangeimagesare 2.5D as theyonlygivedepthvalues of x-y coordinatesfromoneviewpoint. • 3D imagesscantheheadcompletelyandusuallyrepresentedbypolygonal mesh orcloud of points

  4. 3D vs 2D 2D detectionperformancedegradesunder • posechanges, • nonrigidmotions, • changes in illumination • occlusion • aging. 3D detection is consideredto be effectedlessbythesefactors.

  5. Approaches • PointClouds • DepthbasedfromRangeImages • GeometricMethods • Differential • Global • Local • ShapeDescriptors

  6. PointClouds (ICP) • Point Set: Pointswith x,y,z coordinatesrepresentthesurface. • Probeface is registeredtogalleryfacesusingIterativeClosestPointAlgorithmand dense matchbetweenfeaturesandpoints is obtained. • Mostsystemsuse a referenceortemplatefaceforalignmentwith ICP (e.g. AverageFace) sothatalignment is performedonlyonce. • ICP onlyaccountsforrigidmotion. Itultimatelyfindspointtopointcorrespondence.

  7. PointClouds

  8. PointClouds (Nonrigid) • Landmarksareusedforpre-alignmentex: ProcrustesAnalysis. • Afterrigidalignment, facemay be deformedbysplinesto fit thereferenceface. B-SplinesandThin-PlateSplinesarecommon. • Nonrigidregistration is neededforrobustnessagainstexpressions. • Deformation can be performed on bothlocaland global scales. • Subregions of facemay be matchedindependently.

  9. PointClouds (Similarity) • Afterfitting, pointsmay be sampledfrombothsurfacesaccordingtoaverageface model • Point set differencesmay be usedfordissimilaritymeasureafterregistration. • Alternatively, Hausdorffdistancesmay be usedfordistancefunctionbetween 2 pointsets. • Hausdorffdistancemeasuresthemaximum of thesmallestdistancesbetweenthepoints in bothsets. • PCA can also be applied on PointCloudandnewaxeswithmaximumvariancearedetermined.

  10. DepthImageBased • Depthimageintensitiesindicatethedepth at thatcoordinate. • 2D imagematchingtechniques can be usedsuch as PCA, LDA and ICA. • Posesarefirstnormalizedusingfeaturesor ICP.

  11. RangeImages

  12. GeometricMethods (Profiles) • Curvesareextractedfromsurfaceandused in 2D analysis • CentralandLateralprofilesmay be usedforrecognition. • IterativeConditionalMode (ICM) optimization is usedformatchingprofiles. • Curvaturesalongprofilesused as features. • VerticalSymmetry Profile curveseen as mostdiscriminative.

  13. Seven verticalprofiles

  14. GeometricMethods (Curvature) • Curvature of surfacesarecomputed. • Extractlandmarksusingcurvature (i.e. Gaussiancurvature) andsegmenttheface • Gaussiancurvature is theproduct of twoprinciplecurvatures at onepoint. • Maximumand minimum principlecurvaturesmay be representedbytwoEnhancedGaussianImages. • ExtendedGaussian Image (EGI), constructed by mapping principalcurvatures and their directions at each surface points, ontotwo unit spheres. Theirvaluesaremappedtolengths of vectorslikehistogram. • Similaritymatch is performedbyFisher’ssphericalapproximation on EGI’s

  15. PrincipleCurvatures of Saddle

  16. ExtendedGaussianImage

  17. GeometricMethods (others) • Normals on thesurfacemayalso be computed as 3 componentvectors. • Differencebetweennormals of 2 imagesmay be computedbyangledifferencehistograms. • Landmarkdistancesandangles can also be computed as features • SVM basedclassifiers can classifyfacesaccordingtothegeometricfeatures. • Expressionresistantregionssuch as noseandeyeshavehighrecognitionratesandtheirscoresmay be fused. • Forehead, jawline, eyecornercavitiesandcheeks of samepersonremainssame in differentrangeimages. • Localcurvaturesarealsoresistanttonon-rigidmotion since expressionchangesthefaceglobally.

  18. SurfaceNormals

  19. ShapeDescriptor-based • Localand global facialfeaturesareextracted • PointSignaturesare popular andused in coarseregistrationandregiondetection. • PointSignaturesmay be computed on rigidregionsandcomparedforexpressionresistance. • A mesh is fittedtothepointcloudandshapedescriptorsareextractedandtheirmomentsarecomputed. • SphereSpinImagesdescribethesurfacelocally.

  20. PointSignatures • Draw a spherearoundpoint p of surface. • Intersectspherewithsurfaceandget a spacecurve C whichmay not be planar. • Fit a plane P tothiscurveapproximately. • Callits normal n1 andtranslatetheplanealong n1 toincludepoint p. Calltheplane P2 andproject C to P2 to form a newcurve C2. • Distance of points on C2 form signeddistance profile • Directionfrom p topoint on C2 withlargestpositivedistance is unitvector n2. Theta is thecwangle of point on C2 about n1 from n2. • Wediscretizethetaandrecordsigneddistanceswhichproducesthepointsignature.

  21. PointSignature

  22. SphereSpinImages (SSI) • An SSI, associated with a point on the surface, is a 2D histogram constructed fromdistances of otherpoints in itsneighborhood surface falling in thespherecentered at thatpoint • Itcaptures the characteristic of local shape. • Usuallysmall set of pointsareselectedfor SSI analysisby minimum principlecurvature. • Similaritybetweenseries of SSI foreachsubject is measuredby a simplecorrelationcoefficient.

  23. FeatureExtraction • Mostapproachesusecoarsetofinelocalization. • Simplestmethodslocatevalleyssuch as eyesocketsandthepeak as thenosehowevertheyassumenormalizedfaces. • Appereancebasedmodelsprojectimageintosubspacewith PCA, ICA, DCT, GaussianDerivativeFiltersorGaborWavelets • Geometric-based methods use angles, distances andareasbetweenlandmarks • In the structure-based methods, the ensemble of candidate landmarks isfitted to a model of feature locations and the likelihood is considered.

  24. TheWork of BosphorusGroup BosphorusUniversity ComputerEngineering

  25. AverageFace Model • Instead of aligningeachfaceseparatelycreateAverageFace Model fromgallery. • Instead of registeringprobetoeverygalleryimage , register it tothe AFM. MultipleAFM’sarealsopossible. • Fewlandmarksareextracted. Thenprobe is firstcoarselyregistered, rigidlyregistered (ICP) thennon-rigidly (TPS). • Theprobeimage is resampledafterregistering.

  26. AFM and 7 Landmarks

  27. ICP • Dense correspondence is establishedandthepointswithoutcorrespondenceorwithhighdistanceareremoved. • Theobjectivefunctionto be minimized is: where is quaternionrotationmatrix, is translationvector, is measured data set and is the model data set. Pointswiththesameindicescorrespondtoeachotherandpointsetshavesame size.

  28. Estimation of RotationandTranslation Thecrossvariance of sets P and X is givenby

  29. TPS WarpedFace (a) AFM, (b) the original face, (c) the warped/cropped version of the originalface, and (d) TPS warped and registered facial surfaces.

  30. Construction of AFM • Consensuslandmarkdistancesarecomputed • Landmark of Consensusshape set tofullfrontalposition • Allshapesarewarpedtoconsensusby TPS • Depthvaluesareresampledfrominterpolatedfaces • Imagesarecroppedbymasking • Alldepthvaluesareaveragedtoobtain AFM.

  31. RegionalFace Model • Landmarksaredetectedautomaticallywithfacialsymmetryaxisbyusingprinciplecurvatures (maximum, minimum, Gaussian). • RegionalRegistration of patchesprecededby global is performedby ICP independentlytotheAverageFace Model. • PointCoordinatefeatures i.e. depthvaluesareextracted • LDA is applied on pointcoordinates of eachregiontoobtainadditionalfeatures in transformedspace. • Normal CurvatureDescriptorsarealsoobtainedbycomputingprinciplecurvaturesanddissimilarity is computedfromthem. • Classifierresultsfromdifferentregionsarefused.

  32. 2D/3D Hybridmethods • Informationcomingfrom 2D and 3D arecombined. • Combination of multipleimagesincreasetheaccuracy. • 2D imagemappedto 3D canonicalsurface as texture • ICP wasperformed on 4D data (3D + intensity) whichuses 4D EuclideanDistance • IntensityandDepth data has beenintegratedby HMM.

  33. Conclusion • 3D facemodeling is morerobustagainstenvironmentalconditions. • Multimodalachievesbetterperformance in general. • A methodthatperformsbestunderallcircumstances is not known yet.

  34. References • Andrea F. Abate et al. 2D and 3D face recognition: A survey. PatternRecognitionLetters, 28, 2007. • Kevin W. Bowyer , Kyong Chang, Patrick Flynn. A survey of approaches and challenges in3D and multi-modal 3D + 2D face recognition. ComputerVisionandImageUnderstanding, 101, 2006. • Berk Gökberk. ThreeDimensionalFaceRecognition. PhDThesis, ComputerEngineering, BosphorusUniversity, 2006. • Neşe Alyüz, Berk Gökberk, and Lale Akarun. RegionalRegistrationforExpressionResistant 3-D FaceRecognition. IEEE Transactions On InformatıonForensics And Security, Vol. 5, No. 3, 2010.

  35. References • Albert Ali Salah, Neşe Alyüz, Lale Akarun. Registration of three-dimensional face scanswithaveragefacemodels. Journal of ElectronicImaging, vol. 17(1), 2008. • Hatice Çınar Akakın et al. 2D/3D FacialFeatureExtraction. Proceedings of SPIE-IS&T ElectronicImaging, SPIE Vol. 6064. • Chin-SengChua, Feng Han, Yeong-KhingHo. 3D Human Face Recognition Using Point Signature. Proceedings of Fourth IEEE InternationalConference on AutomaticFaceandGestureRecognition. 2000.

More Related