Machine Vision Laboratory The University of the West of England Bristol, UK

1. 2nd September 2010 ICEBT Machine Vision Laboratory The University of the West of England Bristol, UK Photometric Stereo in 3D Biometrics Gary A. Atkinson PSG College of Technology Coimbatore, India

2. Overview What is photometric stereo? Why photometric stereo? The basic method Advanced methods Applications

3. Overview What is photometric stereo? Why photometric stereo? The basic method Advanced methods Applications

4. 1. What is Photometric Stereo? A method of estimating surface geometry using multiple light source directions Captured images

5. 1. What is Photometric Stereo? A method of estimating surface geometry using multiple light source directions Captured images Surface normals

6. 1. What is Photometric Stereo? A method of estimating surface geometry using multiple light source directions Surface normals Depth map

7. 1. What is Photometric Stereo? A method of estimating surface geometry using multiple light source directions Surface normals Depth map

8. Overview What is photometric stereo? Why photometric stereo? The basic method Advanced methods Applications

9. 2. Why photometric Stereo? Why 3D? Why PS in particular?

10. 2. Why photometric Stereo? Why 3D? Why PS in particular? Robust face recognition Ambient illumination independence Facilitates pose correction Non-contact fingerprint analysis Richer dataset in 3D

11. 2. Why photometric Stereo? Why 3D? Why PS in particular? Robust face recognition Ambient illumination independence Facilitates pose, expression correction Non-contact fingerprint analysis • Same day • Same camera • Same expression • Same pose • Same background • Even same shirt • Different illumination Completely different image

12. 2. Why photometric Stereo? Why 3D? Why PS in particular? Robust face recognition Ambient illumination independence Facilitates pose, expression correction Non-contact fingerprint analysis

13. 2. Why photometric Stereo? Why 3D? Why PS in particular? Robust face recognition Ambient illumination independence Facilitates pose, expression correction Non-contact fingerprint analysis

14. 2. Why photometric Stereo? Why 3D? Why PS in particular? Captures reflectance data for 2D matching Compares well to other methods *Depending on correspondence density SFS Shape-from-shading GS Geometric stereo LT Laser triangulation PPT Projected pattern triangulation PS Photometric stereo

15. 2. Why Photometric Stereo? – Alternatives Shape-from-Shading Estimate surface normals from shading patterns.  More to follow...

16. 2. Why Photometric Stereo? – Alternatives Geometric stereo

17. 2. Why Photometric Stereo? – Alternatives Geometric stereo [Li Wei, and Eung-Joo Lee, JDCTA 2010] [Wang, et al. JVLSI 2007]

18. 2. Why Photometric Stereo? – Alternatives Laser triangulation

19. 2. Why Photometric Stereo? – Alternatives Laser triangulation General case

20. 2. Why Photometric Stereo? – Alternatives Laser triangulation

21. 2. Why Photometric Stereo? – Alternatives Projected Pattern Triangulation

22. 2. Why Photometric Stereo? – Alternatives Projected Pattern Triangulation Target Greyscale camera Pattern projector Colour camera Flash Greyscale camera

23. 2. Why Photometric Stereo? – Alternatives Projected Pattern Triangulation

24. 2. Why Photometric Stereo? – Alternatives Projected Pattern Triangulation

25. 2. Why Photometric Stereo? – Alternatives Projected Pattern Triangulation

26. 2. Why Photometric Stereo? – Alternatives Projected Pattern Triangulation

27. Overview What is photometric stereo? Why photometric stereo? The basic method Advanced methods Applications

28. 3. The Basic Method – Assumptions Assumptions: • No cast/self-shadows or specularities • Greyscale/linear imaging • Distant and uniform light sources • Orthographic projection • Static surface • Lambertian reflectance [Argyriou and Petrou] θ I= radiance (intensity) N = unit normal vector L = unit source vector θ = angle between N and L

29. 3. The Basic Method – Preliminary notes Equation of a plane Surface normal vector to plane

30. 3. The Basic Method – Preliminary notes Equation of a plane Surface normal vector to plane

31. 3. The Basic Method – Preliminary notes Equation of a plane Surface normal vector to plane

32. 3. The Basic Method – Preliminary notes Equation of a plane Surface normal vector to plane Rescaled surface normal

33. 3. The Basic Method – Preliminary notes Equation of a plane Surface normal vector to plane Rescaled surface normal typically written

34. 3. The Basic Method – Reflectance Equation Consider the imaging of an object with one light source.

35. 3. The Basic Method – Reflectance Equation Lambertian reflection E= emittance r = albedo L = irradiance

36. 3. The Basic Method – Reflectance Equation Lambertian reflection E= emittance r = albedo L = irradiance Take scalar product of light source vector and normal vector to obtain cosq:

37. 3. The Basic Method – Shape-from-Shading Perform substitution with Lambert’s law

38. 3. The Basic Method – Shape-from-Shading Perform substitution with Lambert’s law If r is re-defined and the camera response is linear, then

39. 3. The Basic Method – Shape-from-Shading Perform substitution with Lambert’s law If r is re-defined and the camera response is linear, then Known: ps, qs, I Unknown: p, q, r 1 equation, 3 unknowns  Insoluble

40. 3. The Basic Method – Shape-from-Shading For a given intensity measurement, the values of p and q are confined to one of the lines (circles here) in the graph. Curves of constant I/r Increasing I/r

41. 3. The Basic Method – Shape-from-Shading For a given intensity measurement, the values of p and q are confined to one of the lines (circles here) in the graph. Increasing I/r Shadow boundary This is the result for a different light source direction.

42. 3. The Basic Method – Shape-from-Shading Representation using conics Possible cone of normal vectors forming a conic section.

43. 3. The Basic Method – Shape-from-Shading Representation on unit sphere We can overcome the cone ambiguity using several light source directions: Photometric Stereo

44. 3. The Basic Method – Two Sources Two sources: normal confined to two points – Or fully constrained if the albedo is known

45. 3. The Basic Method – Three Sources Three sources: fully constrained

46. 3. The Basic Method – Matrix Representation Unit surface normal Unit light source vector (using slightly different symbols to aid clarity)

47. 3. The Basic Method – Matrix Representation Unit surface normal Unit light source vector (using slightly different symbols to aid clarity)

48. 3. The Basic Method – Matrix Representation Unit surface normal Unit light source vector (using slightly different symbols to aid clarity)

49. 3. The Basic Method – Matrix Representation Recall that, based on the surface gradients,

50. 3. The Basic Method – Matrix Representation Recall that, based on the surface gradients, Substituting and re-arranging these equations gives us That is, the surface normals/gradient and albedo have been determined.