Download
1 / 27
270 likes | 285 Views
Explore the dimensions and basis for image spaces, lighting reflectance, Spherical Harmonics, and accuracy of approximation in recognizing objects under varying lighting conditions in computer vision. Learn about the empirical study results and experiments that shed light on object recognition accuracy.
E N D
What is the Question ?(based on work of Basri and Jacobs, ICCV 2001) Given an object described by its normal at each surface point and its albedo (we will focus on Lambertian surfaces) 1.What is the dimension of the space of images that this object can generate given any set of lighting conditions ? 2. How to generate a basis for this space ?
Ball Face Phone Parrot #1 48.2 53.7 67.9 42.8 #3 94.4 90.2 88.2 76.3 #5 97.9 93.5 94.1 84.7 #7 99.1 95.3 96.3 88.5 #9 99.5 96.3 97.2 90.7 Empirical Study (Epstein, Hallinan and Yuille; see also Hallinan; Belhumeur and Kriegman) Dimension:
Domain Domain Lambertian No cast shadows (“convex” objects) Lights are distant n l q
+ + + Lighting to Reflectance: Intuition Lambert Law k(q)= max(cosq, 0)
Lighting to Reflectance: Intuition Three point-light sources, l(q,f), Illuminating a sphere and its reflection r(q,f). Profiles of l(q)and r(q)
Lighting Reflectance where Images ... ...
Spherical Harmonics (S.H.) • Orthonormal basis, , for functions on the sphere. • n’th order harmonics have 2n+1 components. • Rotation = phase shift (same n, different m). • In space coordinates: polynomials of degree n.
k S.H. analog to convolution theorem • Funk-Hecke theorem: “Convolution” in function domain is multiplication in spherical harmonic domain.filter.
Energy of Lambertian Kernel in low order harmonics k-is a low pass filter
Reflectance Functions Near Low-dimensional Linear Subspace Yields 9D linear subspace.
Forming Harmonic Images l lZ lX lY lXY lXZ lYZ
How accurate is approximation?Point light source 9D space captures 99.2% of energy
How accurate is approximation? Worst case. • DC component as big as any other. • 1st and 2nd harmonics of light could have zero energy 9D space captures 98% of energy
How Accurate Is Approximation? • Accuracy depends on lighting. • For point source: 9D space captures 99.2% of energy • For any lighting: 9D space captures >98% of energy.
Accuracy of Approximation of Images • Normals present to varying amounts. • Albedo makes some pixels more important. • Worst case approximation arbitrarily bad. • “Average” case approximation should be good.
Summary • Convex, Lambertian objects: 9D linear space captures >98% of reflectance. • Explains previous empirical results. • For lighting, justifies low-dim methods.
Recognition Find Pose Harmonic Images Query Compare Matrix: B Vector: I Models
Experiments (Basri&Jacobs) • 3-D Models of 42 faces acquired with scanner. • 30 query images for each of 10 faces (300 images). • Pose automatically computed using manually selected features (Blicher and Roy). • Best lighting found for each model; best fitting model wins.
Results • 9D Linear Method: 90% correct. • 9D Non-negative light: 88% correct. • Ongoing work: Most errors seem due to pose problems. With better poses, results seem near 100%.
