1 / 40

What Does Motion Reveal About Transparency ?

What Does Motion Reveal About Transparency ?. Moshe Ben-Ezra and Shree K. Nayar Columbia University ICCV Conference October 2003, Nice, France. This work was supported by an NSF ITR Award IIS-00-85864. Transparency is Very Challenging. Existence of a transparent object.

azure
Download Presentation

What Does Motion Reveal About Transparency ?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What Does Motion Reveal About Transparency ? Moshe Ben-Ezra and Shree K. Nayar Columbia University ICCV Conference October 2003, Nice, France This work was supported by an NSF ITR Award IIS-00-85864

  2. Transparency is Very Challenging • Existence of a transparent object. • Finding its shape and pose

  3. V1 V2 V1 V2 F` V1 F V2   F` Lambertian F Transparent Specular Real and Virtual Features

  4. Environmental Matting* Alternating pattern Object Camera • Does not recover shape and pose. • Requires controlled environment. * Zongker, el al. SIGGRAPH 99,

  5. Shape from Polarization in Highlight* Camera N Light Rotating Polarizer Object • Limited to a single interface at the object’s surface. • Requires controlled environment. * Saito et al. CVPR’99.

  6. Shape from Refraction and Motion* Camera Water Fixed Pattern • Single interface only. * H. Murase. PAMI, 1992

  7. Motion is Key to Transparency

  8. Transparent Shape From Motion And a Parametric Model(such as super-ellipse) Given: Views Recover: Shape: Values of parameters (e, n) Pose: RotationR, TranslationT General analytic solution does not exist.

  9. Transparency From Motion Distant feature Reversed rays are parallel to each other regardless of the complexity of their paths

  10. Approach: Initialization Image Plane Image Plane

  11. Approach: Initial Guess

  12. Approach: Refine

  13.  - Object’s shape parameter vector • R,T - Object’s pose Error Function r1,1 .. r1,n (0,0,1) r2,1 .. r2,n

  14. Simulation Setup Parallel rays from features Transparent object Camera side rays

  15. Example (Simulation) Initial Guess Symmetric Superellipse (n=e) Single Parameter. Newton-Raphson optimization

  16. GT GT Both Init Pos res Both Res Both init Both Res Both Init GT GT Both Init Both Res Evaluation (Simulation) Sphere Cube Water Pipe Lens Ground Truth Initial Guess Computed Result Shape Error

  17. Real Experiment: Sphere

  18. Features

  19. Initial Guess

  20. Setup: Initial Guess Initial Guess: Diameter: 8 inch

  21. Setup: Result Ground Truth: Diameter: 3 inch. Computed: 3.18 inch

  22. Result

  23. Real Test: Water Filled Pipe

  24. Features

  25. Initial Guess

  26. Setup: Initial Guess Initial guess: Diameter: 200.0mm Thickness: 20.0mm

  27. Setup: Result Ground Truth: Diameter: 117.0mm Thickness: 3.0mm Computed: Diameter: 116.1mm Thickness: 2.3mm

  28. Result

  29. Real Test: Superquadric

  30. Features

  31. Initial Guess

  32. Result Ground truth: e = ? Computed: e = 0.18

  33. Shape and pose parameters Multiple interfaces No Segmentationrequired Summary

  34. Parameterizations of Interest • Polynomials: modeling surfaces, lenses • CAD models: shape of industrial objects • Dynamic models: time dependent parameters

  35. Assumptions • Camera parameters are known. • Features are far* and are trackable. • A proper model and a hypothesis (an initial guess) are given. * One possible assumption.

  36. Real Tests Setup

  37. Implementation • Features were manually selected and tracked (9 views). • Captured rays, a model, refraction index and a hypothesis were given as inputs. • Shape and pose were recovered using simple gradient decent (with derivatives).

  38. N1 2 2 1 1 3 3 N2 The Physics of Transparency First Interface: μ1→ μ2 Second Interface: μ2 →μ1

  39. Parametric Shape Examples No analytic solution Spherical Harmonics 8 parameters Super-Ellipse 2 parameters

More Related