1 / 94

Orientation fields and 3D shape estimation

Roland W. Fleming Max Planck Institute for Biological Cybernetics. Orientation fields and 3D shape estimation. Henry Moore. Cues to 3D Shape. specularities. shading. texture.

adelle
Download Presentation

Orientation fields and 3D shape estimation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Roland W. Fleming Max Planck Institute for Biological Cybernetics Orientation fields and 3D shape estimation

  2. Henry Moore

  3. Cues to 3D Shape specularities shading texture Conventional wisdom: different cues have different physical causes  must be processed differently by visual system (‘modules’)

  4. Cues to 3D Shape specularities shading texture Goal: Find commonalities between cues.

  5. Cues to 3D Shape

  6. Cues to 3D Shape Zaidi and Li Fleming, Torralba, Adelson Zucker and colleagues Todd and colleagues Koenderink and van Doorn Malik and Rosenholtz Mingolla and Grossberg

  7. Shape from Specularities Ideal mirrored surface • It is remarkable that we can recover 3D shape: • No motion • No stereo • No shading • No texture • image consists of nothing more than a distorted reflection of the world surrounding the object Fleming et al. (2004). JOV

  8. Shape from Specularities As the object moves from scene to scene, the image changes dramatically. Yet, somehow we are able to recover the 3D shape.

  9. Image from Savarese and Perona Approach I:inverse optics • Estimate shape by inverting the physics of mirror reflections. • Make an explicit model of the environment • Make assumptions about specific environmental features (e.g. ‘lines are straight’)

  10. Approach II:direct perception • Estimate shape directly from the image • Collect image measurements that are reliable across ‘typical’ environments • No need to estimate the environment • ust use the pattern of distortions in the image

  11. Shape from Texture Pattern of compressions and rarefactions across the image indicates something about the 3D shape.

  12. Shape from Texture ? • Real-world illumination is highly structured • Specular reflections of the real world are a bit like texture • Can we solve the 3D shape of mirrors using shape-from-texture ?

  13. Image distortions • Slant distorts texture but not reflections

  14. Image distortions

  15. Image distortions

  16. Image distortions • Curvature distorts reflections but not texture

  17. Curvatures determine distortions highly curved

  18. Curvatures determine distortions slightly curved Anisotropies in surface curvature lead to powerful distortions of the reflected world

  19. Shape-from-textureandshape-from-specularityfollow different rules • For texture, image compression depends on surface slant • first derivative of surface • For reflections, image compression depends on surface curvature properties • second derivatives of surface

  20. Local analysis: banding patterns

  21. Gauge Figure Task • Subject adjusts 3D orientation of “gauge figure” to match local orientation of surface

  22. Slant and Tilt Image from Palmer, 1999

  23. Tilt Slant subjective tilt subjective slant objective slant objective tilt Results I Tilt Slant subjective slant subjective tilt objective slant objective tilt

  24. Results II Tilt Slant Tilt Slant subjective tilt subjective tilt subjective slant subjective slant objective slant objective slant objective tilt objective tilt

  25. Is it just the occluding contour? No, it is not

  26. Interpreting distorted reflections

  27. Population codes

  28. Population codes

  29. Population codes

  30. Population codes

  31. Orientation fields Ground truth

  32. Orientation fieldsare robust

  33. 3D shape appears to be conveyed by the continuously varying patterns of orientation across the image of a surface

  34. Beyond specularity Specular reflection Diffuse reflection

  35. Differences betweendiffuse and specular reflection

  36. Differences betweendiffuse and specular reflection

  37. Differences betweendiffuse and specular reflection

  38. Shiny Painted

  39. Beyond specularity Specular reflection Diffuse reflection

  40. Latent orientationstructure

  41. Orientation fieldsin shading

  42. Orientation fieldsin shading

  43. Reflectance as Illumination a(f) = 1 / f  = 0  = 0.4  = 0.8  = 1.2  = 1.6  = 2.0  = 4.0  = 8.0

  44. highly curved

  45. slightly curved Anisotropies in surface curvature lead to anisotropies in the image.

  46. Texture Anisotropic compression of texture depends on surface slant

  47. Texture Anisotropic compression of texture depends on surface slant

  48. Orientation fieldsin texture

  49. Orientation fieldsin texture

More Related