1 / 28

Nonlinear visual coding from an intrinsic-geometry perspective

Nonlinear visual coding from an intrinsic-geometry perspective. E. Barth * & A. B. Watson NASA Ames Research Center http://vision.arc.nasa.gov. Supported by DFG grant Ba 1176/4-1 to EB and NASA grant 199-06-12-39 to ABW. Intrinsic dimensionality in 2D. i0D: constant in all directions

sherri
Download Presentation

Nonlinear visual coding from an intrinsic-geometry perspective

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nonlinear visual coding from an intrinsic-geometry perspective E. Barth* & A. B. Watson NASA Ames Research Center http://vision.arc.nasa.gov Supported by DFG grant Ba 1176/4-1 to EB and NASA grant 199-06-12-39 to ABW

  2. Intrinsic dimensionality in 2D • i0D: constant in all directions • i1D: constant in one direction • i2D: no constant direction

  3. i0D FT

  4. i1D FT • e.g. straight lines and edges, gratings

  5. i2D FT • e.g. corners, line ends, curved edges and lines

  6. i0D i1D i2D

  7. i0D i2D i1D

  8. Intrinsic dimensionality in 3D • i0D: constant in all (space-time) directions • i1D: constant in 2 directions • i2D: constant in one direction • i3D: no constant direction

  9. i0D FT

  10. i1D FT • e.g. drifting spatial grating

  11. i2D FT e.g. drifting corner, flashed grating

  12. i3D FT e.g. flow discontinuities, flashed corners

  13. Intrinsic dimensionality and motion • FT of (rigid) motion signal is in a plane

  14. The visual input as a hypersurface luminance hypersurface Visualization of surfaces is easier:

  15. Geometric view on intrinsic dimensionality

  16. Curvature and motion (“plane” = “more than line but no volume”)

  17. The Riemann tensor R • most important property of (hyper)surfaces • measures the curvature of the (hyper)surface • has 6 independent components in 3D • vanishes in 1D.

  18. The Riemann tensor components are nonlinear combinations of derivatives, i.e., of linear filters with various spatio-temporal orientations.

  19. Multiple representation of speed. R components and speed v

  20. R and direction of motion q Multiple, distributed representation of direction.

  21. Sectional curvatures

  22. Directiontuningsof R components vertical motion horizontal motion

  23. Barber pole Wallach, 1935

  24. “abolished illusion” Kooi, 1993

  25. Orthogonal orientation and direction tunings Analytical predictions based on R components Typical Type II MT neuron, macaque monkey Direction tuning Rodman & Albright, 1989 Orientation tuning

  26. Multiple motions Analytical predictions based on R components Typical MT neuron, macaque monkey Recanzone, Wurtz, & Schwarz, 1997

  27. Conclusion Hypothesis that a basic (geometric) signal property (the intrinsic dimensionality) is encoded in early- and mid-level vision explains • orientation selectivity (derivatives, and R2, R3) • endstopping (all R components are endstopped for translations) • velocity selectivity • direction selectivity • some global-motion percepts (by integration) • some properties reported for MT neurons. (Reference to 3D world of moving objects is not needed.)

More Related