1 / 33

Light field microscopy

Light field microscopy. Marc Levoy, Ren Ng, Andrew Adams Matthew Footer, Mark Horowitz. Stanford Computer Graphics Laboratory. Executive summary. captures the 4D light field inside a microscope

gbellinger
Download Presentation

Light field microscopy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Light field microscopy Marc Levoy, Ren Ng, Andrew Adams Matthew Footer, Mark Horowitz Stanford Computer Graphics Laboratory

  2. Executive summary • captures the 4D light field inside a microscope • yields perspective flyarounds and focal stacks from a single snapshot, but at lower spatial resolution • focal stack → deconvolution microscopy → volume data

  3. bigscenes small scenes Devices for recording light fields (using geometrical optics) • handheld camera [Buehler 2001] • camera gantry [Stanford 2002] • array of cameras [Wilburn 2005] • plenoptic camera [Ng 2005] • light field microscope (this paper)

  4. Light fields at micron scales • wave optics must be considered • diffraction limits the spatial × angular resolution • most objects are no longer opaque • each pixel is a line integral through the object • of attenuation • or emission • can reconstruct 3D structure from these integrals • tomography • 3D deconvolution

  5. Conventional versus plenoptic camera

  6. uv-plane st-plane 125μ square-sided microlenses Conventional versus plenoptic camera

  7. Σ Digital refocusing • refocusing = summing windows extracted from several microlenses Σ

  8. Example of digital refocusing

  9. Refocusing portraits

  10. Macrophotography

  11. Digitally moving the observer • moving the observer = moving the window we extract from the microlenses Σ Σ

  12. Example of moving the observer

  13. Moving backward and forward

  14. A light field microscope (LFM) eyepiece intermediate image plane objective specimen

  15. A light field microscope (LFM) • 40x / 0.95NA objective ↓ 0.26μ spot on specimen× 40x = 10.4μ on sensor ↓ 2400 spots over 25mm field • 1252-micron microlenses ↓ 200 × 200 microlenses with12 × 12 spots per microlens sensor eyepiece intermediate image plane objective specimen → reduced lateral resolution on specimen= 0.26μ × 12 spots = 3.1μ

  16. 2.5mm 160mm 0.2mm A light field microscope (LFM) sensor

  17. Example light field micrograph • orange fluorescent crayon • mercury-arc source + blue dichroic filter • 16x / 0.5NA (dry) objective • f/20 microlens array • 65mm f/2.8 macro lens at 1:1 • Canon 20D digital camera 200μ ordinary microscope light field microscope

  18. f The geometry of the light fieldin a microscope • microscopes make orthographic views • translating the stage in X or Y provides no parallax on the specimen • out-of-plane features don’t shift position when they come into focus objective lenses are telecentric

  19. Panning and focusing panning sequence focal stack

  20. Mouse embryo lung(16x / 0.5NA water immersion) 200μ pan focal stack light field

  21. (wave optics dominates) (geometrical optics dominates) Axial resolution(a.k.a. depth of field) • wave term + geometrical optics term • ordinary microscope (16x/0.4NA (dry), e = 0) • with microlens array (e = 125μ) • stopped down to one pixel per microlens → number of slices in focal stack= 12

  22. (UMIC SUNY/Stonybrook) (Noguchi) (DeltaVision) 3D reconstruction • confocal scanning [Minsky 1957] • shape-from-focus [Nayar 1990] • deconvolution microscopy [Agard 1984] • 4D light field → digital refocusing →3D focal stack → deconvolution microscopy →3D volume data

  23.  {PSF} 3D deconvolution [McNally 1999] • object * PSF → focus stack •  {object} × {PSF} →  {focus stack} •  {focus stack}  {PSF} →  {object} • spectrum contains zeros, due to missing rays • imaging noise is amplified by division by ~zeros • reduce by regularization, e.g. smoothing focus stack of a point in 3-space is the 3D PSF of that imaging system

  24. Silkworm mouth(40x / 1.3NA oil immersion) 100μ slice of focal stack slice of volume volume rendering

  25. volume rendering all-focus image [Agarwala 2004] Insect legs(16x / 0.4NA dry) 200μ

  26. (DeltaVision) (from Kak & Slaney) 3D reconstruction (revisited) • 4D light field → digital refocusing →3D focal stack → deconvolution microscopy →3D volume data • 4D light field → tomographic reconstruction →3D volume data

  27. Optical Projection Tomography [Sharpe 2002] Implications of this equivalence • light fields of minimally scattering volumes contain only 3D worth of information, not 4D • the extra dimension serves to reduce noise, but could be re-purposed?

  28. Conclusions • captures 3D structure of microscopic objects in a single snapshot, and at a single instant in time Calcium fluorescent imaging of zebrafish larvae optic tectum during changing visual stimula

  29. Conclusions • captures 3D structure of microscopic objects in a single snapshot, and at a single instant in time but... • sacrifices spatial resolution to obtain control over viewpoint and focus • 3D reconstruction fails if specimen is too thick or too opaque

  30. Nikon 40x 0.95NA (dry) Plan-Apo Future work • extending the field of view by correcting digitally for objective aberrations

  31. 200μ Future work • extending the field of view by correcting digitally for objective aberrations • microlenses in the illumination path → an imaging microscope scatterometer angular dependence of reflection from single squid iridophore

  32. http://graphics.stanford.edu/projects/lfmicroscope

More Related