Arbitrary and Dynamic Patterning in a Programmable Array Microscope

1. Arbitrary and Dynamic Patterning in a Programmable Array Microscope (Controlled Light Exposure Programmable Array Microscopy) Wouter Caarls, Anthony H.B. de Vries, Donna J. Arndt-Jovin, Thomas M. Jovin Laboratory of Cellular Dynamics Max Planck Institute for Biophysical Chemistry Goettingen, Germany

2. What is the PAM? • Optically sectioning microscope using conjugate structured illumination and detection through a spatial light modulator • Reduction of offset by capturing the light rejected by the pinholes (non-conjugate image) • Arbitrary pinhole patterns • Optimized for different samples and objectives • Could be chosen automatically • Patterns can be augmented with masks • Selective photoactivation and photobleaching • Adaptive imaging

3. Dual-path PAM

4. Confocal image generation C - 0.5*NC (after registration) Conjugate (duty 1/3) Non-conjugate (duty 2/3) Confocal

5. Photobleaching • Bleaching occurs above and below the point being illuminated • Also when that point is not part of the sample • And even when the point lies to the side of the sample, due to high NA objectives NA = 1.45 In regular images, much of the experienced photobleaching is unnecessary!

6. Controlled Light Exposure Microscopy (CLEM*) • Main idea: switch off unneeded illumination to avoid bleaching • Don’t illuminate background • Don’t illuminate bright foreground • Available on (Nikon) laser scanning systems Marsupial/Rho123 Scalebar 10um *Hoebe et al., Nature Biotechnology, 2007

7. CLE-PAM • Discretized integration time decisions • Per-image decision instead of per-pixel • Allows filtering to avoid noise artifacts Illumination time Black/orange/red/white = Stop after 1/2/3/4 frames

8. Border effects • Decision to illuminate a pixel affects neighboring pixels, due to illumination PSF • Leads to border effects near decision boundaries Desired illumination Actual illumination Naïve reconstruction

9. Solving border effects • Dilate illumination, i.e. illuminate more than necessary • Only use that part of the image which is far enough away from the dilated border of illumination Desired illumination Dilated illumination Useful illumination

10. Background effects • CLE-PAM lowers the signal-to-noise ratio in the background • At very low decision times, this leads to very noisy backgrounds • Especially problematic in maximum intensity projections Single-slice CLEM image Maximum intensity projection

11. Solving background effects • Basic idea behind CLE-PAM is that background is not important • Trade SNR for spatial resolution by Gaussian filtering • Choose sigma such that resulting SNR is equal to expected SNR with full illumination. Original MIP Background-smoothed MIP The foreground is not affected!

12. Time-domain extension • Extend CLEM decision into time domain • If a pixel is background now, it will probably be background in the next frame • Additional threshold, below the low threshold • Dilated decision boundary to account for movement • Looks like sample is “pushing” the illumination 0 1 2 3 4 Time

13. Reduced photobleaching 13m 0m 4m 9m HeLa cells, mitotracker 200nM 20min, normal illumination (67ms) HeLa cells, mitotracker 200nM 20min, CLE-PAM illumination (max 67 ms)

14. Reduced photobleaching, cont. Using CLE-PAM, the time until half the original fluorescence is bleached is extended by a factor of 2.2 I = mean(img) – mean(bkg) For TD-CLEM, unilluminated pixels are set to mean background Normal t1/2 CLE-PAM t1/2

15. Conclusions • The Programmable Array Microscope allows dynamic adjustment of illumination • CLEM avoids unnecessary photobleaching by reducing the illumination outside the sample • With the PAM, CLEM can easily be integrated into a full-field system • Border effects are solved by dilating the illumination • Background is smoothed for presentation • Extension to the time domain • Photobleaching is reduced by more than a factor of 2