Processing and Reconstruction of Cryogenic Electron Microscope Tomography Images

1. Processing and Reconstruction of Cryogenic Electron Microscope Tomography Images Automatic tracking of fiducial markers across very low SNR images Fernando Amat Farshid Moussavi Mark Horowitz LBL meeting-September 2006

2. Cryogenic Electron Microscope Tomography • Take ~100 electron microscope images at different tilt angles and with finite dose budget (low SNR) • Align, reproject 2D images, do 3D reconstruction. • Quality of 3D reconstruction directly related to quality of 2D preprocessing.

3. Brief problem statement Caulobacter Images ? Very low SNR, faint features. Use fiducial markers. Automatically find accurate correspondences in images for alignment.

4. Outline • A Probabilistic Framework solution • Results • Future work • Discussion/Conclusions

5. Steps of Preparation for Reconstruction • Incomplete and unreliable data at first • Incorrect decisions cause more incorrect decisions downstream (errors propagate) Robust probabilistic framework

6. Probabilistic Framework • Maximum Likelihood Estimation. We want to find assignment to variables X ,Θ,y that maximizes : P(X ,Θ,y |O) • X , the set of 3D marker locations ‹R3xM, • y, the set of trajectories across images ‹ R2xMxN • Θ, set of microscope parameters • O, the set of observed peaks in the 2D images ‹ R2xMxN • (M=number of contours, N=number of images)

7. Probabilistic Framework (cont’d) • But we have the observed peaks, not the trajectories. • Correspondence is a discrete problem. • Projection model estimation is a continuous optimization problem. • We need to split the problem P(X ,Θ,y|O)=P(X ,Θ|y,O) * P(y|O) Projection model Correspondence

8. Aligned, reprojected images to 3D Reconstruction Feature Detection/ Location Correspondence Projection Model Estimation 2D Images Probabilistic framework: block diagram Finds O (peaks) Finds argmax P(y|O) Finds argmax P(X ,Θ|y,O) {y} {X ,Θ}

9. M1 K1 M2 K2 M3 K3 Mm Kk Correspondence • What is probability p(M->K) that i-th peak in image 1 corresponds to j-th peak in image 2? • Bipartite graph matching problem- O(N!) • Scores for individual matches may not be informative enough (look at groups of matches) How to make decisions with all this uncertainty? Markov Random Fields

10. Correspondence using Markov Random Fields • Discrete problem in nature • We estimate joint pairwise correspondence for all peaks in image 1 and 2 at the same time • Use simple geometric constraints • No use of projective model->robust to distortions • Invariant to translations->no need of prealign images • Complexity is exponential in number of peaks • Use approximate techniques which treat a joint distribution over M variables as a collection of pairwise distributions (complexity becomes O(M2))

11. Projection model • Find the solution to: D([x,y]T-[R|t][X Y Z 1]T) (1) • [x,y]: known points from correspondence • [R|t]: projective model (partially unknown) • [X Y Z]: 3D markers position (unknown) • D(): cost function • (1) is the ML solution to P(X ,Θ|y,O) assuming certain error model distribution for reprojection errors (related to D())

12. Results of robust model estimation

13. Results of robust model estimation

14. Results of robust model estimation

15. Results: tracking contours

16. Results: tracking contours

17. Results: tracking contours

18. Results: tracking contours

19. Results: tracking contours

20. Results: tracking contours statistics

21. Results: Caulo 19 tomogram Manual reconstruction by Luis R. Comolli

22. Results: Caulo 19 tomogram Fully automatic reconstruction

23. Results: CyKR-He1 tomogram Manual reconstruction by Luis R. Comolli

24. Results: CyKR-He1 tomogram Fully automatic reconstruction

25. Results • Other tomogram reconstructions for different specimens are available. • They are not shown here to keep the talk short.

26. Future work • Occlusion: solve problems in high tilt angles for group of markers • Speed up the process • Extend Markov Random Fields correspondence to multiple images • Iterate correspondence and 3D model estimation using Expectation-Maximization if results are not satisfactory in one single pass

27. Discussions/Conclusions • Fully automated process to align images with fiducial markers: only a template of a marker is needed as an input • Accuracy results comparable to manual alignment in very low SNR images • Robust to distortions and error propagation