Literature Survey of Microscopy 3D Reconstruction Methods

1. Literature Survey of Microscopy 3D Reconstruction Methods Chris Kammerud Imaging, Robotics, & Intelligent Systems Laboratory The University of Tennessee December 3, 2004

2. Outline • Literature Survey • Tomography (Transmission Electron) • Interferometry • Multifocus (Confocal) • Stereo (Scanning Electron) • Conclusions and Future Work • References • Images of wooden frame for mirror

3. Tomography • Tomography methods illuminate an object from different directions, collecting transmission or reflected data in order to build cross-section images • Projection at given angle can be found by calculating line integral of the image in the direction specified by angle Illustration of the nature of projections. Objects, O1 and O2, are projected onto two different planes, P1 and P2. The projections on each plane are found by calculating the integral of the object in the direction specified by the angle of projection1

4. Radon Transform2 • f(x,y) describes some object D • Radon transform defined as mapping of f(x,y) by a line integral through f along all possible lines L, with unit length ds • Method of tomographic reconstruction is to collect data from several projections and do an inverse transformation

5. Data Point Missing Wedge Non-uniform sampling of Fourier space – Lower frequencies sampled more than higher frequencies “Best” Reconstruction • In practice the problem of tomographic imaging is that only a discrete number of projections can be taken, so the reconstruction problem is finding the “best” reconstruction • Central Slice Theorem : Fourier transform of a projection at a given angle is a central section through the fourier transform of the object • Several methods developed • Fourier Methods • Direct Fourier Reconstruction (DFR) 3 • Weighted back-projection (WBP)4,5 • Iterative Methods • Projection onto Convex Sets (POCS) 6 • Entropy Techniques (ET) 7 • Algebraic Reconstruction Techniques (ART) 8

6. Comparison

7. Collector Light Source Splitter Moving reference object (usually mirror) Specimen Interferometry • Interferometry methods extract a specimen’s topography data from an interference pattern created between a reference light beam and light reflected from the specimen9

8. Interferometry Pros/Cons10 • No preparation needed as with SEM where non-conductive specimens need to be coated, coating also can cause stress to free standing items on a MEMS device • AFM microscopes provide greater resolution but at a much slower speed (minutes for a scan, whereas interferometry scans take seconds) • Interferometry methods reach vertical resolutions below 1 nm • Lateral resolution limited by resolving power of light source

9. Interferometry Methods • Phase-shift interferometry 9 • Single frequency light source • Solves equations for height of object using several different phases of interfering reference light • Often generate different phrases by moving reference mirror • Scanning White Light 10,11 • Uses broad spectrum of white light • Fourier methods to construct equations at several different frequencies • Electronic Speckle Pattern Interferometry (ESPI) 12,13 • Surface with light scattering properties shows a speckled appearance when illuminated by laser light • Speckle due to random interference and comparison with a speckle pattern on a reference surface can yield topography of unknown specimen • Stroboscopic illumination 14 • Used for time-varying measurements • Light source flashes at known intervals so that measurements can be made of devices in motion

10. Comparison

11. Multifocus • Multifocus extract depth by changing location of focal plane on specimen, either by changing focus of microscope or moving specimen vertically through the microscope’s fixed focal plane • Confocal very limited depth of focus, can acquire very thin optical slices

12. Laser Focus Plane Optical Slices 1. . . . . Specimen N Adjustable Stage Confocal Methods • Restoration Methods / Deconvolution 15 • Carrington Algorithm (CA) 16 • Expectation Maximation for Maximum likelihood Estimator (EM-MLE) 16 • Iterative constrained Tikhonov-Miller (ICTM) 16

13. Comparison

14. C d P z R α 0 x,ε η x ε y,η P’ y Stereo with SEM • Point P on object projected to P’ on z = 0 plane • Tilting specimen around x-axis causes a shift amount in image plane based on height of P Geometry of SEM imaging 17

15. Matching • Area based matching • Grey-level difference 18 • Correlation • Cross-Correlation 19 • Normalized Cross-Correlation 20 • Mutual correlation coefficient 18 • Feature based matching 21,22 • Edge matching • Corner matching

16. Comparison

17. Conclusions / Future Work • From the survey we can conclude that: • Interferometry and tomography have the most variety in methods, with interferometry having a large amount of intensive research in recent years due to its non-destructive nature and rapid measurement • Difference in systems and the processing needed by user to achieve 3D models, with AFM/Confocal/Interferometry systems needing the least compared with electron microscopes • Future Work • Polish draft, possibly add a few more papers especially in area of interferometry

18. References • 1 A.C. Kak, M. Slaney, “Principles of Computerized Tomography Imaging”, IEEE, New York, 1988. • 2 J. Radon, “Über die bestimmung von funktionen durch ihre integralwerte längs gewisser mannigfaltigkeiten”, Math-Phys. Kl, vol 69, p262-267, 1917. • 3 P.A. Midgley, M. Weyland, “3D Electron Microscopy in the Phyical Sciences: The Development of Z-contrast and EFTEM Tomography.” Ultramicroscopy, v96, n 3-4, p 413-431, Sept. 2003. • 4 R.A. Crowther, D.J. DeRosier and A. Klug, “The Reconstruction of a Three-Dimensional Structure from Projections and its Application to Electron Microscopy”, Proceedings of the Royal Society of London, Series A, Mathematical and Physical Sciences, Vol 317, No 530, p319-340, 1970 • 5 M. Radermacher, “Weighted backprojection methods”, Electron Tomography, J. Frank, Editor, Plenum, New York (1992), pp. 91–116. • 6 J.M. Carazo, “The fidelity of 3D reconstructions from incomplete data and the use of restoration methods”, in: Electron Tomography, Editor J. Frank, Plenum Press, New York, p. 117, 1992. • 7 U. Skoglund, L. Ofverstedt, “Maximum-entropy three-dimensional reconstruction with deconvolution of the contrast transfer function: a test application with adenovirus.” J. Struct Biol. 117(3), p173-188, Nov-Dec, 1996. • 8 R. Marabini, G. T. Herman, J. M. Carazo, “3D reconstruction in electron microscopy using ART with smooth spherically symmetric volume elements (blobs)” Ultramicroscopy, Vol. 72, Issues 1-2, p53-65, April 1998.

19. References • 9 S. Wolfling, D. Banitt, Y.N., Ben, Y. Arieli, “Innovative Metrology Method for the 3-Dimensional Measurements of MEMS.” Proceedings of SPIE The International Society for Optical Engineering, v 5443 , p255-263, 2004. • 10 C. O’Mahnoy, M. Hill, M. Brunet, R. Duane, A. Mathewson, “Characterization of Micromechnaical Structures Using White-light Interferometry.” Meas. Si. Technology, No. 14, 1807-1814, 2003. • 11 M. Hill, C. O’Mahony, H. Berney, P. J. Hughes, E. Hynes and W. A. Lane, “Verification of 2-D MEMS model using optical profiling Techniques”, Optics and Lasers in Engineering, Volume 36, Issue 2, p169-183, August 2001. • 12 N. Butters, J.A. Leendertz, “A Double Exposure Technique for Speckle Pattern Interferometry”, J. Phys. E: Sci. Instrum, No 4, p277-279, 1971. • 13 P.A. Fomitchov, S. Krishnaswamy, “A Compact Dual-purpose Shearography and Electronic Speckle-Pattern Interferometry.” Meas. Sci. Technology, No. 8, p581-583, 1997. • 14 S. Petitgrand, R. Yahiaoui, K. Danaie, A. Bosseboeuf, J.P. Giles, “3DMeasurment of Micromechanical Devices Vibration Mode Shapes with a Stroboscopic Interferometric Microscope.” Optics and Lasers in Engineering, NO. 36, p77-101, 2001. • 15 J.G. Mcnally, T. Karpova, J. Cooper, J.A. Conchello, “Three-Dimensional Imaging by Deconvolution Microscopy.” Methods, 19, 373-385, 1999. • 16 G.M.P. van Kempen, H.T.M. vander Voort, L.J. van Vliet, “A Quantative • Comparison of Two Restoration Methods as Applied to Confocal • Microscopy,” ASCI, Proc 2nd Annual Conference of the Advanced School • for Computing and Imaging, Belgium, June 5-7, p196-201, 1996.

20. References • 17 G. Piazzesi, “Photogrammetry with the Scanning Electron Microscope,” J. Phys. E Sci. Instrum, Vol 6, p392-396, 1973. • 18 K. Minoshima, T. Suezaki, K. Komai, “Genetic Algorithms for High- Precision Reconstructions of Three-Dimensional Topographys Using Stereo Fractographs”, Fatigue and Fracture of Engineering Materials and Structures, v 23, n 5, p 435-443, 2000. • 19 J. Stampfl, S. Scherer, M. Gruber, O. Kolednik, “Reconstruction of Surface Topographies by Scanning Electron Microscopy for Applications in Fracture Research,” Applied Physics A: Materials Science & Processing, Vol. 63, p 341-46, Sept, 1996. • 20 L.R. Hein, F. Silva, A.M.M. Nazar, J.J. Ammann, “Three-Dimensional Reconstruction of Fracture Surfaces: Area Matching Algorithmsfor Automatic Parallax Measruments”, Scanning, v 21, n 4, p 253-263, 1999. • 21 M. Hemmleb and M. Schubert, "Digital Microphotogrammetry— Determination of the Topography of Microstructures by Scanning Electron Microscope", Second Turkish– German Joint Geodetic Days, Berlin, Germany, May, 745–752 (1997). • 22 A. Gleichmann, J. Koehler, M. Hemmleb, J. Albertz, “Photogrammetric Determination of Topography of Microstructures by Scanning Electron Microscope,” Proc. SPIE Vol 2184, p254-263, April 1994.

21. Wooden Frame for Mirror