Supporting Phenotyping through Visualization and Image Analysis

1. Supporting Phenotyping throughVisualization and Image Analysis Raghu Machiraju, Computer Science & Engineering, Bio-Medical InformaticsThe Ohio State University

2. About Myself • Associate Professor, Computer Science and Engineeering, BioMedical Informatics • 7th Year at OSU • Research Interests – Imaging, Graphics and Visualization • Notable Points • Co-Chair of Visualization 2008 Conference, Columbus OH • Alumni in video gaming/animation industry (Pixar, EA), National Government Labs (Lawrence Livermore), Industrial Research (Samsung, IBM, Mitsubishi Electric), Medical Schools (Harvard Medical School)

3. Research Activities • Medical, Biological Imaging and Visualization • Optical Microscopy • In-vivo, fluorescence imaging • Structural/Functional Magnetic Resonance Imaging • Diffusion Tensor Imaging • Mostly interested in: • Segmentation, Registration, Tracking • Applications: phenotyping, longitudinal studies

4. Reconstruction of Microscopic Architecture Stained (H&E) Light Microscopy Stack Confocal Microscopy Stack Cellular structures near mammary gland of a female mouse Source: Dr. Leone, Cancer Genetics, OSU Embryonic Structure of Zebra Fish, Source: Dr. Sean Megason, Harvard Medical School

5. My Colleagues … Kishore Mosaliganti, 5th yearBioinformatics/Cancer Genetics Gustavo Leone, Mike Ostrowski Human Cancer Genetics Program Kun Huang, Biomedical Informatics

6. The Usual Imaging Pipeline Harvest Rb- &Rb+ mice Sectioning - 5 microns Imaging Visualization

7. An Advanced Role for Imaging Support • Mouse Placenta • Role of Rb tumor suppressor gene • Changes in placental morphology • Fetal death and miscarriages • Large data size • High resolution image (~1 GB) • 800~1200 slides/dataset • Quantification • Surface area/volume of different tissue layers • Infiltration between tissue layers

8. Need More - Morphometric Differences Labyrinth-Spongiotrophoblast Interface

9. Wild Type (Top) vs. Mutant (Bottom)

10. Yet Another (A)Typical Example  • Mouse Mammary Gland • PTEN phenotyping • Data characteristics • High resolution 20X images (~1 GB) • 500 slides/dataset • Mammary duct segmentation and 3D reconstruction

11. Digging In - Tumor Micro-Environment • Mouse Mammary Gland • More comprehensive system biology study • Data characteristics • Confocal, multi-stained • 50 slides/dataset • Multi-channel segmentation and 3D reconstruction

12. The Last One - Zebrafish Embryogenesis • Identifying and tracking development in the embryo • Presence of salient structures • 3D cell segmentations and tracking required • Different in-plane and out-plane resolutions • 800 Time steps available Final 3D segmentation A 2D image plane

13. The Underlying Premise Is there an unified way to visualize and analyze the various microscopic image modalities ?

14. The Essentials Of Microstructure • Premise - you can measure, visualize and analyze cellular structures if you characterize and build virtual microstructure • Component • Distributions • Packing • Arrangements • Material Interfaces

15. Essential I- Component Distributions & Packing • Tissue layers differ in spatial distributions • Characteristic packing of RBCs, nuclei, cytoplasm - phases • Differ in porosity, volume fractions, sizes and arrangement • NOT JUST ANOTHER TEXTURE ! • Use spatial correlation functions !

16. Essential II - Component Arrangements • Arrangements • Complex tessellations which can better characterize changes. • A step ahead of looking at only nuclei their packing • Complex geometry • Concentric arrangement of epithelial cells • Torturous 3D ducts and vasculature

17. Essentials III – Material Interfaces Labyrinth-Spongiotrophoblasts Interface

18. The Holy Grail – Virtual Cellular Reconstructions Before using cellular segmentation Using N-pcfs and cellular segmentations

19. 1 TeraByte Pipelines 1Gb x 1 Gb x 900 20 x magnification Image Registration (3-D alignment) Feature extraction Image Segmentation 3-D Visualization Quantification NIH Insight Tool Kit (ITK), NA-MIC Tools (microSlicer3)

20. Conclusions • Highly multi-disciplinary approach. • Need scalability and robustness • Useful workflows need to be constructed • Much application-domain knowledge has to be embedded in algorithms • Validation of methods and proving robustness is a pre-occupation. • The final goal of a virtual cellular architecture is not that elusive 

21. Destroying The Amazon Rain Forest  • K. Mosaliganti and R. Machiraju et al. An Imaging Workflow for Characterizing Phenotypical Change in Terabyte Sized Mouse Model Datasets. Journal of Bioinformatics, 2008 (to appear) • K. Mosaliganti and R. Machiraju et al. Visualization of Cellular Biology Structures from Optical Microscopy Data. IEEE Transactions in Visualization and Computer Graphics, 2008 (to appear) • K. Mosaliganti, R. Machiraju et al. Tensor Classification of N-point Correlation Function features for Histology Tissue Segmentation. Journal of Medical Image Analysis, 2008 (to appear) • K. Mosaliganti and R. Machiraju et al. Geometry-driven Visualization of Microscopic Structures in Biology. Workshop on Knowledge-Assisted Visualization, Proceedings of EuroVis2008 (to appear). • K. Mosaliganti, R. Machiraju et al. “Detection and Visualization of Surface-Pockets to Enable Phenotyping Studies”. IEEE Transactions on Medical Imaging, volume 26(9), pages 1283-1290, 2007. • R. Sharp, K. Mosaliganti et al. “Volume Rendering Phenotype Differences in Mouse Placenta Microscopy Data”. Journal of Computing in Science and Engineering, volume 9 (1), pages 38-47, Jan/ Feb 2007. • P. Wenzel and K. Mosaliganti et al. Rb is critical in a mammalian tissue stem cell population. In Journal of Genetics and Development, volume 21 (1), pages 85-97, Jan 2007. • K. Mosaliganti and R. Machiraju et al. Automated Quantification of Colony Growth in Clonogenic Assays. Workshop on Medical Image Analysis with Applications in Biology, 2007, Piscatway, Rutgers, New Jersey, USA. • R. Ridgway, R. Machiraju et al. Image segmentation with tensor-based classification of N-point correlation functions. In MICCAI Workshop on Medical Image Analysis with Applications in Biology, 2006. • O. Irfanoglu, K. Mosaliganti et al. “Histology Image Segmentation using the N-Point Correlation Functions”. International Symposium of Biomedical Imaging, 2006.

22. Acknowledgements • Joel Saltz, BMI • Richard Sharp, Okan Irfanoglu, Firdaus Janoos, CSE OSU • Weiming Xia, Sean Megason, Harvard Medical school • Jens Rittscher, GE Global Research • NIH, NLM Training Grant • NSF ITR grant

23. Thank You ! Questions ?