1 / 57

TBA

TBA. #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com. Unification of Vision, Geometry and Graphics Through Toolkits. Bill Lorensen GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com. What is a Toolkit?. Mathematics + Algorithms + Software Edelsbrunner, 2001. Dual Interests.

presley
Download Presentation

TBA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TBA #23 GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com

  2. Unification of Vision, Geometry and Graphics Through Toolkits Bill Lorensen GE Corporate R&D Niskayuna, NY lorensen@crd.ge.com

  3. What is a Toolkit? Mathematics + Algorithms + Software Edelsbrunner, 2001

  4. Dual Interests

  5. Marching Cubes 1984

  6. Baseball Visualization 1989

  7. Stream Polygons - 1991

  8. Triangle Decimation - 1992

  9. IEEE CG&A 1992

  10. Swept Surfaces 1993 Removal Path Swept Surface

  11. Virtual Endoscopy 1994

  12. Creating Models from Segmented Medical Data

  13. Surface and Volume Rendering

  14. Hypothesis Many real world problems cannot be solved by a single discipline

  15. Core Technologies for 3D Medical Image Analysis • Pattern Recognition • Tissue classification • Visualization • Surface / volume rendering • Fusion • Quantification • Area, volume, shape • Change detection • Longitudinal tracking • Signal variation • Information Analysis/Visualization • Registration • Intra-modality (MRI to MRI, CT to CT) • Inter-modality (MRI to PET) • Model to Modality (Atlas to MRI) • Metadata to Modality (Clinical data, biochip to MRI/CT) • Filters • Edge preserving • Noise reduction • Non uniform intensity correction • Segmentation • Edge detection • Region growing • Multi-channel

  16. Discipline-specific Toolkits • Use “best of breed” algorithms implemented by domain experts • Point matching • Voronoi diagram computation • Registration • Pose estimation • Isosurface extraction • Mathematical morphology • Skeletonization • Subdivision surfaces • Similarity measures • Surface simplification • Geometric compression

  17. Discipline-specific Toolkits • Examples • vtk, The Visualization Toolkit • Open Inventor, Graphics • Insight, Segmentation and Registration • CGAL, Computational Geometry • vxl, Image Understanding • Khoros, Image Processing

  18. vtk, The Visualization Toolkit • Open source toolkit for scientific visualization, computer graphics, and image processing • C++ Class Library • 250,000 Lines of Code • (~120,000 executable) • 20+ developers • 8 years of development • 1000 user mailing list public.kitware.com/VTK

  19. Insight Segmentation and Registration Toolkit

  20. What is it? • A common Application Programmers Interface (API). • A framework for software development • A toolkit for registration and segmentation • An Open Source resource for future research • A validation model for segmentation and registration. • A framework for validation development • Assistance for algorithm designers • A seed repository for validated segmentations

  21. Who’s sponsoring it? $7.5 million, 3 year contract The National Science Foundation The National Institute for Dental and Craniofacial Research The National Institute of Neurological Disorders and Stroke

  22. Who’s creating it?

  23. Contractor Roles • GE CRD/Brigham and Womens • Architecture, algorithms, testing, validation • Kitware • Architecture, user community support • Insightful (formerly MathSoft)/UPenn • Statistical segmentation, mutual information registration, deformable registration, level sets • Beta test management • Utah • Level sets, low level image processing • UNC/Pitt • Image processing, registration, high-dimensional segmentation • UPenn/Columbia • Deformable surfaces, fuzzy connectedness, hybrid methods

  24. Toolkit Requirements • Shall handle large datasets • Visible Human data on a 512MB PC • Shall run on multiple platforms • Sun, SGI, Linux, Windows • Shall provide multiple language api’s • Shall support parallel processing • Shall have no visualization system dependencies • Shall support multi-dimensional images • Shall support n-component data

  25. Insight - Schedule • Alpha Release, April 4, 2001. • Source code snapshot • Some non-consortium participation • Limited Public Alpha Version, Aug 8, 2001. • Public Beta Release, December 15, 2001. • Software Developer’s Consortium Meeting • Nov. 8-9, 2001, NLM, Bethesda. www.itk.org

  26. Testing Design • Distributed testing • Developers and users must be able to easily contribute testing results • Pulled together in a central dashboard • Separate data from presentation • Cross-platform solution • Strive to have the same code tested in all locations

  27. Using vtk and Insight Registration of Volumetric Medical Data

  28. Mutual Information • Computes “mutual information” between two datasets, a reference and target • MI(X,Y) = H(X) + H(Y) – H(X,Y) • Small parameter set • Developed by Sandy Wells (BWH) and Paul Viola (MIT) in 1995 • Defacto standard for automatic, intensity based registration

  29. Insight Mutual Information Registration • There is no MI open source implementation • The Insight Registration and Segmentation Toolkit has an implementation • GE and Brigham as Insight contractors have early access to the code • Code was developed at MathSoft (now called Insightful) • GE was able to “guide” development with input from Sandy Wells

  30. Longitudinal MRI Study • Register multiple volumetric MRI datasets of a patient taken over an extended time • Create a batch processing facility to process dozens of datasets • Resample the datasets

  31. Approach • Validate the algorithm • Pick a set of parameters that can be used across all the studies • For each pair of datasets • Perform registration • Output a transform • View the resampled source dataset in context with the target dataset

  32. Division of Labor Read data Normalize data Export data vtk MRIRegistration.cxx Import Data Register Report transform itk Read data Reslice Display MultiCompare.tcl vtk

  33. ImageReader ImportImage ImageToImageRigidMutualInformationGradientDescentRegistration ImageCast The Pipeline ImageStatistics ImageShrink3D ImageShiftScale ImageExport Matrix4x4 vnl_quaternion

  34. Oregon Data • 25 Registrations • 13 Subjects • Qualitative comparison • One set of parameters for all studies

  35. Difference Checkerboard Target Original image Source Original image Longitudinal MRI No Registration

  36. Difference Checkerboard Target Original image Source Original image Longitudinal MRI Registration

  37. Multi Field MRI Data • Register 1.5T and 3T to 4T data • Resampled 1.5T and 3T to correspond to the 4T sampling • Volume rendering of the 3 datasets from the same view

  38. Difference Checkerboard Target Original Image Source Original Image 1.5T vs 4T MRI No Registration

  39. Difference Checkerboard Target Original Image Source Original Image 1.5T vs 4T MRI Registration

  40. 3D Visualization of the same subjectScanned with different MR field Strengths 3T 1.5T 4T All Registered To 4T

  41. CT Lung Longitudinal Study • Register two CT exams of the same patient taken at two different times • Side-by-side synchronized view for visual comparison

  42. Difference Checkerboard Target Original Image Source Original Image Lung CT No Registration

  43. Difference Checkerboard Target Original Image Source Original Image Lung CT Registration

  44. microPet/Volume CT

  45. Back to the Software

  46. Why Now? • Internet enables distributed software development • There are some successful Open Source projects • A basic set of algorithms (and sometimes mathematics) exist • Light weight software engineering processes exist • Low investment to support software development • Minimally invasive

  47. Software Trends Lightweight Software Engineering Processes

  48. IEEE Computer October, 1999

More Related