Center for Subsurface Sensing & Imaging Systems

1. Center forSubsurface Sensing & Imaging Systems Overview of Image and Data Information Management in CenSSIS David Kaeli Northeastern University Boston, MA kaeli@ece.neu.edu

2. Overview of the Strategic Research Plan Bio-Med Enviro-Civil L3 S2 S3 S4 S1 S5 ValidatingTestBEDs L2 R2 FundamentalScience L1 R3 Image and Data Information Management R1

3. R3 Research Thrust Overview • Utilize enabling hardware and software technologies to address CenSSIS barriers • Pursue research in enabling technologies • Develop a common set of tools and techniques to address SSI problems: • Hardware parallelization and acceleration • Software toolboxes • Image database management and tools Toolboxes

4. CenSSIS Middleware Tools • Parallelization of MATLAB, C/C++ and Fortran codes using Message Passing Interface (MPI) – a software pathway to exploiting GRID-level resources • Utilizing MPI-2 to address barriers in I/O performance • Building on existing Grid Middleware such as Globus Toolkit, MPICH-G2 and GridPort • Presently illustrating the impact of the GRID on system level projects (tomosynthesis reconstruction) MATLAB MPI C/C++ MPICH-G2 Parallelization Fortran UPC

5. Air Mine Soil Impact on CenSSIS Applications • Reduced the runtime of a single-body Steepest Descent Fast Multipole Method (SDFMM) application by 74% on a 32-node Beowulf cluster • Hot-path parallelization • Data restructuring • Reduced the runtime of a Monte Carlo scattered light simulation by 98% on a 16-node Silicon Graphics Origin 2000 • Matlab-to-C compliation • Hot-path parallelization • Obtained superlinear speedup of Ellipsoid Algorithm run on a 16-node IBM SP2 • Matlab-to-C compliation • Hot-path parallelization

6. Tomographic mammography 3D image reconstruction from x-ray projections Used to detect and diagnose breast cancer Based on well developed mammography techniques Exposes tissue structure using multiple projections from different angles Advantages Accuracy: provides at least as much useful information than x-ray film Flexibility: digital image manipulation, digital storage Structural information: using layered images Safe: low-dose x-ray Lower cost: compared to MRI

7. Y X-ray projections Z No Yes Satisfied ? X Y Image acquisition/reconstruction process X-ray source Acquisition: 11 uniform angular samples along Y-axis X-ray projection: breast tissue density absorption radiograph Algorithm: constrained non-linear convergence and iterative process Uses a Maximum Likelihood Estimation Initialization Set 3D volume Forward 3D volume Compute projections Backward detector Correct 3D volume Exit

8. exchange data Overlap area Parallelization approaches Third approach: Non-overlap with inter-communication (less computation, more communication) First approach: Non inter-communication (more computation, less communication) Second approach: Overlap with inter-communication Reduce communication data Segmentation along Y-axis Using redundant computation to replace communication Segmenting along x-ray beam

9. Tomosynthesis Acceleration • Input data set: phantom 1600x2034x45 • Serial implementations runs in 2-3 hours on a P4 machine • Platforms: • SGI Altix system • UIUC NCSA Titan cluster • UIUC NCSA IBM p690 • UMich Hypnos cluster • P4 cluster at MGH • Number of processors: 32 • Computation: SGI Altix with Itanium 2 processor outperforms the other CPUs • Currently moving this work to the GRID and the Pittsburgh Supercomputer Center • Prototype running on our GRID system at NU

10. Field Programmable Gate Arrays for Subsurface Imaging • Backprojection for Computed Tomography image reconstruction • Sponsored by Mercury Computer • Finite Difference Time Domain (FDTD) in hardware • Collaboration with Humanitarian Demining project • Retinal Vascular Tracing in real time • Collaboration with Real-time Retinal Imaging project • Phase Unwrapping • Collaboration with 3-D Fusion Microscope project • Diverse problems, similar solutions: FPGAs are particularly well suited for accelerating image processing and image understanding algorithms

11. Retinal Vascular Tracing: Register 2-D Image to 3-D in Real Time Direction of blood vessel PCI BUS “SmartCamera”

12. Some Recent Publications on Parallelization • “Execution-Driven Simulation of Network Storage Systems,” Y. Wang and D. Kaeli, Proceedings of the 12th ACM/IEEE International Symposium on Modeling, Analysis of Computer and Telecommunication Systems, October 2004, pp. 604-611. • “Profile-guided File Paritioning on Beowulf Clusters,” Y. Wang and D. Kaeli, Journal o f Cluster Computing, Special Issue on Parallel I/O, to appear, • “An Object-oriented Parallel Library,” C. Oaurrauri and D. Kaeli, International Journal of High Performance of Computing and Networking, to appear. • “Digital Tomosynthesis Mammography using a Parallelized Maximum Likelihood Reconstruction Method,” T. Wu, R. Moore, E. Rafferty, D. Koppans, J. Zhang, W. Meleis and D. Kaeli, Medical Imaging, 5368, February 2004. • “Mapping and characterization of applications in Heterogeneous Distributed Systems,” J. Yeckle and W. Rivera , To appear in Proceed. of the 7th World Multiconference on Systemics, Cybernetics and Informatics (SCI2003). • “Profile-Guided I/O Partitioning,” Y. Wang and D. Kaeli, Proceedings of the 17th ACM International Symposium on Supercomputing, June 2003, pp. 252-260. • “Source-Level Transformations to Apply I/O Data Partitioning,” Y. Wang and D. Kaeli, Proceedings of the IEEE Workshop on Storage Network Architecture and Parallel IO, Oct. 2003, pp. 12-21. Held again in 2004

13. CenSSIS Solutionware – UPRM/NU/RPI Toolbox Development • Support the development of CenSSIS Solutionware that demonstrates our “Diverse Problems – Similar Solutions” model • Develop Toolboxes that support research and education • Establish software development and testing standards for CenSSIS Image and Sensor Data Database • Develop an web-accessible image database for CenSSIS that enables efficient searching and querying of images, metadata and image content • Develop image feature tagging capabilities Toolboxes

14. Status of the CenSSIS Toolboxes • Hyperspectral Image Analysis Toolbox (HIAT) • October 2004 • Multiview Tomography Toolbox (MVT) • fddlib: • January 2003 (v. 1.0) • July 2003 (v. 1.1) • mvt: • October 2004 • Rensselaer Generalized Registration Library (RGRL) • September 2004 HIAT MVT RGRL

15. New toolbox: Improving the quality of radiation oncology @ MGH • Developed a 4D (3D + including time) visualization browser tool kit • Visualize Computed Tomography (CT) images, organ outlines (wire contours) and the isodose lines (treatment dosage) • Present all this information in a user friendly interface

16. 4-D Visualization of Lung Tumors Dosage 4-D Visualization

17. The Future for CenSSIS Toolboxes SCIRun Collaboration with the University of Utah

18. CenSSIS Image Database System • Deliver an web-accessible database for CenSSIS that enables efficient searching and querying of images, sensor data, metadata and image content • More that 4000 metadata-rich images/datasets presently available online (> 10,000 by 2006) • Database Characteristics: • Relational complex queries (Oracle9i) • Data security, reliability and layered user privileges • Efficient search and query of image content and metadata • Content-based image tagging using XML • Indexing algorithms (2D, 3D, and 4D) • Explore object relational technology to handle collections mouse embryo 3 4 2 1

19. CenSSIS Image Database System

20. CenSSIS Image Database System

21. CenSSIS Image Database System Utilize Machine Learning algorithms to improve query view

22. CenSSIS Image Database System Provides data description associated with initial collection, but does not allow for further elaboration or annotation.

23. Image Annotation • Provide the ability to markup image with searchable features • Enable image database to be more effectively data-mined <xml version=“1.0” encoding=“UTF-8”> <embryo> <description> Embryo developmental stages</description> <feature label=“1” xPos1=“29” yPos1=“33” xPos2=“48” yPos2=“50”> 1 cell embryo </feature> <feature label=“2” xPos1=“50” yPos1=“28” xPos8=“70” yPos2=“40”> 2 cell embryo </feature> <feature label=“3” xPos1= “5” yPos1= “5” xPos2=“25 yPos2=“20”> 4 cell embryo </feature> </embryo>

24. XML and Java • XML (Extensible Markup Language) • Provides maximum flexibility and portability • Well-supported standard • Powerful querying tools available in Oracle • The Java2 Platform • Cross-platform compatibility • Standard web-browser interface • Native XML support

25. A raw image file from the CenSSIS Database Image Tagging • QUERY: I want to be able to add to this image textual annotations, providing my medical team with questions about particular ROIs: • Difficult to describe regions in an image • Difficult to pinpoint specific features in images • Global image metadata too coarse to facilitate low-level tagging

26. Image with tags Image Tagging • Metadata associated with specific areas • Query for specific image features

27. The Image Tagging Interface Drawing Tools Tag Options View Options

28. Tags and XML <feature type="Ellipse" label="4 Cell Stage"> <ellipse> <xCenter> 101 </xCenter> <yCenter> 58 </yCenter> <xRadius> 79 </xRadius> <yRadius> 46 </yRadius> </ellipse> <note> [custom XML tags go here] </note> <annotator> awilliam </annotator> </feature>

29. Provide a vehicle for natural collaboration • A richer set of metadata to enable more detailed queries • Potential to perform extensive data mining on image content • An eye toward content-based image retrieval • Tumor tracking paper recently accepted to SIGMOD 2005 The Future Role of Image Annotation

30. The CenSSIS Image Database System • Hosts the image and sensor data of CenSSIS (>500 images online) • http://censsis-db1.ece.neu.edu/ • Provides metadata indexed image searching • Uses XML tags to allow for easy information interchange • Evolved into a project-based management system, allowing users to organize their data hierarchically • Key issue: how do we develop collaboration tools that increase the value of data stored in the database? • Presently exploring how best to integrate both visualization and image annotation into the existing framework (NIH proposal)

31. MVT CenSSIS Image and Data Information Management • Addressing key research barriers in computational efficiency, embedded computing and image/sensor data management • Exploiting Grid resources to enable new discovery in SSI applications • Producing a image/data repository and software-engineered Subsurface Sensing and Imaging Toolsets • Developing enabling tools targeting system-level projects • Near real-time reconstruction and visualization • Visualization of complex motion • Predicting motion in image data using database indexing techniques