The Visible Human Project Today

1. The Visible Human Project Today Presented byBrian D. Athey, Ph.D. Director, Michigan Center for Biological Information (MCBI) UM Office Vice President for Research (OVPR) Assistant Professor Director, University of Michigan Visible Human Project Department of Cell and Developmental Biology University of Michigan Medical School Ann Arbor, Michigan http://vhp.med.umich.edu http://CTAalliance.org September 26, 2002

2. University of Colorado Health Sciences University of Michigan Pittsburgh Supercomputing Center (PSC) Stanford University National Library of Medicine (NLM) Uniformed Service University of the Health Sciences (USUHS) 2002: NLM/NIH NGI Visible Human Collaboratory For Health Sciences Education and Training Abilene Network

3. An Early Test-bed Application: ARPA 1992-1995 To Extend the DoD Medical Training Platform from WRAMC to Battlefield Virtual Environment – Combat Battlefield Past: Enabling Infrastructure Middleware Needed Internet2 Backbone Virtual Workbench Future: Middleware and Applications Integrated Core – Manikin with Synchronized Anatomy from Visible Human Project Dexterous Skills Training

4. Old-time (~1995-1997) VH Client-Server Application Concept Visible Human Applications/Tools Servers

5. Development Focus: Visible Human Project(courtesy Don Jenkins, NLM) Clinical Patterns Signs & Symptoms Pathophysiology Natural Biologic Variation VISIBLE HUMAN DATASETS Disease and Pathology Normal Variation TeachingApplications Public Domain Web Sites Structural Change Scale & Resolution Biological Structure Continuum Development Aging Embryology Gross Anatomy Micro Anatomy Sub Microscopic Physiology Biochemistry ANCILLARY DATA SETS MRI, CT, PET Radiology Anatomic National Anatomy Archive DIGITAL HUMAN SIMULATION

6. The Evolving Amphitheater of Human Anatomy

7. Vision: Medical Student User Product Visible Human Database Quizzes Interface Learning Materials Animations Visuals Simulations Text Labels Case Inquiry

9. Visible Human Infrastructure Development Embraces “Cyberinfrastructure” Vision

10. Vision of Integrated IT and Computing Architecture For Life and Health Sciences Informatics

11. Ontology Integration and Biological Context NLM UMLS Semantic Network + the “Semantic Web” NLM UMLS Ontology (Linked to 50 Others) M Phenotype Digital Anatomist Organs & Tissues 109 Physiology Cell Proteome Cell High Throughput “Structural Biology” Geneotype Gene Gene nM Bio/SPICE (DARPA) and Others Bioinformatics Databases and Data Standards e.g. Celera, GO

12. Years Minutes Seconds Milliseconds Microseconds Nanoseconds Picoseconds Behavior (organs, organisms) Systems Physiology (process engineering) Cellular Processes and Interactions Molecular Dynamics Quantum Mechanics 0.1 nm 10nm 100nm 1um 1mm 1cm 1m Need for Multiple Scales Adapted from the work of Bill Goddard, Caltech

13. Visible Human “Data-locator” Navigable Volume Data

14. UMich Visible Human Knowledge Base Schematic Annotation layer • Hyperlinked texts • Movies • Sounds, etc Semantic layer • Medical lexicons • Semantic relations • Links to regions, volumes, and annotations Anatomical layer • Labels of structures • Voxel data and segmentation • Volume surfaces • Spatial relationships

15. Action Origin Insertion Innervation Arterial Venous Bounded by Original Labeling Scenario System: Skeletal Muscle Region: Pelvic Structure: Sartorius Boundaries Contains Surface Anatomy Common Anomalies Illustrations Source Branches Motor Sensory lumbar plexus (ventral primary rami of spinal nerves L2-L4) Femoral Nerve

16. Database Development • Over 7000 Terminologia Anatomica(TA) terms are now available via the UMVH database. • Structural relationships (i.e. heirarchy) imported from UMLS/TA and linked to VH content. • Non-English Languages (e.g Chinese). • Web-based querying and editing.

17. Integration of VH, and Volume/Structure/Instructional Databases Volume/Structure/Instructional Space User Request Area Structure Volume Labels XML Tagged Rotation- Translation Parameters UMLS MESH TA Arbitrary Section User Skill Instruction Area XML Tagged Allied Health Dental Surgery Nursing Anatomy Audio Video Quizzes Tests Navigable Volume Data Walter Meixner

18. Overview of simulation deformation Server 3D model tissue biomechanics computation 3D rendering computation Force computed 1000 - 10,000 times per sec Image computed 10 - 30 times per sec Internet Haptics process Display process Force sensed at tool tip Tissue deformation observed Parvati Dev Stanford University Tool tip position Remote client

19. MediaserverAccess to resources Library of images and models – A mediaserver Retrieval interface (general purpose) Interface to rotate and peel through dissection images (specialized) Interface to control a simulation (specialized) Search and browse interface Parvati Dev Stanford University Client Client Client Client

21. National Test-bed Demonstration at NREN (June 2001): Four Parallel VH Streams—Memory-to-Memory Transfer (UMich to Ames)

22. University of Michigan Internet Connectivity Research Networks Commercial Networks UCAID Institutions Cleveland POP Qwest Internet vBNS Abilene NASA Michnet DOE MERIT UM Arbor Lakes Mich Giga PoP DREN #3 #1 StarLight Chicago UM BIN GigE FW Europe MCBI MCIT 4x100mbit/sec 4x 100mbit/sec GigE OC-3 FW CAEN MCIT ITCS Gig E Gig E CAC Medical School UM Hospital + MCARE Bioinformatics Proteomics ATLAS Physics MGrid Test sites MGrid Test sites

23. UMich Visible Human Project NGI/I2 Network-Enabled Software Tools and Content Java Browser - Model View Stereo Volume Java Browser: Alexander Ade has created a set of Java-based Browsers that allows web-based volume visualization as either slices or volume-rendered scenes. The applications show anatomical labels queried from an database and displays 3-D models in either mono or stereo views. Edgewarp: Created by Dr. F. Bookstein and Dr. W. Green as a tool for analyzing shape variation by extracting invariant geometric operations from volumes and surfaces, this software program has been extensively modified for the UM VH Project. The addition of features such as labeling, model display and filmstrip creation has expanded its usefulness to medical education. Pittsburgh Supercomputing Center (PSC) Segmentation Tool: To facilitate the creation of surface models, our collaboration partner the PSC has been commissioned with the development of a segmentation tool for the production of contours. This tool has been ported to Linux, Mac and Windows platforms. Data Slice - Embedded Model View

24. Java Browser - stereo model view

25. VH Image Data Representation • Hierarchical multi-resolution tree • Each node is an 8*8*8 cube • 4 levels of representation • Volumetric compression

26. Implementation Approach • Avoid disk seek and network bottlenecks • Use memory based server representations • Apply volumetric compression techniques • Serve volume data by client demand • Take advantage of intelligent clients • Minimize server computational demands • Leverage network tuning from Web100

27. System Architecture Volume Server Router or Hub (NGI) Anatomy Database Mesh Server Intermediate Server Voxel Identity Server Collaborative Server indirect …or… direct

28. Compression Numbers • Lossy simple VQ 9.42:1 = 2.67Gbyte • Lossless TS wavelet VQ 7.03:1 =3.53Gbytes • Normal JPEG @23.3:1 = 1.06Gbytes • JPEG2000 estimated ~40:1 = ~500Mbytes • Lossless SPHIT + arith ~4.22:1 • Lossless SPHIT + huffman ~4.10:1 (2x faster) • Lossy SPHIT & other EWZ useful beyond 50:1

30. A Scalable Test-bed Must Include an Open Source Mechanism Featuring a Standards-based Adaptive Middleware Environment Shared & Private Libraries Shared Services Object Libraries Authoring Tools Solvers,utilities. Validation Runtime Results Version Control Bug Reporting Middleware and Infrastructure Visualization User Interface Peer Review Data Library Users/ Applications

31. ETF-based Raycasting: The Next Step