Medical Images Visualization at the Computer Graphics Group/UFRGS

1. Medical Images Visualization at the Computer Graphics Group/UFRGS Carla Maria Dal Sasso Freitas February, 2000

2. Summary 1. Introduction 3. CG Group overview • Previous works 4. Current project: VPat 5. Final comments

3. 1. Introduction

4. 1. Introduction • Porto Alegre • 470,25 km2 • Population: ~ 1,286.251 • Climate: Subtropical wet with four well-defined seasons • Higher education: 4 large universities (each one with more than 20K students) and several small colleges

5. 1. Introduction • UFRGS (Federal University of Rio Grande do Sul) • Created in 1895 • One of the top five universities in Brazil, both in size and quality • ~ 2,278 faculty members • Students: ~ 25,286 (undergraduate and graduate)

6. 1. Introduction • Informatics Institute • Teaching and research since 1968 • Established as an Institute in 1989 • Departments • Applied Computing • Theoretical Computing

7. 1. Introduction • Faculty • 69 professors (INPG, Grenoble; Karslruhe and Stuttgart, Germany; Newcastle, UK; Stanford, USA; Coimbra, Portugal; Louvain, Belgium; Amsterdam, Netherlands; ...) • Students: 700 undergraduate level + 270 graduate level • Courses at graduate level • M.Sc. in Computer Science • Ph.D. in Computer Science • Professional education

8. 1. Introduction • Research areas • Computer Graphics and Image processing • Computer Architecture/Parallel Processing • Microelectronics/Digital Systems • Data Base Systems • Fault Tolerance • Software Engineering • Theoretical Computer Science • Artificial Intelligence • Computational Mathematics • Computer Networks/Communication • Formal Methods

9. 2. CG Group Overview • Started in 1978, with one professor only • People • 4 Professors • 1 Post-doc • 5 Ph.D. Students • 18 M.Sc. Students

10. 2. CG Group Overview • Research in the 80's: CAD • Research in the 90's • Rendering and animation • Scientific visualization • Meteorological data • Geological data • Medical images • Image processing techniques

11. 3. CG Group Overview • Previous works regarding medical images • Nedel at EPFL • Freitas and group at UFRGS

12. 3. CG Group Overview • Previous works • Nedel at EPFL • Freitas and group at UFRGS

13. 3. CG Group Overview • Previous works • Nedel at EPFL • Freitas and group at UFRGS

14. 3. VPat (Visualization and interaction with Virtual Patients) • Goals • Generation of virtual human models (virtual patients) to use in medical applications such as simulation of surgery and training • Movement simulation • Development of a framework to guarantee software reuse • Integration of the existing tools

15. 3. VPat • Activities • Design of the OO framework • Volume visualization • 3D reconstruction of the human parts from real data • Motion simulation and body deformation (anatomic simulation of human bodies)

16. 3. VPat • Activities • Design of the OO framework • Volume visualization • RenderVox improvement and conversion to the VPat framework • Survey about collaborative visualization • Multimodal visualization • 3D reconstruction of the human parts from real data • Motion simulation and body deformation (anatomic simulation of human bodies

17. 3. VPat • Multimodal visualization with RenderVox • MRI and PET data obtained from different patients, no registration algorithm used (Marcelo Silva, 2000)

18. 3. VPat • Multimodal visualization: ongoing work • Data obtained from the same patient • Cooperation with the best hospital in Brazil • INCOR/University of Sao Paulo • Study of registration methods (Isabel Manssour’s PhD thesis)

19. 3. VPat • Activities • Design of the OO framework • Volume visualization • 3D reconstruction of the human parts from real data • Marching cubes implementation • Study of multi-resolution techniques • Motion simulation and body deformation (anatomic simulation of human bodies)

20. 3. VPat • Activities • Design of the OO framework • Volume visualization • 3D reconstruction of the human parts from real data • Motion simulation and body deformation (anatomic simulation of human bodies) • Mechanical modeling of joints • Skeleton motion control • Soft tissue deformation

21. 3. VPat • Surgery simulation (Luciana Nedel, 1999)

22. 3. VPat • Activities • Design of the OO framework • Volume visualization • 3D reconstruction of the human parts from real data • Motion simulation and body deformation (anatomic simulation of human bodies)

23. 4. Final comments • Related works in our group • Interactive segmentation of medical images • Olabarriaga, 1999 with A. Smeulders, Amsterdam • Development of new filtering techniques • Scharcanski & Jung, 1999/2000 • ultrasound images from fetal hearts • mammographic images

24. http://www.inf.ufrgs.br/cg

25. Collaborative Visualization in Medicine (WSCG 2000, February 7-11, Plzen) • Goal: Medical data visualization overview • Different approaches for collaborative visualization • To get knowledge about difficulties for its utilization

26. Collaborative Visualization in Medicine • CSCW (Computer Support for Cooperative Work) • New successful area • People in different places working together • CSCV (Computer Support for Collaborative Visualization) • Subset of CSCW applications • Shared visualization and control parameters • Challenge: multi-user interactive applications

27. User A Data Filtering(F) Filtering(F) Mapping(M) Mapping(M) Render(R) Render(R) Data Image Image User B Shared Data Shared Control Collaborative Visualization in Medicine • Visualization pipeline (Haber & McNabb, 1990) extension to achieve collaboration

28. Collaborative Visualization in Medicine • Telemedicine • Communication technology used to support interaction between physicians and patients • Applications • Remote clinical consultation • Telesurgery, teleradiology • Collaborative diagnosis • Collaborative visualization systems applied to Medicine • Image presentation to remote collaborators • Image-based interaction

29. Collaborative Visualization in Medicine • TeleInViVo • Fraunhofer Center for Computer Graphics, DARPA e MATMO • Main goals • Therapy planning and treatment • Medical training, surgery and diagnosis

30. Collaborative Visualization in Medicine • TeleMed • Los Alamos National Laboratory and National Jewish Center for Immunology and Respiratory Medicine • Prototype of the Virtual Patient Records (VPR) • Main goal • Standardize the electronic management of patient information

31. Collaborative Visualization in Medicine • Current trend • Use of the Web as a collaboration environment • Challenges and open questions • Communication technology • Identification (security) handling • Shared data coherence • Synchronization of user activities • User-friendly interface • Real-time visualization and interaction • VR devices accuracy and touch-feedback • Realistic images

32. Collaborative Visualization in Medicine • The building of a collaborative system is an interdisciplinary effort • User-centered approach • Efficient data management • Object-oriented design and programming • Collaborative systems have to be improved to become attractive work tools • Several difficulties for the real utilization of such systems are still found

33. Nedel at EPFL(Ph.D. Thesis) • The skeleton • Anatomic modeling of skeletons • Joints position ( Luciana Nedel, 1998)

34. Nedel at EPFL ( Luciana Nedel, 1998)

35. Nedel at EPFL • Simulation of the muscles action • Action lines • Represent mechanically the force that a muscle produces on a bone • Composed by an origin, an insertion and optionally by one or more control points (Luciana Nedel, 1998)

36. Nedel at EPFL • Muscles deformation • Mass-spring deformation model • Example: extension • Example: compression (Luciana Nedel, 1998)

37. Nedel at EPFL • Example: reconstructed muscle (Luciana Nedel, 1998)

38. Nedel at EPFL • Framework to allow the human body modeling and simulation • Body Builder Plus - integration tool • Allows the design of human models created entirely with bones and reconstructed muscles • Combines deformable muscles with metaballs representing some muscles, organs and fat tissues

39. Nedel at EPFL • Body Builder Plus: examples... (Luciana Nedel, 1998)

40. RenderVox • Interactive volume visualization of medical images using ray casting • Available tools • Navigation through the slices data set • Cutting planes • Cutting volume/ subvolume • Hybrid (geometric and volume) visualization • Multimodal visualization

41. RenderVox • Camera control and slice visualization Marcelo Silva, 1999

42. RenderVox • Interactive interface (Marcelo Silva, 1998/2000)

43. RenderVox • MRI of a head (Marcelo Silva, 1998)

44. RenderVox • Transparency levels using classification tables (Marcelo Silva, 1998)

45. RenderVox • Cutting with planes (Marcelo Silva, 1998/1999)

46. RenderVox • Cutting with volumes (Marcelo Silva, 1998/1999)

47. RenderVox • Cutting with planes and volumes (Marcelo Silva, 1998/1999) • Cutting with non-planar tools (Marcelo Silva, 1998/1999)

48. RenderVox • Hybrid rendering (geometric models & volume data) (Marcelo Silva, 2000)

49. GeoVis • Visualization of layers 1 and 2 of characteristic “Marcos de Inundação”, grid dimension 30 x 30, layer 1 in wireframe (Karen Basso, 1999)

50. GeoVis • Visualization of layer 1 of characteristic “Marcos de Inundação”, and “Isolita” attribute. (Karen Basso, 1999)