1 / 31

3D multimodal visualization techniques for tumor resection in neurosurgery

3D multimodal visualization techniques for tumor resection in neurosurgery. Ralph Brecheisen. 18 december 2007. Graduation committee: Prof. dr. ir. Bart ter Haar Romeny Prof. dr. ir. Jack van Wijk Dr. ir. Anna Vilanova i Bartroli Dr. ir. Bram Platel. Supervisors:

jalen
Download Presentation

3D multimodal visualization techniques for tumor resection in neurosurgery

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D multimodal visualization techniques for tumor resection in neurosurgery Ralph Brecheisen 18 december 2007 Graduation committee: Prof. dr. ir. Bart ter Haar Romeny Prof. dr. ir. Jack van Wijk Dr. ir. Anna Vilanova i Bartroli Dr. ir. Bram Platel Supervisors: Dr. ir. Anna Vilanova i Bartroli Dr. ir. Bram Platel

  2. Introduction Department of Neurosurgery azM Image-guided Surgery TU/e (Dr. Bram Platel) Intraoperative navigation Enhance i-MRI with preoperative data Resection verification Polestar N20 i-MRI Medtronic StealthStation Tumor resections Improve surgery planning

  3. Brain tumor resections Functional activations High-risk Procedures Blood vessels Fiber tracts Tumor Image: Yale University

  4. Investigate requirements Blood vessels Functional activations Fiber tracts Tumor Anatomical context ? - 2-dimensional - Not spatially related Too difficult! Images: University of Florida, Music Therapy World, King Edward Memorial Hospital, Neuro Imaging UCLA, National Institutes of Health

  5. What do we want to do? 3D! Blood vessels Functional activations Fiber tracts Anatomical context Tumor Combine in single visualization Images: university hospital Maastricht, Stanford University, M. Meissner Viatronix Inc

  6. What else do we need? Reduce occlusions and clutter Intraoperative navigation Hide/show Surgical tools 3D pointers Grid lines Cut outs (clipping) Geometric models Semi- transparency

  7. How to deal with the volume data? • Reviewed existing implementations • Main points: • Data representation • Rendering

  8. Data representation B Different scan times Interpolation inaccuracies Different volume of interest Memory waste A Resampling Intraoperative data

  9. Rendering Ray casting Texture slicing screen screen D D D • - Adaptive sampling • Programmable GPU • Works for old GPU’s-Not very flexible

  10. Review conclusion • Nobody does combination of • Ray casting • Render without resampling • Intersecting semi-transparent geometry How do WE render multiple volumes?

  11. y x z How to render a single volume? Viewing window • Vertex • Color • Position 2D projection Polygon Bounding box

  12. Rasterization I • Fragment • Color • Depth • ScreenCoord (i,j) j Dfrag Cube Viewing window (0,0) i Screen

  13. Rasterization II Default: depth test LESS nearest fragments written to screen Other depth tests: EQUAL, GREATER

  14. How to link bounding box with dataset? Vertex (1,1,1) (0,0,0) 3D texture = dataset (in GPU memory) • Fragment • Color/depth/screenpos • Interpolated texture coordinate

  15. Programmable fragment shaders User-defined parameters - E.g. texture ID’s Fragment shader Updated color Fragment Tex ID Looping Boolean logic, math functions Random texture lookup

  16. v v* Hardware-accelerated ray casting Fragment shader: Take sequence of texture samples along viewing direction Mix samples Screen frag Projection image 3D texture Nearest fragments

  17. How to mix texture samples? opacity bone 1 skin 0 Transfer function image courtesy: Washington University Medical Center α1 α2 α3 … Color samples Blending equation

  18. How could we render multiple volumes? Check position at every step! Use single pass Screen B P P’ P’’ A 3D Texture 1 3D Texture 2

  19. Multi-volume depth peeling Nearest fragments Second-nearest fragments Subdivide volumes in regions DEPTH LAYERS Screen 2D accumulation texture CACC

  20. Initialization 2D textures Screen DNEAR CNEAR CACC = (0,0,0,0) Depth test LESS

  21. if(Dfrag≤ Dnear ) then Discard() endif Depth peeling I Screen second-nearest fragments depth test LESS Peeling shader DFAR DNEAR CFAR

  22. Ray casting I depth test EQUALDNEAR DNEAR Raycasting shader: (1) Compute start/end positions in region (2) Take Cacc (3) Blend with Cnear (4) Blend with texture samples (5) Output color DFAR CACC CACC CNEAR

  23. DNEAR DFAR CNEAR CFAR if(Dfrag≤ Dnear ) then Discard() endif Depth peeling II Screen Swap textures third-nearest fragments Peeling shader DFAR DNEAR CFAR

  24. Ray casting II depth test EQUALDNEAR Screen DNEAR DFAR Raycast shader CACC CACC CNEAR Screen

  25. Intersecting geometric models Almost for Free! Screen Sphere Volume bounding box Raycasting shader: … (2) Take Cacc (3) Blend with Cnear … Cnear = blue Cnear = 0,0,0,0

  26. Cut outs (clipping) Clipping sphere Screen DFAR DNEAR

  27. Results A picture says… DEMO!

  28. Performance NVIDIA GeForce 8800 GTX, 768MB (screen: 500x500)

  29. Conclusions • Investigated surgery planning requirements • Developed new multi-volume rendering algorithm based on depth peeling • N volumes, N transfer functions • Translucent geometry intersections • Convex clipping • Interactive displacement of volumes and geometry • Basic user interface

  30. Future work • Build neurosurgery application • Integrate with i-MRI and navigation system • Combine gray value TF’s with object segmentation • Integrate DTI tool Anna • Enhance structures using multimodal info • Memory handling and performance

  31. Questions?

More Related