Shape-aware Volume Illustration

1. Shape-aware Volume Illustration Wei Chen, Zhejiang University, Purdue University Aidong Lu, University of North Carolina at Charlotte David S. Ebert, Purdue University

2. Motivation & Related Work The Main Idea Algorithm & Implementation Experimental Results Conclusions & Future Work Contents

3. Motivation Illustration is a visual representation

4. Motivation Previous work on simulating appearance Rendering primitives: point, line, surface... drawings [LEM 02, BKR 05, FBS05]

5. Motivation • Previous work on simulating appearance • Visibility guided or selective illustration: [DWE03, CSC06, VKG04, BG06, BGKG06, ONOI04]

6. Motivation Previous work on simulating appearance System work: VolumeShop [BG05], IVIS[SDS05]…

7. Motivation Most previous work relies on transfer functions to express important features Expressiveness is greatly influenced by the shape and shape variations

8. Related work Volume Illustration Incorporate NPR techniques into volume rendering. Ebert et al. [ER00] Illustrate the internal structures by synthesizing a 2D texture on the cutting planes of polygonal models. Owada et al. [ONOI04] Hardware-accelerated volume illustration [SE03,HBH03, BG05, SES05, BG06]

9. Related work Volume Illustration

10. Related work Shape Representation and Processing Boundary representation Volumetric Representation Voxelization /Distance field computation Marching Cube

11. Related work Volumetric Manipulation Traditional GPU-based volume deformation techniques [RSSSG01], [WRS01] Feature-aligned volume deformation [CSC06] VolumeShop system [BG05]

12. Related work Example-based Modeling and Rendering Texture Synthesis [WM01] Image & Curve Analogy [HJO01] [HOCS02] Mesh Contour Analogy [ZG04] Example-based Volume Illustration [Lu2005]

13. The Key Idea Creating volume illustration in a shape-aware manner A shape-aware volume representation Curve analogy based shape deformation Shape-aware volume illustration

14. The pipeline

15. Volume Segmentation Available Segmentation Approaches Manual segmentation Threshold-based or TF guided segmentation Level-set based segmentation (ITK)

16. Volume Binarization The bone The skin 0 2 1 The foot dataset

17. Volume Filtering Volume Filtering For Binary Volume Image Morphology Algorithms Level-set based segmentation (ITK) [Whi00]

18. Feature Preserving Mesh Processing Mesh smoothing [JDD03] Mesh repairing [Ju04] Mesh simplification [ZG02] Geometric Processing #V 229,298 #V 13,689

19. Geometric Processing Another example for the hand model

20. Shape Variations are to be considered Using gradient domain based surface deformation techniques [SLCo05,YZX04] Curve Analogy based shape deformation [ZHS05] Introducing a proxy surface and connecting both surfaces with mean value coordinates [JSW05] Curve Analogy Based Surface Deformation ?

21. Curve Analogy Based Deformation For a model M0, generate a simplified model M1 Generate the mean value coordinates for each vertex of M0 associated with M1 Specify a curve C1 in M1 and project it to the 2D plane Draw a curve C0 in the 2D illustration Deform C1 with respect to C0 Deform M1 with the deformed C1 by the mesh deformation algorithm Deform M0 by applying the mean-value coordinates to the deformed M1 Simplification Deformation MVC M1 (#V 1917) Deformed M1 Deformed M0 M0 (#V 11067)

22. Curve Analogy Based Deformation Apply deformation to another model Deformation Skin (#V 1917) Deformed Skin Applying Mean Value Coordinates Computing Mean Value Coordinates Bone (#V 13689) Deformed Bone (#V 13689)

23. Curve Analogy Based Deformation …… Specify a curve C1 in M1 and project it to the 2D plane Draw a curve C0 in the 2D illustration Deform C1 with respect to C0 using Laplacian editing [SLCo05] Deform M1 with the deformed C1 by the mesh deformation algorithm [ZHS05] …… C1 Deformed C1 C1 C0 C1 C0 M1 Deformed M1

24. Shape-aware Volume Representation A combination of a distance volume and a segmentation volume Each voxel records a distance value and a segmentation identification The distance volume The segmentation volume Their composition

25. Compute the signed distance volume Construct an auxiliary octree grid to accelerate the computation of the distance volume [Ju04] Encode the distance as an unsigned integer Integrate all polygonal models into a single model, and compute its distance volum. Compute a distance volume for each individual object Shape-aware Volume Representation

26. Generate the segmentation volume Based on the computed distance volumes Each voxel of the segmentation volume is first initilized as zero For each distance volume of the ith model, check the sign of each voxel If it is negative, the corresponding voxel in the segmentation volume is assigned an identification i Shape-aware Volume Representation 6 5 4 3 2 1 1 2 3 4 5 6 5 3 2 0 0 1 2 3 4 5 4 1 4 3 2 1 -1 -1 0 1 2 3 4 0 0 3 3 2 1 -1 -2 -2 -1 0 1 2 2 2 1 0 -1 -2 -2 -2 -2 -1 0 1 -1 3 2 1 0 -1 -1 -1 -1 -1 0 1 4 3 2 1 0 0 0 0 0 0 1 2 2 1 1 1 1 2 3 5 4 3 1 1 i The input model i The distance volume The segmentation volume

27. Generate an offset volume by choosing all points satisfying dist(p)<t Useful to illustrate the object boundary May build a thin offset volume form each distance volume Offset Volume Blue: muscle Yellow: bone Green: bone boundary Red: skin

28. Gives a novel explanation to the data and yields a direct expression of shape Reconstruct smooth boundaries by exploiting the information of the distance volume Be able to distinguish individual objects Benefits of New Representation

29. The representation and deformation scheme can achieve two goals. Suitable for applying various rendering styles to different regions of interest. Mimic artistic styles for object boundaries. Any volume rendering system can render the new representation Our implementation is based on IVIS volume illustration system [SDS05]. The volume is encoded in two volume textures. Shape-aware Volume Illustration

30. The Uniform Illustration Equation Solid Texturing Color Shading Opacity Modulation Directly interpolate the computed colors (in RGBA space) on the eight nearest voxel centers Yields better results Takes about eight times the computational cost Shape-aware Volume Illustration

31. Experimental Results • P4 3.2 GHz, 1.5G RAM • nVidia Quadro FX 3400 • Cg shading language

32. Experimental Results • IVIS system • 3D texture slicing number: 1000 • Image resolution: 480x480

33. Foot Deformation

34. Shape-aware Volume Rendering

35. Bunny Deformation

36. Hand Example

37. MRI Brain Data Example

38. Kidney Data Example

39. 1 2 3 The idea Incorporating shape into volume illustration • Algorithm • New representation • Curve analogy based deformation • Mixed volume illumination equation • Applications • Enhance the creation capability • Allows efficient illustration of the deformation • of solid space Conclusions • Contributions

40. Future Work • More efficient algorithm • Represent and learn intrinsic artistic shape styles from hand-drawn images • Optimize computing of the distance volume • Deformation-driven volume illustration of dynamic scenes • Model-based volume illustration for special objects

41. Acknowledgements Tao Ju (Washington University in St. Louis) Kun Zhou (Microsoft Research Asia) Xinguo Liu, Jing Huang (Zhejiang University) Nikolai Svakhine (Adobe) Oliver Deussen (Uni-Kanstanz university, Germany) Stefan Bruckner (Tu Wien, Austria) The Hand dataset is the courtesy of Tiani Medgraph, Austria. NSF of China (No.60503056) DOE DE-FG02-06ER25733, NSF 0633150, EPA VisualGrid NSF Grants 0081581, 0121288, 0328984, and the U.S. Department of Homeland Security.

42. Thank You ! chen23@purdue.edu http://web.ics.purdue.edu/~chen23

43. Work Pipeline of Volume Illustration 3D/4D Data Field Raw Data Input Data Data Preprocess Data Acquisition Interactive Illumination Final Result 2D Illustration Post-process ? ? ! ?

44. Work Pipeline of Volume Illustration 3D/4D Data Field Raw Data Input Data Data Preprocess Data Acquisition Interactive Illumination Final Result 2D Illustration Post-process !