Volume Visualization Visualization II MSIM 842, CS 795/895

1. Volume VisualizationVisualization IIMSIM 842, CS 795/895 Instructor: Jessica Crouch

2. Volume Viz Problem: • Data points fill a 3D volume • You can only display one 2D image at a time • Naïve rendering of a volume will just let you see the surface • How can you visualize the volume data in a way that lets you understand the internal structure of the data?

3. Structured vs. Scattered Data • The best approach will depend on the characteristics of the data • Data points arranged in rows, columns, and layers (3D image) can be referred to as voxels (volumetric-pixels) • Data points that follow a non-gridded pattern are called “scattered” Wikipedia image

4. Volume Viz & Data Dimensionality • Each data point has • 3D coordinates (x,y,z), specifying a position in the volume and one of the following: • Scalar – a single number, 0D data • Vector – a list of numbers, 1D data • Tensor – a matrix or ND array of numbers, for N>1

5. Data examples • Scalar Data • CT Scan → x-ray opacity for each voxel • PET Scan → positron radiation for each voxel • Atmospheric simulation (or measurement) → pollutant concentration • Ocean temperature distribution • Vector Data • Velocity vectors at each voxel • Fluid flow (river water, arterial blood, etc.) • Displacement vectors at each voxel • Provides mapping between different deformed configurations of the same object • Tensor data • Higher dimensional data is more complicated: • Stress and strain matrices • Gradients of 3D functions

6. Simplest Case • Let’s consider continuous, scalar, voxel data • Geometrically regular arrangement (3D grid) • Just 1 data value per voxel • Think of CT data (3D x-ray) • What would you be interested in seeing? • How would you construct a visualization?

7. Scalar Voxel Viz. • Simplest possible visualization is to just look at the slices • Use color mapping (or grayscale mapping) to see slice values • Slices can be useful • We can do more interesting things

8. Scalar Voxel Viz. • Bone voxels will be bright (have large values), other voxels will be dark • How could you visualize the surface of the bones? • Several alternatives exist, including: 1. Render an isosurface: Use Marching Cubes Algorithm 2. Ray Casting: Integrate scalar variable along the view direction 3. Splatting

10. Isosurfaces from 3D • How to construct a polygonal surface from a 3D dataset? • Find points on the isosurface of a scalar variable • 3D analog of 2D contour maps • You pick a meaningful threshold value • Voxels with data values below the threshold sit on one side of the isosurface, voxel with data values above the threshold sit on the other • Simplifying assumption is that threshold value = 0 • To force this, just subtract the desired threshold value from all data points • Tessellate points on the isosurface to create a polygonal representation of the surface • The “Marching Cubes” algorithm does this • This is one of the famous, older (>20 yrs) viz. algorithms • Render the tessellated isosurface using normal 3D graphics methods

14. Isosurfaces & Opacity

15. The Visible Man: Image Data

16. Visible Man Isosurfaces

18. Marching Cubes Considered • How effective is the visualization perceptually? • What can you see? • What can’t you see? • How is this in terms of efficiency? • Phase I • Phase II • Rendering • Compare other approaches…

19. Ray Casting Volume Data

20. Review Ordinary Ray Casting • How does it work?

22. Ray Casting with a Voxel

24. Ray Casting for Approximation of Light Integral

26. Examples

27. Ray Casting Considered • How effective? • How are the images different from those produced by Marching Cubes? • How efficient? • Another alternative…

28. Volume Splatting • Ray casting is an image space method • Ray are generated per pixel, and travel out into the volume • Volume splatting is an object space method • Data points in the volume are mapped (splatted) onto the image plane

29. Splatting: A Feed Forward Process Feed forward: Splatting Splat!

30. Splatting (feed-forward) ?

31. Fill the holes We need to fill the pixel values between the volume projection samples ?

32. 3D Kernel for Splatting Need to know the 3D extent of each voxel, and Then project the extent to the image plane This is called ‘footprint’

34. Visibility • How does occlusion work? • Splatting uses back to front compositing • Samples in front are added on top of and partially obscure previously splatted (further back) samples

35. Gaussian kernel • A popular kernel is the Gaussian function (think bell curve) • Sigma controls the width • Gaussians have circular “splat” footprints

36. Ray Casting vs. Splatting • What are the differences in the images produced? • Think about aliasing • Efficiency considerations? • In what order are the voxel data points accessed? • For ray casting? • For splatting?

37. Summary of Techniques • Marching cubes: • Construct polygonal isosurfaces • Render with regular graphics pipeline • Ray casting: • Image space algorithm • Shoot rays from image into volume, integrate color & opacity along each ray • Volume splatting: • Object space algorithm • Project each voxel onto the image plane These are good ways of looking directly at the recorded data. What could you do to focus attention on significant features in the data?

38. Feature Extraction • What is a significant feature? • Answer is application dependent • Often, regions with a high gradient are significant • What is the gradient of a voxel volume? • Look at the gradient of a 2D image…

39. Derivative in X direction

40. Derivative in Y direction

41. Gradient Direction Vectors dI/dx: dI/dy:

42. Volume gradient • Is a 3x1 vector [dI/dx dI/dy dI/dz] • Vector points “uphill” in direction of increasing data value • Vector length (magnitude) indicates how quickly data values in the neighborhood are changing

43. Gradient Magnitude • Given scalar data, the gradient magnitude can be computed • Result is another scalar data set • Each voxel value gets the length of the gradient vector computed for the original data • Large gradient magnitude values in a region indicate sharp changes in the original data

44. Gradient Magnitude Viz. • Sharp changes are often significant • May indicate an edge or surface • Large gradient in CT data indicate a voxel is on bone surface • May indicate where something important is happening • Large temperature gradient in atmosphere can indicate a weather “front” • Can apply the three visualization methods already discussed to the gradient magnitude data

45. Other feature extraction methods • Can apply other types of pre-processing to the input data to find areas of interest • Search for blobs of a certain size • Search for areas with a certain texture or pattern • Filter out high frequency noise from input data • Create new data set by assigning voxel values based on how well original data matches the search criteria • Have to know what you’re looking for, what defines “significant” for a particular application

46. Still to come… • Consideration of volume viz. for higher dimensional data • Vectors • Tensors • Time-varying sequences • Data with probabilistic uncertainty

48. Streamlines Follow the flow of a vector field, tracing the path a particle would take

49. Tensor Glyphs • Ellipsoids, Cuboids, Superquadratics (see right), & other 3D polyhedra and implicit surface shapes • For 3x3 matrices, illustrates 3 axes (or eigenvectors)