Real-Time Volume Graphics

1. Real-Time Volume Graphics

2. Pre-Integration and High-Quality Filtering

3. This is what we want …

4. But often this is what we get …

5. Framebuffer Filtering Classification Shading Integration Sampling Volume Rendering Artifacts • Volume Rendering Pipeline • Where are errors introduced ? Linear filtering 8 bit blending Low sampling rate High frequencies Quantized Gradients

6. Sampling Artifacts

7. 284 Samples 128 Samples Sampling Artifacts • Reason: • Low sampling rate • Solutions: • Increase sampling rate to Nyquist frequency => at least 2 samples per voxel • Adaptive Sampling • Higher sampling rate where data contains high frequencies

8. Sampling Artifacts • Remove woodgrain artifact by stochastic jittering of ray-start position • Look-up noise texture to offset ray-position in view direction

9. Filtering Artifacts • 643 volume, object-aligned slices bi-cubic filter bi-linear filter

10. Filtering Artifacts • Reason: • Internal filtering precision dependent on input texture format • Linear filters do not approximate the ideal reconstruction filter (sinc-Filter) very well • Solutions: • Use internal format with higher precision • Implement a HQ filter in a fragment program

11. Discretized version of convolution integral Filtering Artifacts / HQ-Filters filter convolution discrete signal h(x) f[x] g(x) = f[x] * h(x) Filter width: 2m

12. Filtering Artifacts / HQ-Filters • Procedural evaluation of kernel and convolution • Texture-based kernel and convolution FetchInputSamples();ComputeWeights_X();for (i=0; i<3; i++) Convolution_X();ComputeWeights_Y();Convolution_Y(); pre-sampled B-spline kernelfrom [Hadwiger et al. 2001] see for example NVIDIA’sBicubicTexMagnification demo

13. Filtering Artifacts / HQ-Filters • Procedural evaluation of kernel and convolution • Texture-based kernel and convolution FetchInputSamples();ComputeWeights_X();for (i=0; i<3; i++) Convolution_X();ComputeWeights_Y();Convolution_Y(); R G B A t0 t1 t2 t3 pre-sampled B-spline kernelfrom [Hadwiger et al. 2001] see for example NVIDIA’sBicubicTexMagnification demo

14. Transform original kernel into look-up texture Replicate kernel tile over output grid Filtering Artifacts / HQ-Filters kernel tile

15. Easy when kernel separable Use same 1D kernel tile for all axes Sample two (or three) timesand multiply weights Separable bi-cubic andtri-cubic filters need onlyone 1D RGBA kernel tile Filtering Artifacts / HQ-Filters

16. Filtering Artifacts / HQ-Filters • Distribution instead of gathering • Works for all filter shapes and sizes • Works for separable and non-separable kernels • Multi-pass evaluation of filter convolution sum

17. Tri-cubic gradients and second derivatives for isosurfaces possible in real-time! Filtering Artifacts / HQ-Filters

18. Filtering Artifacts / HQ-Filters linear cubic

19. Filtering Artifacts / HQ-Filters linear cubic

20. Classification Artifacts Pre-Classification Filtering Classification … … Post-Classification Filtering Classification … … Pre-Integrated-Classification Pre-Integration Filtering Integral Lookup … …

21. T(s) T(s(x)) s Classification Artifacts • High frequencies in the transfer function T increase required sampling rate s(x) x x

22. 50 x more slices Classification Artifacts multiple peaks T(s) s

23. Classification Artifacts / Pre-integration • Reason: • Classification can add high frequencies to rendered data • Solution: • Pre-Integrated Classification=> split numerical integration into • one pre-integration for the TF • one integration for the scalar field • slab-by-slabrendering

24. Classification Artifacts / Pre-integration slice-by-slice slab-by-slab image-order sample-by-sample raySegment-by-raySegment object-order

25. Eye Classification Artifacts / Pre-integration Slab Screen sb sf

26. Pre-Integration of all possible combinations save values In table sf sb sb sf Classification Artifacts / Pre-integration • Pre-processing

27. Classification Artifacts / Pre-integration • Rendering project back slice sf sb front slice back slice

28. Classification Artifacts / Pre-integration // define inputs from application struct appin { float4 Position : POSITION; float4 Normal : NORMAL; float4 TCoords0 : TEXCOORD0; }; // define outputs from vertex shader struct vertout { float4 HPosition : POSITION; float4 TCoords0 : TEXCOORD0; float4 TCoords1 : TEXCOORD1; }; vertout main(appin IN, uniform float4x4 ModelViewProj,// model-view-projection matrix uniform float4x4 ModelViewI, // inverse of the modelview matrix uniform float4x4 TexMatrix, //Texture matrix for texture 0 uniform float SamplingDistance ) Cg Vertex Program

29. Classification Artifacts / Pre-integration { vertout OUT; OUT.TCoords0 = mul(TexMatrix, IN.TCoords0); // transform view pos vec and view dir to obj space float4 vPosition = mul(ModelViewI, float4(0,0,0,1)); float4 vDir = normalize(mul(ModelViewI, float4(0.f,0.f,-1.f,1.f))); //compute position of virtual back vertex float4 eyeToVert = normalize(IN.Position - vPosition); float4 backVert = {1,1,1,1}; backVert = IN.Position - eyeToVert * (SamplingDistance /dot(vDir,eyeToVert)); //compute texture coordinates for virtual back vertex OUT.TCoords1 = mul(TexMatrix, backVert); // transform vertex position into homogenous clip-space OUT.HPosition = mul(ModelViewProj, IN.Position); return OUT; } Cg Vertex Program

30. utilize sf and sb as texture coordinates for the dependent texture sb sf Classification Artifacts / Pre-integration • Rendering fetch sf and sb during rasterization sf sb slice

31. fetch pre-integrated ray-segment sb sf • Rendering texture polygon with pre-integrated value sf sb slice

32. Classification Artifacts / Pre-integration struct v2f_simple { float4 Hposition : POSITION; float3 TexCoord0 : TEXCOORD0; float3 TexCoord1 : TEXCOORD1; float4 Color0 : COLOR0; }; float4 main(v2f_simple IN, uniform sampler3D Volume, uniform sampler2D TransferFunction, uniform sampler2D PreIntegrationTable) : COLOR { float4 lookup; //sample front scalar lookup.x = tex3D(Volume, IN.TexCoord0.xyz).x; //sample back scalar lookup.y = tex3D(Volume, IN.TexCoord1.xyz).x; //lookup and return pre-integrated value return tex2D(PreIntegrationTable, lookup.yx); } Cg Fragment Program

33. Classification Artifacts / Pre-integration • Pre-integration table to large • 12bit x 12bit x RGBA8 = 4096x4096x32bit = 64MB • Pre-integration table to slow to compute • even with integral functions • Solution: • use Integral functions • Store integral functions in 1D floating point texture • two 1D table lookups

34. Classification Artifacts / Pre-integration • Almost no performance penalty in sw raycasting • Cache last scalar value • Currently slower on graphics hardware • Temporary fragment registers are zeroed • Two lookups into the volume per fragment => multiple integration steps @ once (raycasting) • Pre-integrated volume rendering still not “correct” • Assumes linear progression of scalar value along ray=> Gaussian Transfer Functions (Kniss Vis’03)

35. 128 slices pre-classification 128 slices post-classification 284 slices post-classification 128 slices pre-integrated Classification Artifacts / Pre-integration

36. Classification Artifacts / Pre-integration single step

37. Classification Artifacts / Pre-integration multiple peaks

38. Classification Artifacts / Pre-integration many peaks

39. Classification Artifacts / Pre-integration acoustic volume inside car cabine

40. Classification Artifacts / Pre-integration inhomogeneous region homogeneous region

41. Shading Artifacts

42. Shading Artifacts • Reasons: • Gradients are pre-computed and quantized • Interpolation of normals causes unnormalized normals • Solutions: • Store gradients in high-precision texture,re-normalize in fragment program  too much memory • Compute gradients on-the-fly  high-quality gradient • many texture fetches • Sobel gradient

43. Shading Artifacts • Add offset to texture coordinate of fragment // samples for forward differences half3 normal; half3 sample1; sample1.x = (half)tex3D(Volume, IN.TexCoord0+offset.xwww).x; sample1.y = (half)tex3D(Volume, IN.TexCoord0+offset.wyww).x; sample1.z = (half)tex3D(Volume, IN.TexCoord0+offset.wwzw).x; // additional samples for central differences half3 sample2; sample2.x = (half)tex3D(Volume, IN.TexCoord0-offset.xwww).x; sample2.y = (half)tex3D(Volume, IN.TexCoord0-offset.wyww).x; sample2.z = (half)tex3D(Volume, IN.TexCoord0-offset.wwzw).x; // compute central differences gradient normal = normalize(sample2.xyz – sample1.xyz); Cg Fragment Program

44. Shading Artifacts • Pass in shifted texture coordinates // samples for forward differences half3 normal; half3 sample1; sample1.x = (half)tex3D(Volume, IN.TexCoord2).x; sample1.y = (half)tex3D(Volume, IN.TexCoord4).x; sample1.z = (half)tex3D(Volume, IN.TexCoord6).x; // additional samples for central differences half3 sample2; sample2.x = (half)tex3D(Volume, IN.TexCoord3).x; sample2.y = (half)tex3D(Volume, IN.TexCoord5).x; sample2.z = (half)tex3D(Volume, IN.TexCoord7).x; // compute central differences gradient normal = normalize(sample2.xyz – sample1.xyz); Cg Fragment Program

45. Blending Artifacts 16 bit floating point blending 32 bit floating point blending 8 bit fixed point blending

46. Blending Artifacts • Reason: • 8 bit blending accumulates error in the framebuffer • Solutions: • Blending into a floating point pbuffer • Nv4x support 16 bit floating point blending • Nv3x, R3xx and R4xx do not support floating point blending=> implement blending in a fragment program • Ping-pong blending to prevent read/write race conditions

47. Blending Artifacts • The graphics hardware does not support floating point blending (NV3x, R3xx, R4xx)=> implement blending yourself • Clean: ping-pong Blend A B

48. Blending Artifacts • The graphics hardware does not support floating point blending (NV3x, R3xx, R4xx)=> implement blending yourself • Clean: ping-pong Blend A B

49. Blending Artifacts • The graphics hardware does not support floating point blending (NV3x, R3xx, R4xx)=> implement blending yourself • Clean: ping-pong Blend A B

50. Blending Artifacts • The graphics hardware does not support floating point blending (NV3x, R3xx, R4xx)=> implement blending yourself • Dirty: ping without the pong Blend A B