Image Synthesis

1. Image Synthesis Shadows / Transparency

2. Inter-Object realism • Covers different kinds of interactions between objects • Increasing realism in the scene • Relationships between objects easier to understand • Shadows, Reflections, Transparency

3. Shadow techniques In illuminated scenes objects cast shadows on otherobjects • Important visual cue for object positioning • Scene without shadows doesn’t match reality

4. Shadow techniques Classification of shadow techniques • How to account for interaction between light, shadow casters and shadowed objects Projective Shadows Shadow Volumes Shadow Maps

5. Projective shadows Projective shadows • Shadows are projected copies of the shadowing objects onto shadowed surface • Good for complex objects casting shadows onto simple surfaces Objects Objects & projected objects Shadows

6. Projective shadows Projection of shadow casters onto planes • Shadow projection is a projective mapping of the object onto the shadowed plane • Directional or positional lights can be handled • Shadow projection transform Sisaddedto the end of the modelview matrix • S: render the scene from the light source position with the shadowed plane as the image plane L L+t(P-L)=(x,y,z)T P Ax+By+Cz+D=0

7. Projective shadows Multi-pass algorithm • Draw the surface to be shadowed • Render the object twice • Render the object with original matrix, colors and textures as specified for that object • Disable texturing and lighting • Render the object as its shadow with appended matrix S and the color of the shadow

8. Projective shadows Problems of projective shadows • Projection must be clipped to shadowed surface • Solution: clip the projected polygon to the surface using the stencil buffer Draw polygon setting the stencil buffer Draw shadows where stencil is set Use stencil as mask

9. Projective shadows Problems of projective shadows • Shadow color modulates receivers color • Receiver should not just becomedarker • Solution: • Render unlit surface again where stencil is set due to shadow projection • Shadow shows color of textured surface

10. Projective shadows • Problems of projective shadows • Z-Fighting: Projectedobjecthas „same“ depthvaluesasshadowed plane z-fighting correct

11. Projective shadows • Problems of projective shadows • In complex scenes multiple shadow projections are needed • #Matrices depends on #ShadowedSurfaces and #Lights • Objects have to be rendered multiple times • #PotentiallyShadowedSurfaces • Need to find surfaces that are in shadow • Only works well on planar surfaces • For complex objects clipping has to be performed against every face

12. Projective shadows Projectiveshadows in complex scenes Culling of non-shadowed surfaces Shadow clipped to multiple surfaces Modulate surface with shadow color

13. Shadow volumes Shadow volumes • Relies on a geometric representation of the spatial region that is shadowed by an object • Shadow polyhedra is generated in advance • Project shilouette edges of the object onto shadowed surface and connect points • Findingshilouettes not a simple task • Shadow algorithm now involves the stencil test and the depth test

14. Shadow volumes • Whenis a point in shadow? • Shadowvolumealgorithm[Crow 1977] • Intersectrayfromeyewithshadowvolume • Start with 0, invertateveryintersectionpoint • If 0, point not in shadow • Ifeye in shadow, startwith1

15. Shadow volumes • Shadow volume geometry Without shadows Shadow volume Final scene

16. Shadow volumes • Algorithm • Render the unlit sceneand clear the stencil buffer • Disabledepthwriteandcolorwrite but leavedepthtestenabled • Render front-facing polygons of the shadow volume and increment the stencil where they pass the depth test • Render back-facing polygons of the shadow volume and decrement the stencil where they pass the depth test • Result • Stencil value classifiesfragmentsasshadowed or unshadowed • Shadowed pixels end up with non-zero stencil

17. + + + - - - Illuminated (Behind Shadow Volumes) Shadow Volume Count = +1+1+1-1-1-1 = 0 Lightsource Shadowing object zero +1 zero Unshadowedobject +2 +2 Eyeposition +1 +3

18. + + + - Shadowed (inside Shadow Volumes) Shadow Volume Count = +1+1+1-1 = 2 Lightsource Shadowing object zero +1 zero Shadowedobject +2 +2 Eyeposition +1 +3

19. Illuminated (before Shadow Volumes) Shadow Volume Count = 0 Lightsource Shadowing object zero +1 zero Shadowedobject +2 +2 Eyeposition +1 +3

20. Problems Missed shadow volume intersection due to near clip plane clipping; leads to mistaken count Far clipplane zero +1 +1 +2 zero +3 +2 Near clipplane

21. Shadow volumesZFail • Algorithm • Render the unlit sceneand clear the stencil buffer • Disabledepthwriteandcolorwrite but leavedepthtestenabled • Render back-facing polygons of the shadow volume and increment the stencil where they fail the depth test • Render front-facing polygons of the shadow volume and decrement the stencil where they fail the depth test • Result • Stencil value classifiesfragmentsasshadowed or unshadowed • Shadowed pixels end up with non-zero stencil

22. Shadowvolumes OpenGLimplementation (Zfail) glDisable(GL_LIGHTING);glEnable(GL_DEPTH_TEST);glDepthFunc(GL_LESS);glDepthMask(0);glColorMask(0,0,0,0);glStencilMask(255);glEnable(GL_CULL_FACES); glCullFace(GL_FRONT);glStencilOp(GL_KEEP, GL_INCR, GL_KEEP); for(i=0;i<numPolysInScene; i++) renderShadowVolumeForPolygon(i); glCullFaces(GL_BACK);glStencilOp(GL_KEEP, GL_DECR, GL_KEEP); for(i=0;i<numPolysInScene; i++) renderShadowVolumeForPolygon(i);

23. Shadow volumes Problems of shadow volumes • Huge number of shadow geometries have to be generated, stored and rendered • Shadow volumes are in general too costly to be applied for complex scene • Volumes have to be re-generated whenever the light source moves relatively to object • Using simplified geometries can help

24. Hardware shadow volumes: Zfail • Render scene to initialize depth buffer • Depth values indicate the closest visible fragments • Use a stencil enter/leave counting approach • Draw shadow volume twice using face culling • 1st pass: render back faces and increment when depth test fails • 2nd pass: render front faces and decrement when depth test fails • Don’t update depth or color • Pixel’s stencil is non-zero if pixel in shadow, and zero if illuminated • Problem now is far clipping plane

25. Hardware shadow volumes: Zfail • Conclusion: shadow volume approach requires clipping against near or far plane to be avoided • Avoiding near plane clipping • Not really possible • Objects can always be behind you • Avoiding far plane clipping • Perspective make it possible to render at infinity

26. Hardware shadow volumes: Zfail • Recall: glFrustum projection matrix

27. Hardware shadow volumes: Zfail Projection matrix with infinite Far

28. Hardware shadow volumes: Zfail What happens to vertex at infinity? • Most distant vertex is (0,0,-D,0) where D>0 • OpenGL is looking along negative z • Transform (0,0,-D,0) to window space • Vertex is still in valid depth range; it is not clipped

29. Hardware shadow volumes Shadow volume must be closed Three sets of polygons close the shadow volume • Possible silhouette edges extruded to infinity away from the light position L • All of the occluder’s back-facing (w.r.t. the light) triangles projected away from the light to infinity • All of the occluder’s front-facing (w.r.t. the light) triangles are “slightly” projected away from the light Use homogeneous vector differences to project vertex V to infinity along the light source direction • V-L representdirectionheadingawayfromthelight • Has w coordinateequaltozero

30. Hardware shadow volumes • Example

31. Hardware shadow volumes Algorithmusingprogrammablevertexshader • Specifyforeachedge a shadowedgebyduplicatingedgevertices • Create so-calledzero-areaquads • Test forsilhouetteedge, front/back facingtriangles • Extruderespectiveverticestoinfinity in thevertexshader

32. Hardware shadowvolumes Algorithmusing DirectX10Graphics • Includesa geometryshader • Sitsbehindthevertexshader • Providesfunctionalitytoamplify/deamplify primitives • Forshadowvolumes: duplicatesilhouetteedgesandextrudetoinfinity

33. Shadow maps Shadow maps – a two-pass approach • Render the scene from the light source position using a virtual image plane • Grab the depth buffer and store it as a shadow map • Before rendering a fragment from the camera • Project it towards the light source onto the shadow map • Discard fragment if distance to light source is less than entry in shadow map Shadow map Scene

34. Shadow maps Implementation • Generatehomogeneoustexturecoordinates (s,t,r,q) aslightspacecoordinates (x,y,z,w) Texturematrixtobeissued

35. StepstoShadow Mapping Scene from Eye Scene from Light Depthfrom Light ProjectedDepthDepthComparison Final Scene

36. Shadow maps • Algorithm • Render scenefromlightsource • Read depthbufferandstoreas 2D texturemap • UseglCopyTexImage2DwithglReadBuffer(GL_DEPTH_BUFFER) • Render lit/texturedscene • Useshadow map as projective texture • Compare r/q texturecoordinatewithshadowmapentry • Alternativelyemployhardwaresupport • Does depth comparisonastexturefilteringoperation • Shadowmapfragmentcoloris 0 or 1 • Modulatefragmentcolorwithshadowmapcolor

37. Shadow maps Pros and cons of shadow maps • General method for computing all shadows in a scene • Representation of shadows is independent of scene complexity • But, artifacts due to discrete sampling of the shadow map and quantization of depth values 1024x1024 4096x4096

38. Shadow textures Shadow textures • Render shadow on thereceiverandloaditasshadowtexture • Light sourceandshadowobjectdependent Unshadowed Scene Shadow Texture Texture mapped floor

39. Shadow techniques Soft shadows • Similar to motion blur • Assumearealightsource • Jitterlightpositionwithinarea • Render fromjitteredpositions • Combine results in theaccumulationbuffer Use decreasing scale with increasing distance to center of area

40. Transparency How to handle non-opaque (semi-transparent) objects • Multiple fragments contribute to a pixel • Visibility order of fragments is important • Depth buffer can be used for hidden surface removal but not for semi-transparent objects

41. Transparency See-through objects Depth test discards objects Object order affects final color Blending non commutative 1C1 + (1-1)0C0  0C0 + (1-0)1C1

42. Transparency General solution • Visibility sort all semi-transparent objects in the scene • Render opaque objects first and update the depth values • Render semi-transparent objects in visibility order • Apply depth test but do not alter depth values • Use alpha-blending in order to obtain linear combination of src and dst color

43. Transparency Correct transparency by sorting and alpha-blending

44. Depth peeling • Recall: standarddepthtestgivesnearestfragmentateachpixelwithoutimposingany order • But: thereis also a second, third … fragment, depthtestdoes not provide a meansto render thenthnearestsurface. • Depthpeelingsolvesthisproblem: • Withn passesover a scene, n layersdeeperintothescenearegenerated • E.g. with 2 passes, thenearestandsecondnearestsurfacesareextracted • Boththedepthand RGBA informationisobtainedforeachlayer

45. Depth peeling Example

46. Depth peeling • The method peels different depth layers in front-to-back order • Drawback: as many rendering passes as objects depth complexity (maximum number of surface points mapping to one fragment)

47. Depth peeling Howtodeterminethedepthcomplexity • Render thescenewithoutdepthtestandincrementthestencilbit (orentries in a texture render targetusing additive blend) whenever a fragmentisgenerated • „Reduce“ thebuffertoobtainthemaximum • In each pass, maximumof 4 texelsiscomputedandstoredinto 1 texelofoutputtexture Reduce m x m region in fragment shader 1 pass: render quad that covers 1/4 pixels2 pass: render quad that covers 1/16 pixels …

48. Depth peeling Algorithm: • First pass: render as normal - depthtestgivesusnearestsurface • Second pass:Usedepthbuffercomputed in thefirst pass to “peelaway” depthslessthanorequaltonearestdepthsfromthefirst pass All otherdepths (forfragmentspassingthefirsttest) areresolvedbythestandarddepthtest • The second pass generatesdepthsofthesecondnearestsurface, whichareusedtopeelawaythefirstandsecondnearestsurfaces in thethird pass

49. Depth peeling Depthpeelingissues • Depthpeelingneedstwodepthtests • In thenth pass a testwiththe (n-1)thnearestdepths • The standarddepthtest • The firsttestcanbedone in a fragmentshader • Therefore, in each pass thedepthvaluesarerenderedinto a texture render target • The depthtestremainsactiveevenifrenderingisto such a target • In theupcoming pass, a shaderreadsthedepthvaluefromthistargetanddiscardfragmentswithdepthvalueslessthanthisvalue • All survivingfragmentsarepassedthroughthestandardtest • Use 32 Bit floattexture render targetforhighestaccuracy