95.4501

1. 95.4501 Ambient Occlusion

2. Ambient Occlusion • Ambient occlusion (AO): a “smart” ambient term that adds shadowing to diffuse objects lit with environment lighting; i.e., shadows in corners and creases in room without lights. • Screen space ambient occlusion (SSAO), a variation computed directly from information on the screen; e.g., the depth buffer or camera space positional information containing z… Corners have more occlusion than flat walls.

3. Ambient Occlusion (AO) Diagram from Hoang and Low (Multi-Resolution SSAO) AO is the lighting phenomenon under the direct illumination of a diffuse, uniform, spherical light source surrounding the scene.

4. The Issue • How much does the environment lighting contribute at a point? It depends how much occlusion there is in the volume visible from the sample point... Diagram from Loos and Sloan(Volumetric Obscurance)

5. Technically It’s A Volume Contribution Occlusion  is cosine-weighted fraction of a tangent hemisphere [1 - (di / dmax)] cosi Maximal occlusion 1 occurs when occlusion occurs immediately at p; i.e., di is 0 Minimal occlusion = 0 (no contribution, denoted ) for sample at or outside radius of influence or on negative side of normal) The AO Definition

6. Variations Obscurance SSAO Z-Buffer Based SSAO Multi-Resolution SSAO (MSSAO) Volumetric Obscurance (VO) Horizon-Based SSAO (HSSAO)

7. 95.4501 Brief OverviewOf a Number of Techniques

8. Technique 1: Obscurance obscurance Paper Available base point in green • From a point ALONG THE BASE POINT NORMAL, find how many samples in 3D sphere are obscured. COMPLEX (LIKE BUMPMAPPING) The Alchemy Screen-Space Ambient Obscurance Algorithm, McGuire, Osman, Bukowski, Hennessay, High Performance Graphics 2011.

9. Technique 2: Crisis • Use 3D sphere of points to sample the z buffer in the neighbourhood of the point (approximate occlusion by ratio of occluded to total points). works with z-buffer base point in green wall 0.5 inside corner 0.75 outside corner 0.25 Z-buffer View 3D View SIMPLE Screen Space Ambient Occlusion, Kajalin (Crysis), pp 413-424, ShaderX7, 2009.

10. Technique 3: MSSAO (AO Definition in Eye Space) Distance based Occlusion Actual occlusion is ignored but you have access to all unoccluded points. Uses a position bufferinstead of just z-buffersampling via 2D sphere MORE COMPLEX + DEMO LATER Additionally focuses on how a hierarchy can be used to improve AO for both near and far... We will use it as a basis for a reimplementation Multi-resolution screen-space ambient occlusion,, pp. 101-102, ACM Symposium on VR Software and Technology 2010. Weignore the factor 1/.

11. Technique 4: Volumetric Obscurance Say it’s fully occluded (1) if on negative side of normal; otherwise not (0). • This is a simplistic modification of #3 with only 1 pass (though it probably uses much larger filters). We will implement this as a MOD of MSSAO just to see how good/bad it is.

12. Technique 5: Horizon-Based SSAO MORE COMPLEX + DEMOS NVIDiA SDK 10 NVIDiA SDK 11 horizon-based • How far toward horizon you can rotate before hitting… Quantizes horizon angle + quantizes rotation angle around normal... Requires many samples in 3D sphere... Image-Space Horizon-Based Ambient Occlusion (NVIDIA demo), Bavoil and Sainz, pp 425-444, ShaderX7, 2009.

13. 95.4501 Preparing To Implement: Some Ideas

14. Ratios: We’ll want [1 - (di / dmax)] cosi d • Standard ratio increases linearly from 0 to 1. dmax dmax is radius of influence d2 • Squared ratio increases more slowly. dmax2 makes occlusion nearby (more crispy) Recall: occlusion 1 (at radius 0), occlusion 0 (at radius dmax). From Bavoil and Sainz (Horizon based…) and Hoang and Low (Multi-Resolution SSAO)

15. Details That Make it Better • Use per pixel normals (more accurately computed from derivatives)… From Bavoil and Sainz (Horizon based…) and Hoang and Low (Multi-Resolution SSAO)

16. Details That Make it Better • Blurring the result helps… even more if the blurring preserves edges. From Kajalin (Crysis)…

17. Details That Make it Better • Sample in direction of normal to eliminate those that are on the negative side (so fewer samples can be used). Applies mostly in 3D. From Briney et al, Fast Fake Global Illumination, ShaderX7, 2009

18. 95.4501 Hoang and Low multi-resolution screen-space ambient occlusion(MSSAO)

19. Advantages • Far (low frequency detail) and near (high frequency detail) integrated together... • Can be done with smaller filters without resorting to random sampling for lower resolutions. • Results do not suffer from noise and excessive blur (even though some blurring is used). Multi-resolution screen-space ambient occlusion, Hoang and Low, pp. 101-102, ACM Symposium on Virtual Reality Software and Technology 2010

20. Ambient Occlusion (AO) Look at Demo Diagram from Hoang and Low

21. What We Will Learn By ReImplementing the MSSAO Algorithm • It’s in OpenGL, so a switch from DirectX. • Some simple demo changes to allow experiments (including bug fixes) and changing the number of hierarchical steps. Original demo needs at least 3; we want 1 or more... • First use of poisson disk filter. • First use of downsampling median filter. • First use of gaussian blur filter. • First use of bilateral upsampling filter (what’s that). • Minor use of temporal coherence (what’s that). • Experiment with volumetric obscurancewhen done to see how good/bad that is...

22. Remember: We Sample in Eye Space To approximate integration, we sum the sample contributions and divide by their number. So all samples are on the surface We’ll revisit the full AO definition later. Weignore the factor 1/.

23. How Its Done • The idea will be presented in 3 ways: • In words: a bit verbose but accurate... • Diagrammatically: via diagrams based on (but not the same as) Hoang and Low’s slides. • In code: a piece of code with much less detail that might give you a better perspective...

24. How Its Done: In Words • The standard drawing pass uses MRT to create 2 render targets; a position buffer in camera coordinates and a normal buffer... (a buffer of attributes is referred to as a g-buffer; short for geometry buffer – predates the notion of geometry shaders which isn’t used here). • Draw successive full screen quads to ping pong the position/normal textures into lower and lower resolution textures again using MRT (a simple “median downsample” shader). • On your way back up from lower resolution levels, compute an AO texture using a diamondgrid filter and combine with a lower level AO texture (if there is one) with an additional blur pass. Note to Wilf... Make blur optional. • For the last pass, use poissonfilter and no blur...

25. How it’s Done: Diagrammatically using 3 levels... g-buffer (1024x1024) Initial Data g-buffer (1024x1024) g-buffer (512x512) g-buffer (256x256) Downsample diamond grid filter Compute AO #1 AO buffer (256x256) diamond grid filter Blur Blur AO buffer (256x256) poisson filter AO buffer (512x512) Compute A0 #2 Combined with Blurred AO #1 Blur AO buffer (512x512) Blur AO buffer (1024x1024) Compute A0 #3 2 Combined with Blurred AO #2 No Blur in last stage

26. How Its Done: In Code #define DownsampleGBufferDownsample Render2GBuffer (); //Hoang and Low’s name... for (int index = 1, size = RESOLUTION / 2; index < LEVEL_COUNT; ++index, size /= 2) { DownsampleGBuffer (size, index); //Each step decreases resolution... } for (intindex = LEVEL_COUNT - 1, size = minResolution; index >= 0; --index, size *= 2) { RenderAO (size, index); //Each step increases resolution. } Index (LEVEL_COUNT) strictly decrementing for downsampling g-buffer and strictly incrementing for building AO buffer bottom up...,

27. Combining 5 Levels Into ONE FINAL RESULT Each one ½ the resolution of previous one

28. Comparison With Other Techniques Comparison With Other Techniques Blizzard (noisier) MSSAO

29. Comparison With Other Techniques Horizon-based AO (noisier) MSSAO

30. Comparison With Other Techniques MSSAO Volumetric Obscurance

31. Ground Truth Comparison With Non-Real-time Technique MSSAO Blender

32. Summarizing: Less noise and blur and better high-frequency details MSSAO HBAO Blizzard

33. Performance Results • Scenes rendered at 1024x1024 on GeForce GTX 460M • Exclusive of geometry pass

34. Let’s Attack The Problem in Stages • Start with a brief look at the code (including simple extensions I made). • Have a look at the framebuffer creation code including the initial compiling of the shaders (and how we’ll simplify that). • Deal with the simpler aspects first; namely the downsampling code and shader(DownsampleGBuffer) and the blur code and shader (Blur). • Deals with the more complex code dealing with AO processing (RenderAO). • Temporal coherance (a minor aspect). Main Focus

35. 95.4501 Looking at the Code

36. Addition To Find Out Why Original Demo Wouldn’t Run • Implemented detectOpenGLError () and peppered code with it. inline void detectOpenGLError () {//WILF ADDITION... GLenum code = glGetError (); if (code != GL_NO_ERROR) { printf ("\nEncountered error %x \"%s\".", code, gluErrorString (code)); } } • Shader “geometry.vert” wasn’t compiling. //varying out vec4 pos; //wilfout vec4 pos;

37. Additions For Switching Between Implementations //WILF VERSION EXPERIMENTS... #define ORIGINAL_VERSION 0 #define WILF_VERSION 1 //#define WILF_version ORIGINAL_VERSION #define WILF_versionWILF_VERSION #define LEVEL_COUNT 5 // number of mip-map levels.//NOTE: ORIGINAL_VERSION needs to be 3 or more...

38. Additions To Support One Shader Include File • Shader #include is NOT part of this OpenGL. Added includePreprocessor to “shader.cpp”. • GLuintLoadShader(EShaderType type, char *file_name){ • printf ("Load shader \"%s\".\n", file_name); • char *temp = ReadFile(file_name); • temp = includePreprocessor (temp); //WILF • const char *shader_text = temp; • “Code to compile shader and return handle to it.” • }

39. Modification To Main • Looks Like it Used to Run from command line via “xxx.bat”. Took that out and instrumented it so one of 3 supplied models could be run. enumModelType {UseSponza, UseSibenik, UseConference}; ModelTypemodelType = UseSponza; //WILF

40. 95.4501 Framebuffer Creation Code

41. Declarations // buffers and textures ([0] = finest resolution) GLuintframeBufs [LEVEL_COUNT]; GLuintdepthBufs [LEVEL_COUNT]; GLuintposTex [LEVEL_COUNT]; GLuintnormTex [LEVEL_COUNT]; GLuintaoTex [LEVEL_COUNT]; GLuintaoTexBlur [LEVEL_COUNT]; GLuintlastFrameAOTex; GLuintlastFramePosTex; All handles Took out “randRotTex” since not used by any shader.

42. Frame Buffers Setup void SetupFBOs () {//WILF MADE SHORTER... #define initializeColorTexture(name,size) \ glBindTexture (GL_TEXTURE_RECTANGLE, name); \ glTexImage2D (GL_TEXTURE_RECTANGLE, 0, GL_RGBA32F, size, size, 0, GL_RGBA, GL_FLOAT, NULL); \ glTexParameterf (GL_TEXTURE_RECTANGLE, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); \ glTexParameterf (GL_TEXTURE_RECTANGLE, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); \ glTexParameteri (GL_TEXTURE_RECTANGLE, GL_TEXTURE_MAG_FILTER, GL_NEAREST); \ glTexParameteri (GL_TEXTURE_RECTANGLE, GL_TEXTURE_MIN_FILTER, GL_NEAREST); #define initializeDepthTexture(name,size) \ glBindRenderbuffer (GL_RENDERBUFFER, name); \ glRenderbufferStorage (GL_RENDERBUFFER, GL_DEPTH_COMPONENT, size, size); #define attachColorTexture(which,name) \ glFramebufferTexture2D (GL_FRAMEBUFFER, which, GL_TEXTURE_RECTANGLE, name, 0); #define attachDepthTexture(name) \ glFramebufferRenderbuffer (GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, name); #define errorCheck(name) \ if (glCheckFramebufferStatus (GL_FRAMEBUFFER) == GL_FRAMEBUFFER_COMPLETE) \ printf ("Framebuffer created successfully.\n"); \ else \ printf ("Framebuffer created unsuccessfully. Error code: %d.\n", glCheckFramebufferStatus (name)); #undefinitializeColorTexture #undefinitializeDepthTexture #undefattachDepthTexture #undefattachColorTexture #undeferrorCheck(name } code

43. Frame Buffers Setup void SetupFBOs () {//WILF MADE SHORTER... glGenFramebuffers (LEVEL_COUNT, frameBufs); glGenRenderbuffers (LEVEL_COUNT, depthBufs); glGenTextures (LEVEL_COUNT, posTex); glGenTextures (LEVEL_COUNT, normTex); glGenTextures (LEVEL_COUNT, aoTex); glGenTextures (LEVEL_COUNT, aoTexBlur); glGenTextures (1, &lastFrameAOTex); glGenTextures (1, &lastFramePosTex); initializeColorTexture (lastFrameAOTex, RESOLUTION); initializeColorTexture (lastFramePosTex, RESOLUTION); for (inti = 0, size = RESOLUTION; i < LEVEL_COUNT; ++i, size /= 2) { initializeColorTexture (posTex [i], size); initializeColorTexture (normTex [i], size); initializeColorTexture (aoTex [i], size); initializeColorTexture (aoTexBlur [i], size); initializeDepthTexture (depthBufs [i], size); glBindFramebuffer (GL_FRAMEBUFFER, frameBufs [i]); attachDepthTexture (depthBufs [i]); attachColorTexture (GL_COLOR_ATTACHMENT0, posTex [i]); attachColorTexture (GL_COLOR_ATTACHMENT1, normTex [i]); attachColorTexture (GL_COLOR_ATTACHMENT2, aoTex [i]); attachColorTexture (GL_COLOR_ATTACHMENT3, aoTexBlur [i]); if (i == 0) { attachColorTexture (GL_COLOR_ATTACHMENT4, lastFrameAOTex); attachColorTexture (GL_COLOR_ATTACHMENT5, lastFramePosTex); }errorCheck (frameBufs [i]); } }

44. Main Rendering Task glPushMatrix(); camera.ApplyCameraTransform (); Render2GBuffer (); DownsampleGBuffer LOOP in decreasing resolutionRenderAO LOOP in increasing resolution glPopMatrix(); if (oddFrame) glBindTexture(GL_TEXTURE_RECTANGLE, aoTex[0]); else glBindTexture(GL_TEXTURE_RECTANGLE, lastFrameAOTex); oddFrame = !oddFrame; glCopyTexSubImage2D (GL_TEXTURE_RECTANGLE, 0, 0, 0, 0, 0, RESOLUTION, RESOLUTION); glutSwapBuffers(); Optional temporal cohesion

45. 95.4501 Setting Up The Shaders

46. 3 Groups of Shaders SetupGeometryProgram (initial MRT textures) geometry.vertgeometry.frag Render2GBuffer SetupDownsamplePrograms (lower resolution MRT textures) ortho.vertdownsample.frag DownsampleGBuffer SetupAOPrograms (upsampling into AO texture) ortho.vertaoWilf.frag RenderAO MRT has a position texture + normal texture

47. Setting Up The AO (Upsampling) Shaders • Easier to look at actual code BUT we made one important design change. • Original version: Used 3 different AO shaders • aoLast.fragao.fragaoFirst.frag highest resolution middle resolutions lowest resolution • As mentioned on previous slide, we use one • aoWilf.frag

48. To Support the Use of JUST ONE SHADER • Created a struct in .cpp code • Added 3 shader variables to aoWilf.frag structShaderSetting {boolusePoisson; booluseUpsampling; booluseTemporalSmoothing;}; ShaderSettingcomputeShaderSetting (long levelCount) {ShaderSetting setting;setting.usePoisson = levelCount == 0; //highest is 0setting.useUpsampling = levelCount < (LEVEL_COUNT - 1) //lowest && LEVEL_COUNT > 1;setting.useTemporalSmoothing = true; //or false return setting;} • uniform boolusePoisson; uniform booluseUpsampling; uniform booluseTemporalSmoothing; Note to Wilf... Make blur optional. Now look at actual code

49. 95.4501 Drawing the INITIAL MRT Textures(Render2GBuffer)

50. Gbuffer Rendering (INITIAL MRT Textures) GLenum bufs01[2] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1}; GLenum bufs51[2] = {GL_COLOR_ATTACHMENT5, GL_COLOR_ATTACHMENT1}; void Render2GBuffer () { glUseProgram (geometryProg); glBindFramebuffer (GL_FRAMEBUFFER, frameBufs[0]); glDrawBuffers (2, oddFrame ? Bufs01 : bufs51); glPushAttrib (GL_VIEWPORT_BIT); glViewport (0, 0, RESOLUTION, RESOLUTION); glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); DrawModel (); glPopAttrib (); } first shader (geometry.vert + ...frag” USE MRT 0: posTex [i]1: normTex [i])5: lastFramePosTex All we draw Switching back and forth is for optional temporal cohesion