Advanced Graphics Part 3

1. Advanced Graphics Part 3 • What now? • So far: • Revised OpenGL • Discussed performance optimisation • Now: the OpenGL photorealism/special effects toolkit. • Some revision + some new material COMP9018 - Advanced Graphics

2. Reference • David Blythe • Advanced OpenGL programming • One of the SIGGRAPH lecture courses. COMP9018 - Advanced Graphics

3. The advanced OpenGL toolkit • Most of the OpenGL special effects are based on five basic ideas or combinations of them. • Textures (incl multitextures) • Blending (using alpha channel) • Accumulation buffer (adding images together) • Stencil buffer ("the cookie cutter") • Fog/depth cues (colours affected by distance from viewer) COMP9018 - Advanced Graphics

4. Plan ... • Look at each of these individually, then look at how they can be combined to do special effects, like shadows, reflection, caustics, etc. COMP9018 - Advanced Graphics

5. Textures • Staple of current polygonal graphics architectures. • Why? Allows you to add detail easily, without additional geometry. • Will quickly revise some aspects. • Quick demos: Nate's texture, VRML demos. COMP9018 - Advanced Graphics

6. Texture use • OpenGL supports 1D, 2D, 3D textures. • Most of the time, textures are images, not procedural. OpenGL supports only images. • Texel is a pixel in a texture image. Often used to avoid confusion with pixel. We'll be talking about texels mapping to pixels, so this is useful. COMP9018 - Advanced Graphics

7. Using textures • Each vertex in a polygon is defined as having texture coordinates. • Texture is stretched (i.e mapped) onto each polygon. • Texturing is one thing that really benefits from hardware acceleration. COMP9018 - Advanced Graphics

8. Steps in using textures • Creating the texture • Indicate how texture is applied • Enable texture mapping • Draw scene with both glVertex and glTexCoord calls. COMP9018 - Advanced Graphics

9. Creating textures • Messy! • Involves loading stuff up into memory and then telling system to accept it as texture. COMP9018 - Advanced Graphics

10. glTexImage2D() • One ugly function • glTexImage2D(target, level, internalFormat, width, height, border, format, type, data) • target: ? • level: for mipmaps – more later • internal format: how OpenGL stores the data internally. • width & height: obvious, but note: must be of form 2m + 2b where b is ... COMP9018 - Advanced Graphics

11. glTexImage2D cont'd • ... border. Must be either 0 or 1. Specify borders on side? OpenGL must support at least 64x64, but may support more. • format & type: how is image data stored in memory? • Finally, the data. COMP9018 - Advanced Graphics

12. Notes on format • Textures don't have to be RGB. • Can be RGB, RGBA, LUMINANCE or ALPHA or INTENSITY. • What's the "A" in RGBA? What's alpha? To do with blending. Talk about in a little while COMP9018 - Advanced Graphics

13. What if image is not a power of 2? • Use gluScaleImage() to correct. • Note: Textures can be, say 64x32. COMP9018 - Advanced Graphics

14. Funky stuff • Can read current frame buffer directly for texture map: • glCopyTexImage2D(target, level, internalFormat, x, y, width, height, border). • Can be useful sometimes ... many special effects use this as a hack. COMP9018 - Advanced Graphics

15. Repetition and Clamping • Can either say want lots of repetitions or just one copy. • Can control separately for each texture dimension (e.g. can repeat vertically, but not horizontally). • How to set glTexParameteri(GL_TEXTURE_2D, GL_WRAP_{ST}, GL_REPEAT | GL_CLAMP). COMP9018 - Advanced Graphics

16. What's with the target parameter? • Target is used for optimisation. • Why? Graphics card only has certain amount of memory. • Remember: right place at right time stuff? • target has two possible values: GL_TEXTURE_2D, or GL_PROXY_TEXTURE_2D. • Proxy is to make enquiries about space on graphics card, efficiency, etc. COMP9018 - Advanced Graphics

17. 1D and 3D textures • 1D textures useful, e.g. paintbrush or something. • 3D textures often arise in medical data (e.g. CAT scan, MRI etc). Examples of volume rendering. • HUGE amounts of data. COMP9018 - Advanced Graphics

18. Different texture modes • Replace: Overwrite previous values. • Modulate: Modify existing colour using textures (eg for lighting) • Decal: Like a sticker. Anyone built model cars? Attach sticker to outside, but transparent. • Blended: Mixed in some way. • Example: Nate’s texture. COMP9018 - Advanced Graphics

19. Different modes • How to think of them: existing value of pixel is f and texture is t, c is the current texture colour • Then replace is C = Ct, A = At (ie only texture matters). • Modulate is C = Cf.Ct, A = Af.At • Decal is C=Cf.(1-At)+Ct.At, A=Af • Blend is C=Cf.(1-Ct)+Cc.Ct; A = Af.At COMP9018 - Advanced Graphics

20. Uses of different mode • Replace: In 2D. • Modulate: Typical for use with lighted polygons. Most commonly, polygon used is white. • Decal: Insignias etc. • Blend: Not very frequently used, but can be used to mix in a "background colour". Kind of like "modulate". COMP9018 - Advanced Graphics

21. Enable texture mapping • glEnable(GL_TEXTURE_*D) COMP9018 - Advanced Graphics

22. Supplying coordinates • glTexCoord{1234}{sifd}{v}(coordinate). • 4? Yes, you can specify coordinates in homogenous form as well. (s,t,r,q). • Options: Repeating vs clamped. • Can do this independently for each texture. • glTexParameter(target, pname, param) • target = GL_TEXTURE_*D, pname, value). COMP9018 - Advanced Graphics

23. Automatic coord generation • Can use OpenGL to work out texture coords for you. • But quite complicated – and less efficient. • How? • You supply a linear equation for each texture coord, expressed in terms of a point's position coords of the form: • (p1,p2,p3,p4) depend on application. COMP9018 - Advanced Graphics

24. Texture coord generation • Can specify in either object coordinates (i.e. when glVertex() gets called), or eye coordinates. • 99 per cent of time, you want object (pre-transform) coordinates. • When would you need eye coordinates? Some effects depend on eye position, e.g. some textures. COMP9018 - Advanced Graphics

25. The OpenGL code • glTexGen{ifd}(coord, pname, param) • coord is GL_S, GL_T, GL_R or GL_Q • pname is GL_TEXTURE_GEN_MODE – then either GL_OBJECT_LINEAR, GL_EYE_LINEAR or GL_SPHERE_MAP • pname is GL_OBJECT_PLANE or GL_EYE_PLANE, then the next parameter is the vector p1, p2, p3, p4 COMP9018 - Advanced Graphics

26. Demo • Have a look at texgen.c COMP9018 - Advanced Graphics

27. So much to textures ... • Still to do: • Texture filtering • Mipmaps • Multitexturing • Sphere maps/Environment Maps • Specular reflection and textures • Light maps • Bump maps COMP9018 - Advanced Graphics

28. Texture management • Use texture objects. • Can speed things up. • Usage very similar to display lists. COMP9018 - Advanced Graphics

29. Initialisation • Get some texture ids • glGenTextures(n, textureids) • n is number of textures you want, textureids is an array of ints identifying the textures. COMP9018 - Advanced Graphics

30. Usage • Usage is a little odd. Think of current texture as part of the context. • glBind(type, textureid) sets the current texture. • Any definitions affect the current textureid. So to set up call glTexParameteri() and glTexImage2D(). • To change current texture, call glBind on another texture • Look at texbind.c COMP9018 - Advanced Graphics

31. Texture usage and hardware • Textures can live in memory or on graphics card. • If in memory have to be copied onto card whenever they need to be used. SLOW! • Why not keep all textures on card? Because card has limited memory. • Resident = texture lives on graphics card. • glAreTexturesResident(n, textureids, residentstates) can be used to work COMP9018 - Advanced Graphics

32. But I want CONTROL! • Not much use if you don't have control. • First of all, glDeleteTextures(n, textureids) is useful. • glPrioritizeTexures(n, textureids, priorities) • priority 1 = gotta be on the card, 0 = who cares? • LRU algorithms are sometimes used if textures have equal priority. COMP9018 - Advanced Graphics

33. Filtering • General problem: Texels and pixels are not going to be the same size. • Three possibilities: • Magnification of texture: one texel maps to many pixels. • Minification of texture: many texels map to one pixel. • Exact match: Yeah right! COMP9018 - Advanced Graphics

34. Magnification vs Minification COMP9018 - Advanced Graphics

35. Solving the magnification problem • Magnification leads to pixelation effects ... we can see the texels on the screen. • Ugly! • How to fix? • Linearly interpolate: Each intensity value is the centre of a pixel and then take a weighted average of surrounding four pixels. • Called bilinear filtering. COMP9018 - Advanced Graphics

36. Bilinear filtering (mag) Each box is one texel • Interpolate along AB to get intensity at E. • Interpolate along DC to get intensity at F. • Interpolate along EF to get intensity at G. E B A G Centre of pixel maps to this point in texture space D C F Texture Space (zoomed in to pixel level) Texture space COMP9018 - Advanced Graphics

37. Comparison • On previous slide, if using point sampling (GL_NEAREST), then it will be coloured black. • If using bilinear filtering, it is a weighted sum of the four surrounding pixels, depending on distance. • In this case, it would be coloured grey. COMP9018 - Advanced Graphics

38. What about minification? • Ok, linear interpolation fixes problem with close up, but what about far away? • Can also use linear interpolation, but what use is that? • Point of using linear interpolation was to "smooth" edges and pixels. • But real problem with min is the opposite. • Ideal solution: average values of pixels covered. • Obviously not feasible COMP9018 - Advanced Graphics

39. MIPMaps • A precomputation approach • Say we have a 64x64 texture, then we calculate 32x32, 16x16, 8x8, 4x4, 2x2 and 1x1 ahead of time. • Each is created by filtering higher resolution images down, so it is nicely rounded. • Fits efficiently into memory. • Depending on distance to object, we will use a different mipmap. COMP9018 - Advanced Graphics

40. MIPMap idea From OpenGL Programming Guide COMP9018 - Advanced Graphics

41. MIPMap issues • Idea: At different distances use the most appropriate MIPMap. • Generally, the most appropriate is the one that gives the closest to 1:1 texel:pixel ratio. • But can also interpolate between MIPMap levels. • Called trilinear mipmapping. COMP9018 - Advanced Graphics

42. How does this fix things? • Prevents aliasing. If we use point sampling, then basically it picks a random texel as colour. • This precomputes averages. COMP9018 - Advanced Graphics

43. Mipmaps in OpenGL • Two options: • Do it yourself: Rememeber glTexImage2D(target, level, internalFormat, width, height, border, format, type, data)? The level is the mipmap level. • Get gluBuild2DMipmap() to do the work for you. • Why not use gluBuild2Dmipmap? • Can use better filters • Might want to play tricks • Some textures should be mipmapped carefully COMP9018 - Advanced Graphics

44. Filtering ... • How to handle the issue of minification and magnification? • Have to apply a filter ... something that will combine data from multiple texels. • Note: Think of texel as providing value in CENTER of texel, not for whole texel. COMP9018 - Advanced Graphics

45. Magnification • When magnifying we can do two ways: • Point sampling: • Map pixel centre into texel space. Pick nearest texel as colour of pixel. glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST) • Linear sampling: • Map pixel centre into texel space. Pick neareast 4 texels and average using bilinear interpolation. glTexParameteri(", ", GL_LINEAR) COMP9018 - Advanced Graphics

46. Minification • Point sampling and linear sampling can be used as before. But glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST|GL_LINEAR) • But can now use mipmaps!! COMP9018 - Advanced Graphics

47. Minification types • GL_NEAREST_MIPMAP_NEAREST (choose nearest mipmap level, then choose nearest pixel in that mipmap) • GL_LINEAR_MIPMAP_NEAREST (choose nearest mipmap level, then do bilinear interpolation) • GL_NEAREST_MIPMAP_LINEAR (interpolate between nearest point in two mipmaps) • GL_LINEAR_MIPMAP_LINEAR (trilinear interp) COMP9018 - Advanced Graphics

48. Environment mapping • Reminder: Environment mapping is a hack to add reflection to polygon rendering. • Basic idea: • Render/photograph the world around the object of interest onto an enclosing shape. Turn this into a texture map • Reflect the ray from the viewer off the surface (like ray tracing) • Use the reflected ray as an index for the texture map. • Q: What “shape” is texture map. COMP9018 - Advanced Graphics

49. Environment mapping Representation of scene Reflected Ray Object Pixel in cube map that is mapped on to object N Vector from viewer COMP9018 - Advanced Graphics

50. Environment mapping in practice • Uses texture maps. • Is an example of view dependent textures: depends on position on viewer. • Coordinates are calculated whenever view changes. • Means that every time viewer's position changes, so does texture coordinates. COMP9018 - Advanced Graphics