Physically based simulation on GPU using OpenGL and GLSL

1. Physically based simulation on GPU using OpenGL and GLSL Yalmar Ponce Atencio

2. Outline • Multitexture mapping OpenGL fixed functionality vs. GLSL • Render to texture PBuffers vs. The OpenGL Framebuffer Object Extension • Applications (physical simulation) • Particle systems • Cloth • Rigid and deformable objects

3. Outline • Multitexture mapping OpenGL fixed functionality vs. GLSL • Render to texture PBuffers vs. The OpenGL Framebuffer Object Extension • Applications (physical simulation) • Particle systems • Cloth • Rigid and deformable objects

4. Brief history • Since the OpenGL 1.0 definition (1992) • However, limited. Just to apply images to the surface of an object • Later, in OpenGL 1.1 (1995) texture objects was added • In OpenGL 1.2 (1997) 3D textures was added • In OpenGL 1.3 (1999) new hardware capabilities was exposed. Allow access to two or more texture objects simultaneously • In OpenGL 1.4 (2003) was added support for depth textures and shadows

5. Simple texturing sample  • Good example http://www.xmission.com/~nate/tutors.html

6. Multitexture with fixed functionality = + • Need GL_ARB_multitexture extension (now is part of OpenGL 2.0 core) • glActiveTextureARB (GL_TEXTUREn_ARB) • glMultiTexCoord…

7. Combining textures • First, a class or function for loading image files is needed void DrawFrame(void) { ... glTranslatef(0.0,2.0,0.0); glBindTexture(GL_TEXTURE_2D, texture1); drawBox(2.0); glTranslatef(0.0,-2.0,0.0); glBindTexture(GL_TEXTURE_2D, texture2); drawBox(2.0); ... }

8. Combining textures • Next, setup for using multitexturing void drawFrame(){ ... glEnable(GL_TEXTURE_2D); glActiveTextureARB(GL_TEXTURE0_ARB); glBindTexture(GL_TEXTURE_2D, texture1); glTexEnvf (GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE,GL_COMBINE_EXT); glTexEnvf (GL_TEXTURE_ENV, GL_COMBINE_RGB_EXT, GL_REPLACE); glActiveTextureARB(GL_TEXTURE1_ARB); glBindTexture(GL_TEXTURE_2D, texture2); glTexEnvf (GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE,GL_COMBINE_EXT); glTexEnvf (GL_TEXTURE_ENV, GL_COMBINE_RGB_EXT, GL_INCR); drawBox(2.0); ... } void drawBox(float size){ // Texture Coordinate (0,0) is top left. glBegin(GL_QUADS); glMultiTexCoord2f(GL_TEXTURE0, 0.0, 1.0); glMultiTexCoord2f(GL_TEXTURE1, 0.0, 1.0); glVertex3f(0.0, 0.0, 0.0); glMultiTexCoord2f(GL_TEXTURE0, 0.0, 0.0); glMultiTexCoord2f(GL_TEXTURE1, 0.0, 0.0); glVertex3f(0.0, size, 0.0); glMultiTexCoord2f(GL_TEXTURE0, 1.0, 0.0); glMultiTexCoord2f(GL_TEXTURE1, 1.0, 0.0); glVertex3f(size, size, 0.0); glMultiTexCoord2f(GL_TEXTURE0, 1.0, 1.0); glMultiTexCoord2f(GL_TEXTURE1, 1.0, 1.0); glVertex3f(size, 0.0, 0.0); glEnd();}

9. Bump mapping example = +

10. Bump mapping example • Configuring a given texture unit glActiveTexture(GL_TEXTUREn);glBindTexture(GL_TEXTURE_2D, texName);glTexImage2D(GL_TEXTURE_2D, …);glTexParameterfv(GL_TEXTURE_2D, …);…glTexEnvfv(GL_TEXTURE_ENV, …);glTexGenfv(GL_S, …);glMatrixMode(GL_TEXTURE);glLoadIdentity(); • Setting texture coordinates for a vertex glMultiTexCoord4f(GL_TEXTURE1,s1, t1, r1, q1);glMultiTexCoord3f(GL_TEXTURE2,s2, t2, r2);glMultiTexCoord2f(GL_TEXTURE3,s3, t3);glMultiTexCoord1f(GL_TEXTURE4,s4);glVertex3f(x, y, z);

11. Bump mapping example:Mapping code void display(){ ... //Set up texture environment to do (tex0 dot tex1)*color glActiveTextureARB(GL_TEXTURE0_ARB); glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE_ARB); glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB, GL_TEXTURE); glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB, GL_REPLACE); glActiveTextureARB(GL_TEXTURE1_ARB); glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE_ARB); glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB, GL_TEXTURE); glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB, GL_DOT3_RGB_ARB); glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE1_RGB_ARB, GL_PREVIOUS_ARB); drawTorus(2.0, 0.5); ...}

12. Bump mapping example:Results • References • http://www.paulsprojects.net/opengl/ • http://www.opengl.org/resources/tutorials/sig99/shading99/course_slides/

13. Why must be used GLSL? • Programmability allow to exploits much more the texture mapping capabilities • With programmable shaders, APPs can read and use the texture values in a way makes sense • Textures can also be used to store intermediate results: • Lookup tables • normals • Factors • Visibility information • Etc.

14. Combining textures:GLSL version • A class or function that load vertex and/or fragment programs is needed • Changing the DrawFrame routine … ... GLSLKernel myShader; ... myShader.fragment_source("fragment_program"); myShader.vertex_source("vertex_program"); myShader.install(); ... void DrawFrame(void) { ... myShader.use(); myShader.setUniform("texture0", 0); // Texture Unit 0 myShader.setUniform("texture1", 1); // Texture Unit 1 drawBox(2.0); myShader.use(false); ... }

15. Combining textures:GLSL version • and the drawBox routine too void drawBox(float size){ // Texture Coordinate (0,0) is top left. glBegin(GL_QUADS); glTexCoord2f(0.0, 1.0); glVertex3f(0.0, 0.0, 0.0); glTexCoord2f(0.0, 0.0); glVertex3f(0.0, size, 0.0); glTexCoord2f(1.0, 0.0); glVertex3f(size, size, 0.0); glTexCoord2f(1.0, 1.0); glVertex3f(size, 0.0, 0.0); glEnd(); }

16. Combining textures: GLSL version • The vertex program • The fragment program void main(void) { gl_TexCoord[0] = gl_TexCoord0; gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex; } uniform sampler2D texture0, texture1; void main (void){ vec4 texel0 = texture2D(texture0, vec2(gl_TexCoord[0])); vec4 texel1 = texture2D(texture1, vec2(gl_TexCoord[0])); gl_FragColor = 0.5*(texel0 + texel1); }

17. Using multitexture with several shaders • References • OpenGL Shading Language book • http://www.clockworkcoders.com/oglsl/ • http://www.lighthouse3d.com/opengl/glsl/

18. Conclusions • GLSL or another GPU programming language allow exploit much more texture mapping capabilities • Several shaders can be used in an APP, anyone to a specific APP feature • Walls with bump mapping • Water effects • Fire effects • Complex material effects (roughness, reflection, refraction, glass, etc.)

19. Outline • Multitexture mapping OpenGL Fixed functionality vs. GLSL • Render to texture PBuffers vs. The OpenGL Framebuffer Object Extension • Applications (physical simulation) • Particle systems • Cloth • Rigid and deformable objects

20. Later, PBuffers • Pixel buffers • Designed for off-screen rendering • Similar to windows, but non-visible • Window system specific extension • Select from an enumerated list of available pixel formats using • ChoosePixelFormat() • DescribePixelFormat() • ...

21. Framebuffer Object • Only requires a single OpenGL context Only • switching between Framebuffers is faster than switching between PBuffers (wglMakeCurrent) • No need for complicated pixel format selection • Format of Framebuffer is determined by texture or Renderbuffer format • Puts burden of finding compatible formats on developer • More similar to Direct3D render target model • Makes porting code easier • Renderbuffer images and texture images can be shared among Framebuffers • e.g. share depth buffers between color targets • saves memory

22. Framebuffer Object • OpenGL Framebuffer is a collection of logical buffers • Color, depth, stencil, accumulation • Framebuffer object extension provides a new mechanism for rendering to destinations other than those provided by the window system • Window system independent • Destinations known as “Framebuffer attachable images”. Can be: • Off-screen buffers (Renderbuffers) • Textures

23. Framebuffer Object • Framebuffer-attachable image • 2D array of pixels that can be attached to a framebuffer • Texture images and renderbuffer images are examples. • Attachment point • State that references a framebuffer-attachable image. • One each for color, depth and stencil buffer of a framebuffer. • Attach • The act of connecting one object to another. Similar to “bind”. • Framebuffer attachment completeness

24. Framebuffer Object • Introduces two new OpenGL objects: • Framebuffers and Renderbuffers • Framebuffer (FBO) • Collection of framebuffer attachable images (e.g. color, depth, stencil) • Plus state defining where output of GL rendering is directed • Equivalent to window system • When a framebuffer object is bound its attached images are the source and destination for fragment operations • Color and depth textures • Supports multiple color attachments for MRT • Depth or stencil renderbuffers

25. Framebuffer Object arquitecture TEXTURE OBJECTS … GL_COLOR_ATTACHMENT0 TEXTURE IMAGE GL_COLOR_ATTACHMENTn DEPTH ATTACHMENT RENDER BUFFER OBJECTS STENCIL ATTACHMENT RENDERBUFFER IMAGE OTHER STATES

26. Framebuffer Object API • Creating a FBO (similar to create a texture) void GenFramebuffersEXT(sizei n, uint *framebuffers); • Delete a FBO void DeleteFramebuffersEXT(sizei n, const uint *framebuffers); • Check if a given identifier is a FBO boolean IsFramebufferEXT(uint framebuffer); • Binding a FBO void BindFramebufferEXT(enum target, uint framebuffer); • Check status of the FBO enum CheckFramebufferStatusEXT(enum target);

27. Framebuffer Object API • Attaches image from a texture object to one the logical buffers of the current bound FBO void FramebufferTexture1DEXT(enum target, enum attachment, enum textarget, uint texture, int level); void FramebufferTexture2DEXT(enum target, enum attachment, enum textarget, uint texture, int level); void FramebufferTexture3DEXT(enum target, enum attachment, enum textarget, uint texture, int level, int zoffset); • Automatically generate mipmaps for texture images attached to target void GenerateMipmapEXT(enum target);

28. Binding a FBO void BindFramebufferEXT(enum target, uint framebuffer); • target must be FRAMEBUFFER_EXT • framebuffer is FBO identifier • All GL operations occur on attached images • If (framebuffer == 0), GL operations operate on window system provided framebuffer (it’s default)

29. Attaching textures to a FBO void FramebufferTexturenDEXT(enum target, enum attachment, enum textarget, uint texture, int level); • target must be FRAMEBUFFER_EXT • attachment Is one of: • GL_COLOR_ATTACHMENTn_EXT • DEPTH_ATTACHMENT_EXT • STENCIL_ATTACHMENT_EXT • textarget must be one of: • GL_TEXTURE_2D • GL_TEXTURE_RECTANGLE_ARB • GL_TEXTURE_CUBE_MAP_... • level is the mipmap level of the texture to attach • texture is the texture object to attach • If (texture==0), the image is detached from the framebuffer

30. Framebuffer completeness • Framebuffer is complete if all attachments are consistent • Texture formats make sense for attachment points i.e., don’t try and attach a depth texture to a color attachment • All attached images must have the same width and height • All images attached to COLOR_ATTACHMENTn_EXT must have same format • If not complete, glBegin will generate error INVALID_FRAMEBUFFER_OPERATION

31. Checking Framebuffer Status enum CheckFramebufferStatusEXT(enum target); • Should always be called after setting up FBOs • Returns enum indicating why FBO is incomplete FRAMEBUFFER_COMPLETE COMPLETEFRAMEBUFFER_INCOMPLETE_ATTACHMENTFRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENTFRAMEBUFFER_INCOMPLETE_DUPLICATE_ATTACHMENTFRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXTFRAMEBUFFER_INCOMPLETE_FORMATS_EXTFRAMEBUFFER_INCOMPLETE_DRAW_BUFFER_EXTFRAMEBUFFER_INCOMPLETE_READ_BUFFER_EXTFRAMEBUFFER_UNSUPPORTED FRAMEBUFFER_STATUS_ERROR • If result is “FRAMEBUFFER_UNSUPPORTED”, application should try different format combinations until one succeeds

32. FBO usage • FBO allows several ways of switching between rendering destinations • In order of increasing performance: • Multiple FBOs • Create a separate FBO for each texture you want to render to??? • Single FBO, multiple texture attachments • Textures should have same format and dimensions • Use FramebufferTexture() to switch between textures • Single FBO, multiple texture attachments • Attach textures to different color attachments • Use glDrawBuffer() to switch rendering to different color attachments

33. FBO Performance Tips • Don’t create and destroy FBO every frame • Try to avoid modifying textures used as rendering destinations using TexImage, CopyTexImage, etc. • More references and examples • http://www.gpgpu.org/developer/

34. Outline • Multitexture mapping OpenGL Fixed functionality vs. GLSL • Render to texture PBuffers vs. The OpenGL Framebuffer Object Extension • Applications (physical simulation) • Particle systems • Cloth • Rigid and deformable objects

35. Brief overview • Use textures to represent physical features (e.g. positions, velocities, forces) • Fragment shader calculates new values and render this results back to textures • Each rendering pass draws a single quad, calculating values to next time step on the simulation • Actually, graphic processors supports textures with NPT (non-power-of-two) and signed float values in each (RGBA) color channel • GL_TEXTURE_RECTANGLE_ARB • Float precision of fragment programs allows GPU physic simulation match CPU accuracy • New fragment programming model (longer programs, flexible dependent texture reads) allows much more interesting simulations

36. Basic Example:Cloth simulation • Can be used the Verlet integrator • Jakobsen, GDC 2001 • Avoids storing explicit velocity x’ = 2x – x* + ā.∆t²x* = x • Not always accurate, but stable! • In a GPU version, current and previous positions should be stored in 2 RGB float textures • Then swap current and previous textures for each time step

37. Cloth simulation:The algorithm • GPU • Two passes (two shaders) • Perform integration (move particles) • Satisfy constraints • Distance constraints between particles • Floor collision constraints • Collision constraints with other objects (spheres) • CPU • XYZ vertices  RGB texture pixels (read pixels) • Compute normals and render final mesh

38. Cloth simulation:Diagram old positions TEX0 VERLET INTEGRATOR FRAGMENT SHADER next positions SATISFY CONSTRAINTS FRAGMENT SHADER curr positions next positions TEX2 curr positions TEX1 Adjacency TEX3

39. Cloth simulation:Integration pass code uniform sampler2DRect oldp_tex;uniform sampler2DRect currp_tex;void Integrate(inout vec3 x, vec3 oldx, vec3 a, float timestep2){ x = 2*x - oldx + a*timestep2; }void main(){ float timestep = 0.002; vec3 gravity = vec3(0.0, -9.8, 0.0); vec2 uv = gl_TexCoord[0].st; // get current and previous position vec3 oldx = texture2DRect(oldp_tex, uv).rgb; vec3 x = texture2DRect(currp_tex, uv).rgb; // move the particle Integrate(x, oldx, gravity, timestep*timestep); // satisfy world constraints x = clamp(x, worldMin, worldMax); SphereConstraint(x, spherePos, 1.0); gl_FragColor = vec4(x, 1.0);}

40. Cloth simulation:Satisfy constraints code // constrain a particle to be a fixed distance from another particlevec3 DistanceConstraint(vec3 x, vec3 x2, float restlength, float stiffness){ vec3 delta = x2 - x; float deltalength = length(delta); float diff = (deltalength - restlength) / deltalength; return delta*stiffness*diff;}// as above, but using sqrt approximation sqrt(a) ~= r + ((a- r*r) / 2*r), if a ~= r*rvec3 DistanceConstraint2(vec3 x, vec3 x2, float restlength, float stiffness){ vec3 delta = x2 - x; float deltalength = dot(delta, delta); deltalength = restlength + ((deltalength - restlength*restlength) / (2.0 * restlength)); float diff = (deltalength - restlength) / deltalength; return delta*stiffness*diff;}// constrain particle to be outside volume of a spherevoid SphereConstraint(inout vec3 x, vec3 center, float r){ vec3 delta = x - center; float dist = length(delta); if (dist < r) { x = center + delta*(r / dist); }}

41. Cloth simulation:Constraints pass code uniform vec2 ms; // mesh sizeuniform float constraintDist;uniform vec3 spherePos;uniform sampler2DRect x_tex;uniform vec3 worldMin;uniform vec3 worldMax;void main(){ const float stiffness = 0.2; // this should really be 0.5 vec2 uv = gl_TexCoord[0].st; // get current position vec3 x = texture2DRect(x_tex, uv).rgb; // get positions of neighbouring particles vec3 x1 = texture2DRect(x_tex, uv + vec2(1.0, 0.0)).rgb; vec3 x2 = texture2DRect(x_tex, uv + vec2(-1.0, 0.0)).rgb; vec3 x3 = texture2DRect(x_tex, uv + vec2(0.0, 1.0)).rgb; vec3 x4 = texture2DRect(x_tex, uv + vec2(0.0, -1.0)).rgb; // apply distance constraints // this could be done more efficiently with separate passes for the edge cases vec3 dx = vec3(0.0); if (uv.x < ms.x) dx = DistanceConstraint(x, x1, constraintDist, stiffness); if (uv.x > 0.5) dx = dx + DistanceConstraint(x, x2, constraintDist, stiffness); if (uv.y < ms.y) dx = dx + DistanceConstraint(x, x3, constraintDist, stiffness); if (uv.y > 0.5) dx = dx + DistanceConstraint(x, x4, constraintDist, stiffness); x = x + dx; gl_FragColor = vec4(x, 1.0);}

42. Cloth simulation:A screenshot 128x128 mesh 256x256 mesh

43. Cloth simulation:Used textures Previous positions Current positions

44. More complex example:Rigid and deformable bodies • As cloth modeling: • Vertices  particles • Current and previous positions are mapped on textures • Edges  contraints • A rigid tetrahedron • Four vertices • Six constraints v3 v1 v2 v0 Texture0 Texture1 ContraintsTexture

45. Rigid tetrahedron simulation

46. h e g f d a c b Extensions:Rigid cube Texture0 Texture1 ContraintsTexture

47. Rigid cubes:Screenshot

48. Improving the performance:Render to vertex array • Enables interpretation of floating point textures as geometry • Possible on NVIDIA hardware using the “NV_pixel_data_range” (PDR) extension • Allocate vertex array in video memory (VAR) • Setup PDR to point to same video memory • Do glReadPixels from float buffer to PDR memory • Render vertex array • Happens entirely on card, no CPU intervention

49. Videos • Cloth with 64x64 mesh on CPU • Cloth with 64x64 mesh on GPU • Cloth with 128x128 mesh on CPU • Cloth with 128x128 mesh on GPU • 400 cubes on GPU

50. Results:P4 3.6Ghz + NVidia GeForce 6800