OpenGL ES and BREW™

1. OpenGL ES and BREW™ Dave Astle, GameDev.net, Qualcomm dave@gamedev.net, dastle@qualcomm.com

2. Overview • The BREW EGL and OpenGL ES API layer • Setup steps for IGL and IEGL • Tips using BREW OpenGL ES with ATI hardware • Qualcomm/BREW OpenGL ES platforms • General and platform-specific performance tips • Things ahead for OpenGL ES applications • OpenGL ES BREW samples • Links to more info

3. BREW gaming APIs • Current graphics APIs available: • IGRAPHICSISPRITE • IFONT • IIMAGE • IBITMAP / IDIB • ITRANSFORM • IDISPLAY • IGL (standard OpenGL ES Common-lite profile) • IEGL (standard EGL) • Other useful APIs for gaming • IMEDIA (video and audio) • ISOUND • IPOSDET (gps) • ISOCKET • ICAMERA • Some BREW APIs in the works • Fixed point math library • sin, cos, tan, etc. • MDP (Multimedia display processor) • IJOYSTICK

4. QDSP ARM OpenGL ES API layer EGL (AEEGL.h/.c) OpenGL ES (AEEGL.h/.c) BREW API OEMgles (OEMgles.h/.c) OEMegl (OEMegl.h/.c) OEM lnterface OpenGL-ES/EGL (libGLES_CL.lib) graphics driver DMSS Interface 3D GRP (Available on Enhanced Platforms) Hardware Interface

5. OpenGL ES API layer with ATI2300 EGL (AEEGL.h/.c) OpenGL ES (AEEGL.h/.c) BREW API OEMgles (OEMgles.h/.c) OEMegl (OEMegl.h/.c) OEM lnterface graphics driver DMSS Interface ATI 2300 Hardware Interface

6. Steps for using IGL • Include the AEEGL.h header file #include "AEEGL.h" • Create an instance of the IGL interface: if (ISHELL_CreateInstance(pMe->a.m_pIShell, AEECLSID_GL, (void**)&pMe->m_pIGL) != SUCCESS) return FALSE; // you can now use the BREW IGL interface // Ex. IGL_glGetError(pMe->m_pIGL); • When the application exits, release the IGL interface IGL_Release(pMe->m_pIGL);

7. Steps for using IEGL • Include the AEEGL.h header file #include "AEEGL.h" • Create an instance of the IEGL interface: if (ISHELL_CreateInstance(pMe->a.m_pIShell, AEECLID_EGL, (void**)&pMe->m_pIEGL) != SUCCESS) return FALSE; // you can now use the IEGL interface Ex. IEGL_eglWaitGL(pMe->m_pIEGL); • When the application exits, release the IEGL interface: IEGL_Release(pMe->m_pIEGL);

8. Steps for using IGL with gl wrappers • Include the IGL.h wrapper header file #include "IGL.h" • Create an instance of the IGL interface and initialize the IGL wrappers if (ISHELL_CreateInstance(pMe->a.m_pIShell, AEECLSID_GL, (void**)&pMe->m_pIGL) == SUCCESS) IGL_Init(pMe->m_pIGL); else return FALSE; // You can now use the standard OpenGL ES functions // Ex. glGetError(); • When the application exits, release the IGL interface IGL_Release(pMe->m_pIGL);

9. Using IEGL with egl wrappers • Include the IGL.h wrapper header file #include "IGL.h" • Create an instance of the IEGL interface and initialize the IEGL wrappers if (ISHELL_CreateInstance(pMe->a.m_pIShell, AEECLSID_EGL, (void**)&pMe->m_pIGL) == SUCCESS) IEGL_Init(pMe->m_pIEGL); else return FALSE; // You can now use the standard EGL functions // Ex. eglWaitGL(); • When the application exits, release the IGL interface IGL_Release(pMe->m_pIGL);

10. IGL setups • For using Standard OpenGL ES APIs • #include “IGL.h” • Create Instance for IGL • Must call IGL_Init() • Must call IGL_Release() on exit • Must build your app with GL.c • For using BREW IGL APIs • #include “AEEGL.h” • Create Instance for IGL • Must call IGL_Release() on exit • OpenGL ES APIs vs. IGL APIs • IGL • Standard typical BREW interface, works like all other BREW interfaces • Provides the ability to create an interface in BREW • Must create an interface first to initialize the OpenGL ES APIs • OpenGL ES • Standard OpenGL ES APIs • Portable across all platforms supporting OpenGL ES • The wrappers provide a way to build a dynamic downloadable application without linking the OpenGL ES library into the application. The OpenGL ES library is already on target and optimized with the hardware.

11. BREW OpenGL ES APIs OpenGL ES APIs // Smooth shading glShadeModel( GL_SMOOTH ); // Depth Test glEnable( GL_DEPTH_TEST ); // GL Initialization glViewport( 0, 0, GetScreenWidth(), GetScreenHeight() ); // Init Projection Matrix glMatrixMode( GL_PROJECTION ); glLoadIdentity(); Both API formats can be used in BREW IGL APIs // Smooth shading IGL_glShadeModel( pIGL,GL_SMOOTH ); // Depth Test IGL_glEnable( pIGL,GL_DEPTH_TEST ); // GL Initialization IGL_glViewport( pIGL, 0, 0, GetScreenWidth(), GetScreenHeight() ); // Init Projection Matrix IGL_glMatrixMode( pIGL,GL_PROJECTION ); IGL_glLoadIdentity(pIGL);

12. The GL.c file Provides wrappers for the OpenGLES and EGL standard API functions For example: GLAPI void APIENTRY glActiveTexture(GLenum texture) { IGL_glActiveTexture(GPIGL,texture); } GLAPI void APIENTRY glDrawArrays(GLenum mode, GLint first, GLsizei count) { IGL_glDrawArrays(GPIGL,mode,first,count); } GLAPI void APIENTRY glDrawElements(GLenum mode, GLsizei count, GLenum type, const GLvoid *indices) { IGL_glDrawElements(GPIGL,mode, count, type, indices); } …

13. Using BREW SDK with ATI hardware • Must have the ATI PCI card installed in your PC • In the BREW SDK modules directory: • C:\Program Files\BREW SDK v2.1.0\Bin\Modules • Must use the ATI_OpenGLESExtension.dll • Remove the OpenGLESExtension.dll from the modules directory • ATI_OpenGLESExtension.dll • Wrapper around ATI’s drivers • Will only work with ATI hardware installed • OpenGLESExtension.dll • QUALCOMM’s software implementation of OpenGL ES • Will work with BREW SDK 2.1 and above • Do not use with ATI hardware

14. Using BREW SDK with ATI hardware drivers • In the BREW SDK bin directory: • C:\Program Files\BREW SDK v2.1.0\Bin • Must have the ATI driver files in this directory • ATI Driver files that need to be in path or SDK \bin directory: • ahi.dll • ahioem.dll • atihwtbl0.txt • ati_opengl_es.dll

15. BREW SDK errors with ATI drivers • If the following files are missing: • \bin • ahi.dll • ahioem.dll • ati_opengl_es.dll • \Modules • ATI_OpenGLESExtension.dll • The following BREW SDK error will occur:

16. BREW SDK errors with ATI drivers • If the following ATI file is missing: • \bin • atihwtbl0.txt • Should at least be in your executable path • The following BREW SDK error will occur:

17. Using BREW EGL types with ATI hardware • The ATI NativeWindowType is now an IDIB* • Standard QUALCOMM BREW EGL data types typedef IDisplay* NativeDisplayType; typedef IDIB* NativeWindowType; typedef IDIB* NativePixmapType; • // Create a surface for the main window • if ((eglSurface = eglCreateWindowSurface(eglDisplay, eglConfig, (NativeWindowType)&screen_surface, NULL)) == EGL_NO_SURFACE) {…}

18. Using BREW EGL with ATI hardware // Get the display device MUST PASS EGL_DEFAULT_DISPLAY if ((eglDisplay = eglGetDisplay(EGL_DEFAULT_DISPLAY )) == EGL_NO_DISPLAY) {…} // Initialize the display if (!eglInitialize(eglDisplay, NULL, NULL)) { …} // Obtain the first configuration with a depth buffer if (!eglChooseConfig(eglDisplay, attrs, &eglConfig, 1, &numConfig)) {…} // Create a surface for the main window if ((eglSurface = eglCreateWindowSurface(eglDisplay, eglConfig, (NativeWindowType)&screen_surface, NULL)) == EGL_NO_SURFACE) {…} // Create an OpenGL ES context if ((eglContext = eglCreateContext(eglDisplay, eglConfig, EGL_NO_CONTEXT, NULL)) == EGL_NO_CONTEXT) {…} // Make the context and surface current if (!eglMakeCurrent(eglDisplay, eglSurface, eglSurface, eglContext)) {…}

19. Default EGL configuration • QUALCOMM • Attributes • ATI • NOTES: • ATI • Only supports one EGL configuration for the W2300 • QUALCOMM • Currently only supports one EGL configuration • Will support additional configurations for future platforms (i.e. enhanced/convergence series )

20. Using ATI OpenGL ES extensions • Must use the standard methods provided in OpenGL ES when calling ATI or ARB extensions in the BREW environment. • (DO NOT use the ATI extension APIs provided for Windows) • Should not call the extension directly • ATI and ARB extensions will not be exposed through the IGL interface. • ATI Windows method of doing this: • #include “gl_extensions.h” #ifdef _win32 GL_API void GL_APIENTRY glBindBufferARB(GLenum target,GLuint buffer); GL_API void GL_APIENTRY glDrawVertexBufferObjectATI(void); #endif • Game.c glBindBufferARB(GL_ARRAY_BUFFER_ARB, n+1); glDrawVertexBufferObjectATI();

21. Using ATI OpenGL ES extensions in BREW • Standard OpenGL ES way of doing this: • Game.h typedef void (APIENTRY *PFNGLBINDBUFFERARBPROC) (GLenum target,GLuint buffer); typedef void (APIENTRY *PFNGLDRAWVERTEXBUFFEROBJECTATIPROC) (void); PFNGLBINDBUFFERARBPROC glBindBufferARB; PFNGLDRAWVERTEXBUFFEROBJECTATIPROC glDrawVertexBufferObjectATI; • Game.c glBindBufferARB = (PFNGLBINDBUFFERARBPROC) eglGetProcAddress("glBindBufferARB"); glDrawVertexBufferObjectATI = (PFNGLDRAWVERTEXBUFFEROBJECTATIPROC) eglGetProcAddress(" glDrawVertexBufferObjectATI"); … glBindBufferARB(GL_ARRAY_BUFFER_ARB, n+1); glDrawVertexBufferObjectATI();

22. Determining which extensions are supported • Standard OpenGL ES glGetString() routine: • glGetString(GL_EXTENSIONS) • Returns a space-separated list of supported extensions • See isExtensionSupported() routine (for reference only) • glGetString(GL_VENDOR) • Returns the company responsible for this GL implementation • 6550 vendor will be QUALCOMM • 6000 series with ATI external chip vendor will be ATI • 7000 series vendor could be just QUALCOMM because of integrated ATI core with QUALCOMM’s MSM chip • glGetString(GL_RENDERER) • Returns the name of the renderer • This name is typically specific to a particular configuration of a hardware platform • Best bet to check this value for hardware configuration

23. Determining which EGL extensions are supported • Standard EGL eglQueryString() routine: • eglQueryString(dpy, EGL_EXTENSIONS ) • Returns a space-separated list of supported extensions • eglQueryString(dpy,EGL_VENDOR) • Returns the company responsible for this GL implementation • 6550 vendor will be QUALCOMM • 6000 series with ATI external chip vendor will be ATI • 7000 series vendor could be just QUALCOMM because of integrated ATI core with QUALCOMM’s MSM chip

24. 6800 6250 6280 6500 6275 6550 7500 6225 6200 7600 7200 6300 6700 2Q ‘03 2006 2Q ‘04 2Q ‘03 4Q ‘04 2Q ‘02 MID ‘04 4Q ‘05 2H ‘05 1Q ‘05 4Q ‘04 1Q ‘05 3Q ‘02 MSM Product Roadmap • MULTIMODE |All GSM/GPRS Capable CDMA2000 + GSM-GPRS WCDMA (UMTS) 1X 1xEV-DO(Rev. 0) 1xEV-DV • (Rev. D) 1xEV-DO(Rev.A) + UMTS/ EDGE/HSDPA GSM/GPRS EDGE/HSDPA CONVERGENCE PLATFORM Dual CPU, Single Chip ENHANCED PLATFORM Enhanced Multimedia & Graphics MULTIMEDIA PLATFORM • Multimedia & • 2D/3D Graphics VALUE PLATFORM Integrated gpsOne Voice & Data Voice

25. Multimedia vs. Enhanced vs. Convergence Platform Graphics

26. Phone Performance Paradigm • Handset devices bring unique considerations for game design • How do you like it when your phone battery dies? • Do not use 4M triangles/sec if you do not need them • Do not force 60 frames/sec when 30 will do • Do not spin while waiting for an event • Use event timer with callback: BREW set timer function • Convert to fixed point to reduce power consumption • Use BREW fixed point math library • Optimize for good visualization vs power consuption • Optimize game play for best game play vs power usage • Optimize memory accesses • Don’t load textures multiple times unnecessarily • Users do not like to wait for downloads • Optimize for small footprint for application and assets • Page in levels rather than downloading all at once

27. Phone Performance Paradigm (2) • Handset has less memory than console devices • Use fixed point data structures for reduced footprint • Keep memory clean – watch out for run time memory bloat • Clean up application after complete – not always automatic • LCD is smaller and configuration varies • Optimize assets for smaller screen size • Design for flexible aspect ratio • Optimize for reduced color depth • Input devices • Current phones rely on keypad and 4 point rocker button • Optimize for keypad interface on today’s phones • Include Enhanced input devices for next year’s phones • Analog joy stick, rocker and shoulder buttons

28. Multimedia Platform Performance Characteristics • Fixed point pipeline • Reduced power consumption • Precision optimized for ARM/QDSP performance • Performance Specifications • 50K lighted triangles/sec • 400K textured, Z buffered pixels/sec • 1M shaded, Z buffered pixels/sec • 8M flat shaded 2D pixels/sec • Performance Characteristics • Optimized for vertex lighting and shaded triangles • Transformations offloaded from ARM • Fast clears • Target depth complexity of 1-2 for QCIF screen at 15fps

29. Multimedia Platforman Architecture View of Graphics ARM 9 CPU Game Control Vertex Assembly MODEM HOST MEMORY- COLOR and Z BUFFERS QDSP-4 Vertex Transforms PER Vertex Lighting Clipping Viewport Transform Back-Face Culling Polygon Offset Audio Triangle Setup ARM 9 CPU QDSP-4 AHB BUSES ARM 9 CPU LCD Interpolation Texture Generation Texture Application Vertex Fog Interpolation Scissor Test Alpha Test Depth Test Blending Logic Ops.

30. Multimedia Platforma Software View of Graphics

31. Multimedia Platform Tips and Tricks • OpenGL ES Lite Profile • Be careful with matrix/vertex precision • ARM is fixed point • Strongly recommend using fixed point for game engine where possible • Consider the ARM-QDSP-ARM architecture • Transform and lighting are fast on QDSP • Most likely will need to optimize for rasterization • Default OpenGL ES state assumes hardware acceleration • Not optimal for software/firmware pipeline • Set initial state for the features you NEED • Default state for texture filtering is trilinear - set it to bilinear • Do not forget to turn off state settings you are not using • Make sure you disable alpha blending after use • Set shading to flat if using texture replace mode, otherwise shading is computed

32. Multimedia Platform Tips and Tricks • Vertex lighting (1 light source) has no performance degradation • Including ambient specular and diffuse components • Use OpenGL vertex lighting to offload lighting from ARM • Careful!! VertexLighting reduces performance onEnhanced Platform • glDrawElements is faster than glDrawArrays • Go ahead and use it – it’s more flexible • ARM rasterization and fragment operations • Send large batches of DrawElements – make big arrays • Pack triangles with similar state into single arrays • Performance saving of 20% to 50% for strip and fan packing • Some game libraries average 3.6 vertices per array (Bad!) • System has early Z test (before textures if state allows) • Where possible draw large objects in front first

33. Multimedia Platform Tips and Tricks • glClear() is accelerated • Use glClear() instead of drawing triangles • Make sure ColorMask is off • Use scissor test to clear a portion of the screen • Do not clear if not necessary • If pasting texture on background use BREW 2D IIMAGE copy function • Use glflush() or glSwapbuffers() to flush 3D pipe • Use glClear(GL_DEPTH_BUFFER_BIT) for clear Z buffer • glFinish() is costly • SwapBuffers() automatically does glFinish() • Do not do redundant finishes • Use glFinish() only when necessary • For off-screen rendering

34. Multimedia Platform Tips and Tricks • Be especially careful with textures Pixels/Sec Pixels/Sec Smooth Shaded Flat Shaded Textured Non Perspective Corrected Perspective Corrected Turn of perspective correction when possible using glHint(): 25% Performance increase turning off perspective correct Use smooth shaded if textured pixels not required: 2X performance Increase

35. Multimedia Platform Tips and Tricks • Be especially careful with textures (continued) Pixels/Sec Pixels/Sec Nearest Bilinear Bilinear MIPMAP Use nearest instead of bilinear filtering when possible: 20% performance increase Use bilinear instead of MIPMAP filtering: 2X performance increase

36. Multimedia Platform Tips and Tricks • Be especially careful with textures (continued) • All OpenGL ES texture formats are supported: • RGB565, RGBA5551, RGBA4444, RGB888, RGBA8888 • Palletted Texture formats are supported • L8, A8, LA88 • All formats get converted to one of three internal formats during texture load • RGB565, RGBA5551, RGBA4444 • Paletted, 24, and 32, bit textures take longer to load • Luminance, Alpha and Luminamce-Alpha get unpacked into 565 and 4444 • Match texture format as closely to internal format as possible • Load textures once per level (or only when necessary)

37. Enhanced Platform Performance Characteristics • Fixed point pipeline • Every datapath and computational unit custom sized • Optimized for power/performance • Reduced area = reduced cost = higher volumes • Performance Specifications • 100K triangles/sec • 7M pixels/sec • Performance Characteristics • QDSP is not held up by rasterization • ARM9 is freed up for game play • Predictable complex rendering performance • Less system impact on graphics performance • Target depth complexity of 1-3 for QVGA screen at 30fps

38. Enhanced Platforman Architecture View of Graphics ARM 9 CPU Game Control Vertex Assembly MODEM HOST MEMORY- COLOR BUFFER QDSP-4 Vertex Transforms PER Vertex Lighting Clipping Viewport Transform Back-Face Culling Polygon Offset Audio ARM 9 CPU MOBILE DISPLAY PROCESSOR QDSP-4 GRP AHB BUSES Triangle Setup Interpolation Texture Generation Texture Application Vertex Fog Interpolation Scissor Test Alpha Test Depth Test Blending Logic Ops. GRP LCD MDP Z-BUFFER (QVGA) 2D Image & Window Operations Sampling & Format Conversion (YCbCt => RGB) Rotations Scaling Blending

39. Enhanced Platform -a Software View of Graphics

40. Enhanced Platform Tips and Tricks • Consider the ARM-QDSP-GRP architecture • Hardware rasterization pipeline provides significant performance increase • Most likely will need to optimize for transform and lighting • Do not use OpenGL ES vertex lighting unless necessary • Default OpenGL ES state is not optimal • Initialize OpenGL ES to the state you require • Vertex assembly optimizations same as Multimedia Platform • Send large batches of DrawElements – make big arrays • Like Multimedia Platform, use glDrawElements instead of glDrawArray • Fog is vertex based – will affect triangle rate • Disable fog when not in use

41. Enhanced Platform Tips and Tricks • Lines are rendered as triangles • Diagonal lines cause testing of non-visible fragments • Draw short diagonal line segments instead of long ones • Unlike Multimedia Platform, no difference between flat and smooth shaded triangle performance • Color buffer is in host memory • Clearing color buffer is time consuming • Do not clear color buffer if not redrawing entire frame • Use scissor test if clearing partial color buffer • Clearing depth buffer is fast • Depth Buffer is dedicated local memory • Do not clear unless you need to • Use glClear(GL_DEPTH_BUFFER_BIT ) to clear depth only

42. Enhanced Platform Tips and Tricks • Early Z test • Texture lookup eliminated • Use depth test to eliminate unnecessary work • Draw big objects in front first • Tuning for textures • Bilinear texture filtering HIGHLY recommended • MIPMAP is OpenGL ES default – turn it off • Bilinear 25Xfaster than MIPMAP textures • 16K texture cache • Use small textures – less than 16K • Sort drawing by textures to increase cache efficiency • Choose texture sizes to reduce stride • Avoid glCopyTexSubImage() - invalidates texture cache • No performance impact for perspective corrected textures • Same internal texture formats are supported as on Multimedia Platform • Use same recommendations for texture formats as Multimedia Platform

43. Convergence Platform Performance • Performance Specifications • 4M shaded triangles/sec • 3.5M textured trianges/sec • 133M textured pixels/sec • 65M multitextured pixels/sec • Target depth complexity > 10 on VGA screen at 30fps

44. Development Timeline

45. Things coming soon • The 6550 slingshot FFA (prototype multimedia phone) • Selected game developers will get these phones for development • QCT will be providing direct engineering support for this platform

46. Enhanced Platform Development FFA Stereo Speaker 1 VGA Camera Rocker Buttons 320x240 18 bit Color Display 128x96 18 bit Color LCD Antenna Joystick Stereo Speaker 2 Shoulder Buttons 1 & 2 Soft Keys LED Flash 5 Way Nav Send/back/End Fire Buttons 3 & 4 2 Mega Pixel CCD Camera Keypad Receiver Mic Removable Storage

47. Enhanced Platform Development FFA Shoulder Buttons 1, 2, 3 & 4 Stereo Speakers Rocker Buttons 320x240 18 bit Color Display Removable Storage Gaming Mode Joystick Hand Grip Accessory

48. Cool things ahead • The MDP (Mobile-Display-Processor) • Frame update synchronization • Rotation (90, 180, 270 degree) • Flip (Flip left to right and top to bottom) • Video functions • Chroma Upsampling/Color Base Conversion (YCbCr 4:2:0 to RGB 666 or RGC 888 format) • Video frame size downscale by n/4 , Video frame size upscale by 2 • Graphics functions • Alpha-blend, Transparency, Text Overlay

49. OpenGL ES Apps on BREW • MotoGP 2 by Climax/THQ • Sample apps

50. Getting started and more information • Download the BREW SDK • www.qualcomm.com/brew/developer • Sample OpenGL ES BREW applications available • QCT OpenGL ES web pages • www.cdmatech.com/solutions/products/q3dimension Demo 01 Demo 02 Demo 03 Demo 04