Adaptive Streaming and Rendering of Large Terrains

1. Adaptive Streaming and Rendering of Large Terrains Raphaël Lerbour Advisors: Kadi Bouatouch (IRISA) Jean-Eudes Marvie (THOMSON)

2. Terrain rendering • 2D maps of elevation and color samples • 3D applications: games, GPS, virtual tourism…

3. Objectives • Render large terrain datasets • Support huge maps (dozens of GB) • Remote loading • With good interactivity • Unpredicted user viewpoint moves • Speed requirements (e.g. 25 frames per second) • Real-time hardware 3D rendering • On multiple target devices and networks • Desktop PCs, handhelds… • Cannot transmit or display entire dataset

4. Solutions • Network limitations: adaptive client-server streaming • Progressive loading guided by rendering needs • Available subset of dataset is rendered while loading • Graphics hardware limitations: adaptive rendering • 3D mesh simplification (data selection) Server Client Network Server database File server Streaming controller Partial database

5. Plan • Introduction • Related Work • Contributions • Data Structure • Adaptive streaming and rendering • 3D terrain rendering features • Planetary terrains • Preprocessing • Conclusion

6. Related work Google Earth • Commercial software • Google Earth, NASA World Wind… • Flight simulators • Continuous levels of detail (LOD)[Lindstrom et al. 1996, Duchaineau et al. 1997, Hoppe 1998] • Focus on reducing mesh complexity at all costs • Every data sample is considered at each frame • Designed for slow 3D rendering hardware • Alternative approaches • Clipmap[Losasso et al. 2004] • Projective grid [Dachsbacher et al. 2004] NASA World Wind

7. Related work • Blocks with discrete levels of detail[De Boer 2000, Pouderoux et al. 2005, Schneider et al. 2006] • Map subdivided into fixed-size blocks • Per-block selection between predefined LODs • Blocks can be loaded independently • All blocks needed to render entire map • Hierarchies of blocks[Levenberg 2002, Cignoni et al. 2003, Livny et al. 2007] • All blocks have the same resolution • Blocks organized in a tree • Changing LOD requires changing tree level • Tree can be loaded progressively Split

8. Discussion • Related work • Few solutions support both streaming and rendering efficiently • Some are based on outdated considerations • Few support planetary terrains and solve corresponding issues • Our approach • Unify adaptive data streaming and selection Same data structure and adaptive schemes, suited for both • But separate from 3D rendering Generic streaming and selection for large 2D maps, then application to 3D terrains • Adapt to any database size and hardware capabilities Favor speed, avoid data redundancy, support planetary terrains • Offer a complete and efficient solution

9. Overview of our work Network Preprocessing File server Adaptive streaming and selection 3D conversion and rendering • Generic adaptive solution • Streaming and rendering of sample maps • Application to 3D terrain rendering • Support of planetary terrains • Offline preprocessing steps with support for huge maps Server database Original database Partial client database

10. Plan • Introduction • Related Work • Contributions • Data Structure • Adaptive streaming and rendering • 3D terrain rendering features • Planetary terrains • Preprocessing • Conclusion

11. Data structure: base • Hierarchy of square blocks [Levenberg et al. 2002] • Can be progressively loaded as a tree, starting with the root • Hierarchical block selection  minimize amount of rendered blocks • Blocks have sets of regular levels of detail (LOD) [De Boer 2000] • Adaptive LOD selection  minimize amount of structure operations

12. Data structure: new properties • No data redundancy:less to store and transmit • LODs of a block share data (common sample grid) • Parent and children share one LOD (local copy when split/merge) • New LOD: samples interleaved between existing ones • Possible to render a block with not all LODs loaded • Possible to render a block and load one of its LODs in parallel

13. Plan • Introduction • Related Work • Contributions • Data Structure • Adaptive streaming and rendering • 3D terrain rendering features • Planetary terrains • Preprocessing • Conclusion

14. Measure of importance • The base for adaptivity in both streaming and rendering • Importance represents desired quality for a block • Thresholds to select LODs and trigger update requests • Defines priority for triggered requests • Any factors may be used • Based on application, nature of the data • Typical factors for 3D terrain rendering: • Distance from viewpoint • Surface area • Roughness

15. Adaptive streaming 0.1 s (at 1Mbps) • Pre-computed server database for minimal activity • Only one file read per LOD request • File position for any LOD known from previous LOD • Data are loaded “as-is” • Conversion on the client, in parallel with rendering • Possible quantization, pre-compression • Same server and data with any client type • We always transmit the most important data • Implicit adaptation to the network speed • Rendering quality constantly improves 10 s 40 s

16. Adaptive streaming Network Server Client Request • Restricted number of pending requests • Potential LOD loading requests come with an importance value • Requests with highest importance are transmitted to server • Others are discarded, need to update importance for next selection • At reception, database is updated and new request is transmitted Importance File server Adaptive streaming Adaptive rendering Reply (data) New LOD Complete database Partial database Rendering system

17. Client database • Incomplete tree of blocks • Data on all leaves  entire surface can be rendered • Asynchronous update operations • Tree update requests selected the same way as loading requests Adaptive streaming Tree update Adaptive rendering Requests Requests Adaptive selection Server Refine Split & merge Rendering Partial database

18. Adaptive rendering • At each frame, we: • Hierarchically cull blocks that are not visible • Compute importance for each block to select LOD to be rendered • Trigger requests when unavailable LODs are selected • Select most important requests from triggered ones • Render visible leaf blocks at selected LOD using sample masks

19. LOD1 used Split Merge Children used LOD0 requested, LOD1 used LOD0 used Adaptive selection 50 frames/s • User selects desired rendering speed • Adaptive quality factor in importance formula • Guides selection of LODs and triggering of operations 100 frames/s 150 frames/s LOD1 LOD0 LOD0 LOD-1 Importance

20. Results • Tested with 3D terrain rendering application • CPU usage: about 5-10% of each frame time • No stutters due to database updates (asynchronous) • Rendering quality improves as fast as network allows • Stable frame rate as long as enough data are loaded • Scales well from high to low performance clients (set-top box) Puget Sound database (Washington state, USA)

21. Plan • Introduction • Related Work • Contributions • Data Structure • Adaptive streaming and rendering • 3D terrain rendering features • Planetary terrains • Preprocessing • Conclusion

22. Hardware 3D rendering • We render 3D geometry with triangles • Conversion from elevation in refine operation • Adapted measure of importance • 3D culling with bounding boxes • Geometry caching in graphics hardware • Well suited for discrete LODs • Saves memory transfers (up to +40% rendering speed) • Sample masks as triangle strips • Applied directly in hardware • Need to solve cracks on block edges

23. Fixing geometry cracks • Additional triangle strip masks on block edges • Block with higher definition adapts • No new samples required • Neighbor knowledge: one per edge • No adaptation needed when more than one • There are unsolvable cases • Split and merge constraints

24. Texture maps • Color maps rendered using textures • 1 LOD = 1 mipmap • Hardware caching and selection • Distinct but linked geometry and texture trees • Specific measures of importance • Single texture coordinates mask for all geometry blocks • Only one texture per geometry block: split and merge constraints • Data filtering for down-sampling • Improved quality for low definition LODs • Progressive Texture Map [Marvie03]

25. Plan • Introduction • Related Work • Contributions • Data Structure • Adaptive streaming and rendering • 3D terrain rendering features • Planetary terrains • Preprocessing • Conclusion

26. Modeling a planet Google Earth (global, cylindrical) • Datum to support ellipsoid reference shape • Sphere projected onto a bounding cube • Produces square maps • Saves most redundant data compared toclassical solution (25% global) • Avoids visual “stretching” on poles Our solution (cube, gnomonic)

27. Projection adjustment • Base: gnomonic 2D map projection • Fast reverse projection (normalization) • 75% less surface on corners than center • New adjusted sampling • Planet surface instead of plane of projection • Steps = independent angles of rotation • Two tangent values computed per sample • 33% less surface on corners than center Plane of projection:

28. Solving specific planet issues 0 • Crack fixing on edges of the cube • Faces specifically numbered and oriented • Implicit and transparent management • Runtime adaptive clipping planes • Good rendering precision, from any distance • Culls hidden parts of the planet (behind the horizon) • No additional comparison 1 2 3 4 5

29. Results • Tested on GeForce 9800 GTX+, all features turned on • 140 FPS when asking for 2 million triangles Earth database from NASA

30. Plan • Introduction • Related Work • Contributions • Data Structure • Adaptive streaming and rendering • 3D terrain rendering features • Planetary terrains • Preprocessing • Conclusion

31. Preprocessing • Huge input, huge output • Memory constraints for loading input files • Linear writing to output files preferred • First step: re-projection of a planetary map • Project specific points of output window to find input window • Recursive output map subdivision • Load input window when fits in memory and re-project samples 1 global input cylindric map 1 of 6 output gnomonic maps

32. Preprocessing • Second step: generation of a server database • Direct input window computation • Top-down subdivision into complete tree of blocks • Load input for any sub-tree when fits in memory • Bottom-up construction of block LODs and linear file writing Input map Tree of blocks

33. Results • Support for input and output maps of arbitrary size • Projection on a cube: -25% database size • LOD compression with Zlib: up to -75% database size 740 MB 2mn Puget Sound 1.25 GB 14.9 GB 5h 9h SRTM 174 GB 126 GB 6.8 GB 35mn 1h30 TrueMarble 41.7 GB 31 GB

34. Plan • Introduction • Related Work • Contributions • Data Structure • Adaptive streaming and rendering • 3D terrain rendering features • Planetary terrains • Preprocessing • Conclusion

35. Conclusion • Generic adaptive solution • Sample maps with any size and type • Adapts to network and rendering speeds • Single data structure, generic methods • No redundant data, fast handling • Application to 3D terrain rendering • Optimizations for 3D graphics hardware • Most rendering artifacts avoided • Uniformly sampled planet surface • Improved rendering precision • Offline generation of huge databases

36. Future work • Enhance current work • Validate the solution on handheld devices • Use GPU shaders for fast and flexible 3D conversion • Further improve precision with local coordinate systems • Find more specific factors for measure of importance • Add new features • Generation of procedural details • Fix for popping artifacts • Advanced LOD compression • Self-shadowing of the terrain • Alternate rendering system with projective grid

37. Publications • Adaptive Streaming and Rendering of Large Terrains: A Generic Solution. Raphaël Lerbour, Jean-Eudes Marvie, Pascal Gautron. In WSCG 2009. • Adaptive Real-time Rendering of Planetary Terrains. Raphaël Lerbour, Jean-Eudes Marvie, Pascal Gautron. Accepted, to be presented at WSCG 2010.