Multi-Layered Impostors for Accelerated Rendering

1. Multi-Layered ImpostorsforAccelerated Rendering Xavier Decoret, iMAGIS This is joint work withGernot Schaufler and Julie Dorsey at MIT and François Sillion at iMAGIS

2. Complex Environments Paris: • 411537 vertices • 137179 triangles • 32 textures(most 256x256) • 6.1 MB geometry Multi layered impostors for accelerated rendering

3. IBR in RT-Graphics • Image-based representations simplify the scene • Images are unaffected by the depicted scene’s complexity • Images are fast to render as textured triangles • Images themselves can be generated with hardware • This works if a geometric model is available Multi layered impostors for accelerated rendering

4. Previous Work • Pre-generated Representations Grossman98, Dally96, Maciel95, Aliaga96, Chen95, McMillan95, Xiong96, Sillion97, Darsa98, Pulli97, Laveau94, Max96,Rafferty98 • Dynamically Updated Representations Torborg96, SGI97, Lengyel97, Regan94,Shade96, Schaufler96, Mark97, Mann97Aliaga99 Multi layered impostors for accelerated rendering

5. Impostors (pre-generated) SGI Performer (Billboards) Maciel95 Id Software (3D Sprites) Multi layered impostors for accelerated rendering

6. Impostors (dynamically generated) Schaufler95 Schaufler Stürzlinger ‘96 Multi layered impostors for accelerated rendering

7. Meshed Impostors[Sillion97] Depth Map Distant Geometry Discontinuities Triangulation Multi layered impostors for accelerated rendering

8. Artefacts in IBR Geometry • Deformation caused by mesh • Resolution mismatch Multi layered impostors for accelerated rendering

9. Artefacts in IBR Geometry ? • Incomplete representation • Rubber sheet effects Multi layered impostors for accelerated rendering

10. Artefacts in IBR Geometry • Image cracks • and more … • Static Shading: • no highlights • no reflections • No moving Objects Multi layered impostors for accelerated rendering

11. Our Contributions Reduce some of the identified artefacts • Reducing Rubber Sheet Triangles • Multi Mesh Impostors • Reducing distortions to improve quality • Dynamic Update Multi layered impostors for accelerated rendering

12. The database • The geometry is organized into objects • We have a set of viewcells One edge Another edge The street graph Multi layered impostors for accelerated rendering

13. Local model Complete Geometry PVS Distant model Model segmentation Multi layered impostors for accelerated rendering

14. Distant model... ...replaced by impostor Combined with local model Single mesh impostor Multi layered impostors for accelerated rendering

15. Rubber Sheet Triangles due to parallax View from above impostor viewpoint Multi layered impostors for accelerated rendering

16. View from above Assume a 2D problem Locating Rubber Sheet Triangles • Parallax creates rubber sheets between objects when objects overlap in depth Front view • In urban walkthrough, parallax is mainly horizontal Multi layered impostors for accelerated rendering

17. O1 O1 Critical zone O2 O2 Edge Edge Multi Mesh Impostor • The critical zone identifies overlaps between 2 objects • When overlapping occurs in image space, one object can uncover the other one • If uncovering is too much, objects must not be on the same mesh Multi layered impostors for accelerated rendering

18. Multi Mesh Impostor • Quantifying overlapping between 2 objects • Place those objects in different layers which are too distant in depth: • construct a relation graph • partition the graph Single mesh Several meshes Multi layered impostors for accelerated rendering

19. y amin M1, M2  [PQ] such as amax A AM1B minimal B AM2B maximal x P Q Objects and Layers Object 1 • Two objects must go into different layers if two points on their geometry can be seen under sufficiently different viewing angles amin and amax. A Object 2 B Q edge P Multi layered impostors for accelerated rendering

20. Relation Graph • One node per object • Edges joining overlapping objects • Coloriate the graph so that joined vertices have different colors • Each color represent a layer • Non unique Multi layered impostors for accelerated rendering

21. All Geometry PVS Edge Single Mesh Multi Mesh Results & Example Multi layered impostors for accelerated rendering

22. End point Notice rubber sheet triangles Results & Examples Start point From which the impostor is computed Multi layered impostors for accelerated rendering

23. End point The view is no longer blocked Results & Examples Start point From which the impostor is computed Multi layered impostors for accelerated rendering

24. Geometry Single Mesh MultiMesh Results & Examples Multi layered impostors for accelerated rendering

25. Offline vs Online • Motion hides small artifacts of stored impostors. • If the user stops, the correct image should appear. • This requires knowledge of the current viewpoint. • Offline approaches do not have this information Multi layered impostors for accelerated rendering

26. Combining preprocessingwith dynamic updates • Single mesh impostor replace too much geometry to be updated. • Layers and their contents are suitable for regeneration of part of the distant model. • Layers are updated front to backto improve image quality: • silhouettes • distortions • resolution mismatches Multi layered impostors for accelerated rendering

27. View Cells Scene Visibility Extractor Offline Model Segmentation MMI Extractor Minimal Scenegraph per viewcell Online Dynamic Update Rendering System Architecture Preprocessing: • Take geometry and view cells • Find visible geometry for cells • Split into near and far part • Create impostors for far part • Store as scene per view cell Walkthrough: • Page in geometry and textures • Do dynamic updates if possible • LOD management Multi layered impostors for accelerated rendering

28. Results & Video (Paris) Storage requirements : 70 Mbs Computation time : 100 edges per hour Achieved frame rate: 50 Hz Multi layered impostors for accelerated rendering

29. Future Work • Smooth transitions between different representations • Automatic generation of street graph from street mesh • Extension of viewcells from edges to areas and volumes • Reverse approach: construction of viewcells to optimize use of impostors. Multi layered impostors for accelerated rendering

30. Acknowledgements This is a joint project between iMAGIS, Grenoble and MIT, Cambridge. It was supported in part by a joint collaborative research grant of NSF and INRIA (INT-9724005), an Alfred P. Sloan Foundation Research Fellowship (BR-3659), and by a grant from Intel Corporation. The following people contributed to the presented results : Julie Dorsey, François Sillion, Gernot Schaufler, Max Chen, Byong Oh Mok, Yann Argotti and Sami Shalabi. Multi layered impostors for accelerated rendering