Matrix of Proposed Activities and Results

1. Testbed for Mobile Augmented Battlefield VisualizationSeptember, 2003William Ribarskyand Nickolas FaustGVU Center and GIS CenterGeorgia Institute of Technology

2. Matrix of Proposed Activities and Results

3. Mobile Situational Visualization GPS and orientation tracker New lightweight wearable system Results of real-time collection of GPS path at night (left); screen shot with annotated path in red (right). • An extension of situation awareness that exploits and integrates interactive visualization, mobile computing, wireless networking, and multiple sensors: • Mobile users with GPS, orientation sensing, cameras, wireless • User carries own 3D database • Servers that store and disseminate information from/to multiple clients (location, object/event, weather/NBC servers) • Location server to manage communications between users and areas of interest for both servers and users • Ability to see weather, chem/bio clouds, and positions of other users • Accurate overviews of terrain with accurately placed 3D buildings • Ability to mark, annotate, and share positions, directions, speed, and uncertainties of moving vehicles or people • Ability to access and playback histories of movement • Placement of multiresolution models from MURI team members into environment

4. Mobile Situational Visualization Weather/Atmospheric Server Spread of dynamic Sarin gas cloud with positions of first responders Accurate Shared Locations Annotation Server Annotated views with updated user location and orientation

5. Mobile Situational Visualization System Buttons Pen Tool Mobile Team Drawing Area Shared observations of vehicle location, direction, speed collaborators Collaboration Example

6. Everybody has a location in space and time in the Virtual World Geographic server lookup approach Users Location Servers Data Servers Location Server Traffic Server Annotation Server Weather Server User User GeoData Server User Collaborative Environment

7. Everybody has a location in space and time in the Virtual World Geographic server lookup approach Users Location Servers Data Servers Collaborative Environment Location Server Traffic Server Annotation Server Weather Server User User GeoData Server User

8. Mobile Situational Visualization Video

9. What is Novel and Compelling About Mobile Situational Visualization? • Mobile battlefield visualization was an original proposed (and accepted) task. That’s pretty compelling! • But, beyond that • Instantplacement of environmental activity information within the geospatial environment combined with fast sharing and use are novel and compelling. • -Fast, accurate, and specific annotation of activity information (both user-controlled and automated logging) • -Immediate updates of databases with this information • -Server structure for sharing this with collaborators or commanders in the area of interest • -Use in computations and simulations (some launched automatically)

10. Matrix of Proposed Activities and Results

11. Berkeley Hundreds of automatically modeled buildings USC Tens to Hundreds of semi-automatically modeled buildings Thousands to tens of thousands of buildings and trees Hundreds of semi-automatically modeled buildings Integrated, Comprehensive Modeling • To build comprehensive models, we need a range of modeling techniques. • We also should combine techniques for richer and more complete models. Geo-accuracy Low Mid High Georgia Tech Low Mid High Model Detail

12. Integrated, comprehensive models with combined techniques Integrated, Comprehensive Modeling • To build comprehensive models, we need a range of modeling techniques. • We also should combine techniques for richer and more complete models. Geo-accuracy Low Mid High Low Mid High Detail

13. New Results on Modeling Large Collections (individual 3D buildings have brown roofs) • Generic models extruded from accurate footprints with accurate locations. (11,000 automatically generated from insurance GIS databases). • -Complete models with roofs • -Generic façade textures • -Databases available for automatically generating whole city (hundreds of thousands) • Automatic generation of accurately located tree models (thousands) from high-res imagery. • Creation of hundreds of specific buildings using commercial or self-developed (semi-automatic) software. 3D CAD modeled objects on high resolution terrain

14. Automatic Identification and Placementof Trees, Shrubs, and Foliage This can be used with Ulrich Neuman’s or Avideh Zakhor’s results to automatically identify, remove, and model foliage.

15. Accurate placement of 3D modeled trees Application to Tree Modeling Automated identification and modeling of trees

16. Matrix of Proposed Activities and Results

17. Organizing Large Collections of 3D Models for Interactive Display Q Q Q Q Q Q Q Q Q Q • Merging of different types and formats • Automated replacement of structures for overlapping areas Common format and organization for different types Q Q Q Q Q Linked Global Quadtrees Q Q Q Q

18. Paging, Culling, and Fast Rendering Linked global quadtree Block Block Block Block Block Q Q Q Q Out-of core Storage Block Quadcell

19. Integrated, Interactive Visualization of Large Collections of Models Video

20. Matrix of Proposed Activities and Results

21. Q Q Q Q Q Q Q Q Q Q Results of view-dependent simplification. The blue box is the viewing window; fully textured models with and without meshes displayed are shown on the left and right, respectively. (Top) Full resolution mesh and textures within the window. (Bottom) Significantly reduced resolution mesh and textures within the window without reduction in visual quality. Handling Complicated Models View-Dependent LOD for large collections of complicated models Q Q Q Q Q N Levels Linked Global Quadtrees Q Bounding box Viewpoint Selected LOD

22. Quadric Error Approach to Simplification • Initial development Garland and Heckbert, SIGGRAPH, 1997 • Quadric approach yields “optimal” simplification by permitting generalized contractions between vertices and keeping track of the deviation from the original mesh Use quadric matrix to find a vertex with error within ε; Δ is the surface at error value ε. v1 v2 contraction Non-topological simplification v1 v2 general contraction

23. Limitations on Basic Quadric Approach [ ] • No concept of view-dependence and continuous LOD • No structure for large collections of objects • Geometry error metric; no appearance-preserving metric (e.g., for textures, shading, lighting). A combined metric is best. Application of appearance-preserving metric to a textured object (Cohen et al., SIGGRAPH 98) w/o appearance metric full resolution with appearance metric

24. Linked global quadtree The vertex front is circled. Green nodes are active-interior, blue nodes are active-boundary, and orange nodes are inactive. Here, vertex V7 is split and vertices V10 and V11 are merged. … … … … The pink, purple, and dark gray triangles are subfaces of V7, V5, and V4, respectively. (a) Full mesh. (b) Tree on left. (c) Tree on right. Q Q Q Q … … … … … … … … Block LOD Hierarchy Façade 1 Façade N … Object 1 Object M … View-Dependent Continuous LOD Tree

25. View-Dependent Appearance-Preserving Simplification original surface M0 (E) (C) (A) current surface Mi-1 possible surface Mi deviation vectors PA PB PC PC Va Vb Va Vb Va Vb PO Vc Vc Vc Collapse Distance Deviation Quadric Error Deviation Two-Way Incremental Texture Deviation (F) (D) (B) PC PO Va Vb Va Vb Va Vb Vc Vc Vc One-Way Incremental Texture Deviation Total Texture Deviation Two-Way Incremental Distance Deviation

26. View-Dependent Appearance-Preserving Simplification Video

27. Matrix of Proposed Activities and Results

28. Implementing and Using the Testbed • Merging of tens of thousands (and more) of models from multiple sources. • Efficient organization and culling of massive collections of 3D objects. • Integration of view-dependent methods for accurate and efficient display of complex models. • Deployment and use of mobile situational visualization capability.

29. Technology Transfer • The VGIS visualization system with capabilities developed here (including mobile visualization) was a key part of the Georgia Tech Homeland Defense Workshop and will be part of the GT Homeland Defense Initiative with support at the State and National levels. • The system is being used as part of the Sarnoff Raptor system, which is deployed to the Army and other military entities. In addition our visualization system is being used as part of the Raptor system at Scott Air Force Base. • We are in discussion with the Department of the Interior on use of our mobile situational visualization capability to develop Anytime-Anywhere information system resource accessibility for countering asymmetric threats.

30. Plans for Next Year • Full deployment of mobile situational visualization capability with sharing of the system and the results with team members. • Further development of automated model building from multisource data. This will be a collaborative effort with other team members. We will move towards a robust system with ability to merge and increment model sets and update models (adding improvements to make generic models more detailed and specific as data are available). • Development of fully scalable 3D object organization and interactive visualization capability extending to hundreds of thousands of accurately located buildings and trees (or more). • Full integration of view-dependent capability for complex models.