450 likes | 784 Views
From Mobile Augmented Reality to Ubiquitous Augmented Reality. Prof. Dr. Dieter Schmalstieg Institut für Maschinelles Sehen und Darstellen Technische Universität Graz. What is Augmented Reality?. Definition of Augmented Reality. 1. Blends real & virtual, in real environment +
E N D
From Mobile Augmented Reality to Ubiquitous Augmented Reality Prof. Dr. Dieter Schmalstieg Institut für Maschinelles Sehen und Darstellen Technische Universität Graz
Definition of Augmented Reality 1. Blends real & virtual, in real environment + 2. Real-time interactive + 3. Registered in 3D [Azuma PRESENCE 1997]
The Promise of Mobile AR • Bring computer support to task location • Examples • Navigation • Tourism • Repair & maintainence • In-situ inspection • As-built documentation • Situation awareness • …
Milgram Continuum Mixed Reality Reality Augmented Reality (AR) Augmented Virtuality (AV) Virtual Reality
Milgram-Weiser Continuum Milgram This is what we are interested in! [Newman VR 2007] Weiser
How to get to Ubiquitous AR • Address scalability problems • Only partially considered in typical research • (Will discuss technical, not commercial aspects)
Content Storage AR System Visualization ABC ABC AR Information Processing Pipeline * Creation by hand * Conversion of legacy data * Creation by inference * Federation * Metadata * HW and SW * Ergonomic * Inexpensive * Robust * Filtering * Context aware [Schmalstieg CG&A 2007]
Topics for Today • Handheld Augmented Reality • Ergonomic • inexpensive system platform • AR Museum Scavenger Hunt • Content creation and delivery • Visualization of subsurface infrastructure • Conversion of legacy data, metadata • Context-aware visualization
Ergonomic Considerations How long until wearable AR becomes ergonomically andsocially acceptable?
Augmented Reality on PDAs • Stand-alone handheld AR on Smartphones • No thin client • High scalability • PDAs are • Inexpensive • Socially acceptable • Familiar • Widespread • All in one solution • Display • Touch screen • Camera
The Invisible Train [Wagner SIGGRAPH EMT 2004]
Real-Time Computer Vision on Phones Just artificial fiducial tracking, but it must work… Adaptive tresholding/vignetting …all with single fixed point unit at ~200MHz BCH or DataMatrix markers [Wagner ISCW 2003]
Tracking Performance Performance scales linearly with clock About 10-20 times slower than PC
Overall Performance 12 triangles 2625 triangles 27219 triangles
Content Creation and Delivery „Expedition Schatzsuche“ • Multiuser Game in Landesmuseum Kärnten • Make museum visit more attractive to kids 8-12y • Runs on Gizmondo (Windows CE game console) [Schmalstieg ISMAR 2007]
XML database Software architecture • Studierstube ES: software stack for client • Sphinx: Game server, XML database + finite state machine • Game state on server (prefered) or client • Interaction on client in 3D (AR) or 2D • Bootstrapping and content streaming
Game Elements • Diamonds/question marks tell state of a station • green: free, yellow: in use, red: already solved • Solve task by taking a picture of exhibit or solve a problem • Map of the museum showing station availability
Game „Brauchtum“ • Retrieve flat iron for finding out initials of wedding couple • Puts the iron into the tailors’ oven • Learn about the wedding rider and finish the sequence
Game „Musical Tradition“ • Playing the silent piano • Listening to the Mandora; • Applying the “Stimmbogen” to horn • Keeping the organ playing by pumping up the bellows
Content Creation • Import from standard tools when possible • Maya, Flash… • XMI for visual programming of state charts • Online tweaking of the database • Lesson: content creation consumed several person month, for a small game!
Do dig or not to dig? • Use AR „X-Ray-Vision“ HGas K531-L 380V STEG 941 250V STEG 902 Application Scenario • Utility companies maintain a GIS of their subsurface infrastructure • Grintec GesmH (Graz), GE SmallworldTM
How Superman does it Lois Lane is going to be alright!
AR Information Filtering • Assume large DB incl. metadata • Avoid display clutter? • Display only relevant information • Important for user • Within field of view • Procedurally determined, per object • Better: (Meta-)data driven,per pixel [Höllerer ISAR 2001]
Scene Graph with Context Markup • Problem • Context (metadata) scattered throughout scene graph • Finding information requires full search • Manipulation may require changing SG in many places • Acerbated by mix-and-match scene graph composition, even at runtime! • Need to delegate behavior modification to local nodes and their interaction with near nodes Where is a “blue” nodehidden in the SG?
Subsurface Data Example Danger: Welding in the town hallmust not happen near gas
Approach • Accumulate context (= user defined state) during traversal • Aggregate markup „set“ • Generic parameter passing technique • Parameters can be sub-scenegraphs • Scene graph becomes a template structure • Binding in the last possible moment [Reitmayr VR 2005]
Example: 3D Model Stylesheets world in miniature Registered overlay generic model
Example: 3D Model Stylesheets Registered overlay World in miniature model wireframe model Z doorstyle model Wall style generic model R2 R1 wall door
Dynamic Binding of Styles Danger (red) defined by rules (telemetry, user) [Mendez ISMAR 2007]
Magic Lenses for Filtering • Spatial + semantic context [Schall Ubicomp 2007]
Visual Focus+Context [Kalkofen ISMAR 2007] Dense scene Depth perception in AR overlay?
Focus+Context Approach G-Buffer Technique similar to NPR Markup „families“ define G-Buffers Single pass / multiple target rendering Compositing for dynamic focus+context Markup defines families multiple GBuffers GBuffer for one family
Video Processing Incorporate video image as GBuffer
Compositing Explicit depth sorting Raycasting-like compositing Scriptable compositing rules „first-hit+context“ strategy
Legacy database conversion • Procedural modeling to convert GIS to AR • Uses Generative Modeling Language[Havemann 2005] • Geometry creation on-the-fly • Markup provides control parameters for procedural modeling Subdivisionsurfaces Progressive revealing LOD
Putting it all together Submitted to CG&A, 2007
Ongoing and Future Work • Organize meta-information into ontologies • Topological relationships: containment, adjacency, … • Pedestrian navigation • Indoor infrastructure (e.g. power, network) • Use semantic reasoning to infer new data • Also infer visualization rules • Natural feature tracking for phones
Thank you for your attention! • www.studierstube.org Collaborators Istvan Barakonyi Sven Havemann Denis Kalkofen Joe Newman Thomas Pintaric Gerhard Reitmayr Gerhard Schall Daniel Wagner