1 / 42

From Mobile Augmented Reality to Ubiquitous Augmented Reality

From Mobile Augmented Reality to Ubiquitous Augmented Reality. Prof. Dr. Dieter Schmalstieg Institut für Maschinelles Sehen und Darstellen Technische Universität Graz. What is Augmented Reality?. Definition of Augmented Reality. 1. Blends real & virtual, in real environment +

ora-hudson
Download Presentation

From Mobile Augmented Reality to Ubiquitous Augmented Reality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. From Mobile Augmented Reality to Ubiquitous Augmented Reality Prof. Dr. Dieter Schmalstieg Institut für Maschinelles Sehen und Darstellen Technische Universität Graz

  2. What is Augmented Reality?

  3. Definition of Augmented Reality 1. Blends real & virtual, in real environment + 2. Real-time interactive + 3. Registered in 3D [Azuma PRESENCE 1997]

  4. The Promise of Mobile AR • Bring computer support to task location • Examples • Navigation • Tourism • Repair & maintainence • In-situ inspection • As-built documentation • Situation awareness • …

  5. Milgram Continuum Mixed Reality Reality Augmented Reality (AR) Augmented Virtuality (AV) Virtual Reality

  6. Milgram-Weiser Continuum Milgram This is what we are interested in! [Newman VR 2007] Weiser

  7. How to get to Ubiquitous AR • Address scalability problems • Only partially considered in typical research • (Will discuss technical, not commercial aspects)

  8. Content Storage AR System Visualization ABC ABC AR Information Processing Pipeline * Creation by hand * Conversion of legacy data * Creation by inference * Federation * Metadata * HW and SW * Ergonomic * Inexpensive * Robust * Filtering * Context aware [Schmalstieg CG&A 2007]

  9. Topics for Today • Handheld Augmented Reality • Ergonomic • inexpensive system platform • AR Museum Scavenger Hunt • Content creation and delivery • Visualization of subsurface infrastructure • Conversion of legacy data, metadata • Context-aware visualization

  10. Ergonomic Considerations How long until wearable AR becomes ergonomically andsocially acceptable?

  11. Augmented Reality on PDAs • Stand-alone handheld AR on Smartphones • No thin client • High scalability • PDAs are • Inexpensive • Socially acceptable • Familiar • Widespread • All in one solution • Display • Touch screen • Camera

  12. The Invisible Train [Wagner SIGGRAPH EMT 2004]

  13. Real-Time Computer Vision on Phones Just artificial fiducial tracking, but it must work… Adaptive tresholding/vignetting …all with single fixed point unit at ~200MHz BCH or DataMatrix markers [Wagner ISCW 2003]

  14. Tracking Performance Performance scales linearly with clock About 10-20 times slower than PC

  15. Overall Performance 12 triangles 2625 triangles 27219 triangles

  16. Content Creation and Delivery „Expedition Schatzsuche“ • Multiuser Game in Landesmuseum Kärnten • Make museum visit more attractive to kids 8-12y • Runs on Gizmondo (Windows CE game console) [Schmalstieg ISMAR 2007]

  17. XML database Software architecture • Studierstube ES: software stack for client • Sphinx: Game server, XML database + finite state machine • Game state on server (prefered) or client • Interaction on client in 3D (AR) or 2D • Bootstrapping and content streaming

  18. Game Elements • Diamonds/question marks tell state of a station • green: free, yellow: in use, red: already solved • Solve task by taking a picture of exhibit or solve a problem • Map of the museum showing station availability

  19. Game „Brauchtum“ • Retrieve flat iron for finding out initials of wedding couple • Puts the iron into the tailors’ oven • Learn about the wedding rider and finish the sequence

  20. State Machine „Brauchtum“

  21. Game „Musical Tradition“ • Playing the silent piano • Listening to the Mandora; • Applying the “Stimmbogen” to horn • Keeping the organ playing by pumping up the bellows

  22. Content Creation • Import from standard tools when possible • Maya, Flash… • XMI for visual programming of state charts • Online tweaking of the database • Lesson: content creation consumed several person month, for a small game!

  23. Context-Driven Visualization for Augmented Reality

  24. Do dig or not to dig? • Use AR „X-Ray-Vision“ HGas K531-L 380V STEG 941 250V STEG 902 Application Scenario • Utility companies maintain a GIS of their subsurface infrastructure • Grintec GesmH (Graz), GE SmallworldTM

  25. How Superman does it Lois Lane is going to be alright!

  26. AR Information Filtering • Assume large DB incl. metadata • Avoid display clutter? • Display only relevant information • Important for user • Within field of view • Procedurally determined, per object • Better: (Meta-)data driven,per pixel [Höllerer ISAR 2001]

  27. Scene Graph with Context Markup • Problem • Context (metadata) scattered throughout scene graph • Finding information requires full search • Manipulation may require changing SG in many places • Acerbated by mix-and-match scene graph composition, even at runtime! • Need to delegate behavior modification to local nodes and their interaction with near nodes Where is a “blue” nodehidden in the SG?

  28. Subsurface Data Example Danger: Welding in the town hallmust not happen near gas

  29. Approach • Accumulate context (= user defined state) during traversal • Aggregate markup „set“ • Generic parameter passing technique • Parameters can be sub-scenegraphs • Scene graph becomes a template structure • Binding in the last possible moment [Reitmayr VR 2005]

  30. Example: 3D Model Stylesheets world in miniature Registered overlay generic model

  31. Example: 3D Model Stylesheets Registered overlay World in miniature model wireframe model Z doorstyle model Wall style generic model R2 R1 wall door

  32. Dynamic Binding of Styles Danger (red) defined by rules (telemetry, user) [Mendez ISMAR 2007]

  33. Magic Lenses for Filtering • Spatial + semantic context [Schall Ubicomp 2007]

  34. Visual Focus+Context [Kalkofen ISMAR 2007] Dense scene Depth perception in AR overlay?

  35. Focus+Context Approach G-Buffer Technique similar to NPR Markup „families“ define G-Buffers Single pass / multiple target rendering Compositing for dynamic focus+context Markup defines families  multiple GBuffers GBuffer for one family

  36. Video Processing Incorporate video image as GBuffer

  37. Compositing Explicit depth sorting Raycasting-like compositing Scriptable compositing rules „first-hit+context“ strategy

  38. Legacy database conversion • Procedural modeling to convert GIS to AR • Uses Generative Modeling Language[Havemann 2005] • Geometry creation on-the-fly • Markup provides control parameters for procedural modeling Subdivisionsurfaces Progressive revealing LOD

  39. Putting it all together Submitted to CG&A, 2007

  40. Ongoing and Future Work • Organize meta-information into ontologies • Topological relationships: containment, adjacency, … • Pedestrian navigation • Indoor infrastructure (e.g. power, network) • Use semantic reasoning to infer new data • Also infer visualization rules • Natural feature tracking for phones

  41. Thank you for your attention! • www.studierstube.org Collaborators Istvan Barakonyi Sven Havemann Denis Kalkofen Joe Newman Thomas Pintaric Gerhard Reitmayr Gerhard Schall Daniel Wagner

More Related