Augmented Reality

Augmented Reality MARE : Multi-user Augmented Reality Environment on Real Table Setup Raphael Grasset Jean-Dominique GascueliMAGIS, France iMAGIS is a joint project between CNRS,INRIA,INPG and UJF

Motivations of our Research • Purpose : real collaborative situation (discussion, meeting, project review..). NEED DIRECT 3D Visualization, Interaction.. [Buscher00] • Applications : scientific visualization, architecture, urban planning, game.. • No satisfactory solutions(Desktop Computer, CAVE, Responsive Workbench)

Our Multi-user AREnvironment A collaborative environment for multi-user in augmented reality. [Schmalstieg00], [Fjeld02] Setup : Centralized configuration • Around the table. Not modified environment. • Augmented user. • Private/Public area. shared area Personal area

Goals • Architecture : Hardware/Software. • Calibration / Registration. • Intuitive 3D interaction method. • Visual integration (occlusion, shadow). • Multi-user management (protection, monitoring). • Moto (simplicity, rapidity) / (viability, robustness)

Architecture : Hardware Heterogeneous devices (displays, 3D/2D, two-hand)

Architecture : Hardware • Personal display • Tracking • Interaction devices (Onyx 3, 2 Pipes, SGI)

Architecture : Software • Modular / Multilayer system • XML configuration approach (device, interaction, object). Application Usermanager Objectmanager Devicemanager Interactionmanager OpenGL Performer, AL, VL, Qt X

Architecture : Software Support of various object (3D, 2D, text, picture, music, video, speech synthesis) Text Screen-Stabilized 3D Object Sound Object Animated Objects Picture Text

Calibration of devices • User-based approach : “plug’n calib and play”. Intuitive and simple method. • Optotrak / FOB / Wacom calibration : select points of reference. Fob emitter Pen Calibration : orient pen in specific direction. [Tuceryan95]

Head calibration Fixed view and register specific object. video see-through optical see-through • Maximum error : 1 cm  sufficient for non-critical application • less jitter that magnetic tracker

Goals • Architecture : Hardware/Software. • Calibration / Registration. • Intuitive 3D interaction methods. • Visual integration (occlusion, shadow). • Multi-user management (protection, monitoring). • Moto (simplicity, rapidity) / (viability, robustness)

Interaction moto • Least intrusive device : real pen and hand interaction + TUI. Low haptic feedback : table contact, respect of gravity. virtual tools + personal menu

Tracked pencil real fonction and 6D0F manipulation

Hand tracked device easy wearable/removable gesture and interaction

Tangible User Interface real reality approach. natural object : cup, paper, box. specialized object : marker

Moving Objects 1/2 • Inspired from real methods lift-move method slide method

Moving Objects 2/2 • 3D Tangible UI : select with a real “ghost”

Adding Objects • Add Objects : Virtual/Real • Virtual Objects : menu, catalogue, gesture. • Real Objects : • predefined model : register virtual and real representation. • generic model : fitting with a simple shape

Multiuser management [Gutwin00] Communication/Coordination : like real meeting. Awareness : enhance device, line of sight, visual and sound feedthrough. Personal view : unix access approach + personal scene graph.

Multiuser management • Protection : social + technical protocol • Assistance : Multimedia help + tool tip • Monitoring : Log + Replay

Pre/Post session : Authoring Tools Preparation • hardware setup • scene setup • assistance setup • scenario setup Debriefing • replay • save selected information Meeting XML files Dedicated Graphical User Interfaces

Contributions • Collaborative AR framework • Easy setup, adaptive to object domain. • New interactive techniques • Virtual Reality and CHI approach. Future Works • Gesture Recognition. • TUI : Vision tracking. • Visual Integration : shadow, lighting • Multiuser : coupling, planning. • CHI evaluation

Questions ?

Calibration : Results • Maximum error : 1 cm  sufficient for non-critical application optical see-through video see-through • less jitter that magnetic tracker

How to see that what users see ?

Navigation • Goal : Visualize the scene from different viewpoint. RA (face to face) RV(immersion) ([Kiyokawa99] • Approach : “dim in the scene” (WIM [Stoakley95]). t1 t1 t2 t2 Tree character Working space

Utilisateur Visibilité Déplacement Annotation Défaut Architecte Maurice Clients Joe Robert Personal View : details

Interaction time-multiplexed vs space-multiplexed Personal menus : tools, add objects. Radio Button Push Button Menu slider

S2002 Bookman Old Style, Bold, 37 points This subtitle is 31 points • Bullets are orange; text is 26 points • They have 110% line spacing, 6 points before/after • Longer bullets in the form of a paragraph are harder to read if there is insufficient line spacing. This is the maximum recommended number of lines per slide (seven). • Sub-bullets look like this.

Virtual Reality (VR) Real (R) Simulation, Animationcomplex computationpersistent information Natural metaphor of communicationintuitive 3D interaction Motivations Augmented Reality (RA) Integrate R+V

Calibration : Results • Maximum error : 1 cm  sufficient for non-critical application • less jitter that magnetic tracker

Augmented Reality