Augmented Reality Collaborative Environment: Calibration and Interactive Scene Editing

1. Augmented Reality Collaborative Environment: Calibration and Interactive Scene Editing Raphaël GrassetXavier DecoretJean-Dominique Gascuel iMAGIS-GRAVIR/IMAG iMAGIS is a joint project between CNRS,INRIA,INPG and UJF

2. Motivations Focus on: • tracking and registration of virtual and real objects. • commercial age: off the shelf and personal hardware. Purpose: propose a collaborative environment for multi-user in augmented reality.

3. Related Works • Some Augmented Reality Environments • GRASP[1995] • Studierstube[1998] • EMMIE[1999]

4. Overview • Motivation and Related Works • Setup: Small Group and Computing Environment • Calibration Procedure • HMD Calibration • Pen Calibration • World Calibration • Dynamic Objects • Mixing Real and Virtual objects • Moving Objects • Conclusion

5. Setup: Small Group Centralized Configuration: • user around a table: face to face collaboration. • personal and shared area. Shared area Personal area(virtual menu, private data)

6. Setup: Computing Environment • Visualization: optical see-through HMD direct view, private information, low weight. • Interaction: stylus • easy use. • Tracking: magnetic tracker • fast, no line of sight.

7. Overview • Motivation and Related Works • Setup: Small Group and Computing Environment • Calibration Procedure • HMD Calibration • Pen Calibration • World Calibration • Dynamic Objects • Mixing Real and Virtual objects • Moving Objects • Conclusion

8. Calibration Procedure Purpose:obtain the relations between different coordinate systems (CS). HMD CS Eye CS Transmitter CS Receptor CS Image CS HMD CS = middle between the 2 optic centers of the HMD. Emitter CS Tip CS World CS Style: fast and accurate interactive procedure.

9. Calibration Procedure Purpose:obtain the relations between different coordinate systems (CS). HMD Calibration HMD CS Eye CS Transmitter CS Receptor CS Image CS HMD CS = middle between the 2 optic centers of the HMD. Emitter CS Tip CS World CS Style: fast and accurate interactive procedure.

10. Calibration Procedure Purpose:obtain the relations between different coordinate systems (CS). HMD CS Pen Calibration Eye CS Transmitter CS Receptor CS Image CS HMD CS = middle between the 2 optic centers of the HMD. Emitter CS Tip CS World CS Style: fast and accurate interactive procedure.

11. Calibration Procedure Purpose:obtain the relations between different coordinate systems (CS). HMD CS Eye CS Transmitter CS Receptor CS Image CS HMD CS = middle between the 2 optic centers of the HMD. Emitter CS Tip CS World CS World Calibration Style: fast and accurate interactive procedure.

12. Related Works • Some Calibration Methods: • Azuma[1994] & Oishi[1996]: lot of hardware. • Tuceryan[1995]:video see-through method

13. Related Works • Fuhrmann[1999] (Studierstube): calibration with a tracked pen. • Genc[2000] (GRASP): vision approach.

14. HMD Calibration Purpose: transformation between the Receptor CS and the Image CS.Style: reduce user effort. Constraints: personnal hardware => removable tracker on the HMD. Depends on the position of the tracker in the HMD (HMD CS Receptor CS), Morphology ..etc..

15. HMD Calibration: Procedure Method: freeze the view and align a virtual object with real object. Emitter CS 1.Freeze the view Misalignment Virtual Emitter HMD CS Eye CS Receptor CS

16. HMD Calibration: Procedure Method: freeze the view and align a virtual object with real object. 2. Move head Emitter CS Virtual Emitter HMD CS Freeze View Eye CS Receptor CS

17. HMD Calibration: Procedure Method: freeze the view and align a virtual object with real object. Emitter CS 3. Unfreeze the view Virtual Emitter HMD CS Freeze View Eye CS Receptor CS

18. HMD Calibration: Procedure Method: freeze the view and align a virtual object with real object. Emitter CS Virtual Emitter HMD CS Eye CS Receptor CS 4. Compute transformation Incremental realignment.

19. Pen Calibration: Principles and Procedures Purpose: transformation between the Transmitter CS and the Tip CS. Method:orient the stylus in n representative directions. Fixed tip Emitter CS Compute by SVD (nx3 equations, 6 unknowns).

20. Pen Calibration: Results Tracker positions Reconstruction of the tip SVD on 100x3 equations SVD on 4x3 equations We propose to keep just 4 measurements.

21. World Calibration Purpose: transformation between the Emitter CS and the World CS. Method: selection of 4 corners of shared area. P4 P1 World CS P2 Emitter CS P3

22. Calibration Results Different views of the source square with a virtual figure added: no disturbing drift. 2 cm Error largely acceptable for most practical applications.

23. Overview • Motivation and Related Works • Setup: Small Group and Computing Environment • Calibration Procedure • HMD Calibration • Pen Calibration • World Calibration • Dynamic Objects • Mixing Real and Virtual objects • Moving Objects • Conclusion

24. Dynamics objects: Mixing Real and Virtual Environment Purpose: • Describe real objects to the system. • Tangible User Interface. • Poor man haptic feedback. Requirements: • Occlusion compatibility (« black render »). • Respect of laws of physics (e.g gravity). • Manipulation awareness. Our approach: interactive system easy use based on visual feedback. • Digitalized objects. • Generic Geometric models. • TUI objects.

25. Related Works • Some interactions with real objects • Whitaker[1995]: image-based and pointer-based object calibration. • Mixed Reality[1999]: features on objects. • Build-IT[1997]

26. Mixing with Already Digitalized Models (1/2) Purpose: obtain transformation between the object CS and the world CS (known geometry + appearance). Constraints: laying on top of something else (3DOF).Method: superpose virtual representation with real representation of the model. Object CS Real object Virtual object

27. Mixing with New Simple Models Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..) Our approach: Description of simple shapes with few points. Example: add a rectangle. 8 DOF

28. Mixing with New Simple Models Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..) Our approach: Description of simple shape with few points. Example: add a rectangle. 5 DOF P1

29. Mixing with New Simple Models Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..) Our approach: Description of simple shape with few points. Example: add a rectangle. 5 DOF P1

30. Mixing with New Simple Models Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..) Our approach: Description of simple shape with few points. Example: add a rectangle. 2 DOF P1 P2

31. Mixing with New Simple Models Purpose: obtain the geometry and the position of generic objects (cube, cylinder, cone, ..etc..) Our approach: Description of simple shape with few points. Example: add a rectangle. 0 DOF P1 P2

32. Mixing with New Simple Models: Results

33. Tangible User Interface:Moving Objects Purpose: use real object for manipulation of virtual models.Constraints: one tracker + several real objects to manipulate. Our approach: TEMPORARILY stickthe magnetic tracker to previously added objects. Real object

34. Video

35. Conclusion Our contributions • Multi-user environment setup. • Fast user calibration method with sufficient accuracy for a large range of non-critical applications. • Dynamic placement and digitalization of objects. • Intuitive metaphor to manipulate objects. Future works • Develop interaction techniques. • Reduce latency.

37. Mixing with already Digitalized Models (2/2) Real object