1 / 15

Cooperative Multimodal Augmented Reality Labyrinth

Cooperative Multimodal Augmented Reality Labyrinth. Didier Perroud Raynald Seydoux Frédéric Barras. Summary. Abstract Objectives Modalities Project modalities CASE/CARE Implementation VICI, Iphone , Voice recognition, Network Demonstration Conclusion. Abstract.

sanne
Download Presentation

Cooperative Multimodal Augmented Reality Labyrinth

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cooperative Multimodal Augmented Reality Labyrinth Didier Perroud Raynald Seydoux Frédéric Barras

  2. Summary • Abstract • Objectives • Modalities • Project modalities • CASE/CARE • Implementation • VICI, Iphone, Voice recognition, Network • Demonstration • Conclusion

  3. Abstract • Coordination betweentwopersons to move a ballinto a labyrinth • Rotation possible on the x and y axis • Gates canbeopenedwith vocal and gesturalcommands

  4. Objectives • Coordinate the following technologies: • Augmented reality with tags • Gesturedetection ( withIphoneaccelerometers) • Voice recognition ( words) • Collaborative environments • Physicengine

  5. Modalities – Project Modalities • Inputs • Hand rotation in x and y axis ( one axis per player)  direct manipulation of the labyrinthboard • Hand pumping for gates’ openings • Voice recognition (words) for selectinggate to open and start the game • Outputs • Image on the beamer • Iphone vibrations

  6. Modalities – CASE/CARE • CASE • Semanticlevelof abstraction • CARE • Gesture orientation: assignment • Gesturepumping/Voice selection: complementary to open a gate • Voice commands: assignment • Decisionlevel fusion • Fission: image, vibration

  7. Implementation VICI • Blocks • Webcam, Tag detection • OpenGL, Physicengine • Multimodality Management • state machine • Augmented reality application • eventbased • Messages from the gateway • Voice events • Gestureevents (orientation X and Y, shake) • Messages to the gateway • Vibration events

  8. ImplementationIphone • Handle the UIAccelerometer interface • GeneratemotionEventwhenshaking • Messages to the gateway • Orientations (X or Y) • Shake • Messages from the gateway • Vibrate

  9. Implementation Voice 1/3 Windows speech API • SDK Features: • API definition files • Runtime component • Control Panel applet • Text-To-Speech engines in multiple languages. • Speech Recognition engines in multiple languages. • Redistributable components • Sample application code. • Sampleengines • Documentation.

  10. Implementation Voice 2/3 Our System • A speech recognition engine • A grammar <grammarxmlns="http://www.w3.org/2001/06/grammar" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.w3.org/2001/06/grammar http://www.w3.org/TR/speech-grammar/grammar.xsd" xml:lang="en-EN" version="1.0"> <rule id="Labyrinth" scope="public"> <one-of> <item>New game</item> <item>Pause</item> <item>Exit</item> <item>Open gate one</item> <item>Open gatetwo</item> <item>Close gate one</item> <item>Close gatetwo</item> </one-of> </rule> </grammar>

  11. Implementation Voice 3/3 • Recognition comparison before training / after training

  12. Implementation Network

  13. Demonstration • Live • Videos

  14. Conclusion • Problemswith the physicengine • Coordination user moves – physic moves • Voice recognition OK • High-levelprograming • Heterogeneity not a problem • Functional prototype

  15. Questions ? • Thankyou

More Related