1 / 24

Using Tweek to Create Graphical User Interfaces in Virtual Reality

Using Tweek to Create Graphical User Interfaces in Virtual Reality. Patrick Hartling IEEE VR 2003. Overview. Problem description Goals for Tweek software Design overview Implementation overview Future work. Problem Description. Interaction in VR is primarily spatial

rico
Download Presentation

Using Tweek to Create Graphical User Interfaces in Virtual Reality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Tweek to Create Graphical User Interfaces inVirtual Reality Patrick Hartling IEEE VR 2003

  2. Overview • Problem description • Goals for Tweek software • Design overview • Implementation overview • Future work

  3. Problem Description • Interaction in VR is primarily spatial • Not all interactions map well to this method • Alphanumeric input • Fine-grained interaction • List selection • Per-application custom interfaces • Map abilities of specific devices to interaction • Typically use 3D input devices • Why abandon effective 2D interaction techniques in 3D space?

  4. Motivation • Why have GUIs in virtual reality? • GUIs are everywhere in the computing world • Most computer users understand how to use a GUI • Why not have GUIs in VR?

  5. Issues with GUIs inVirtual Reality (1/2) • Positives • Typically more flexible and more extensible than other input devices • More input methods means less overloading of button presses, gestures, etc. • More portable between VR systems than hardware input devices • Familiarity from desktop experience

  6. Issues with GUIs inVirtual Reality (2/2) • Negatives • No direct physical correspondence • In VR, a user could grab an object with his/her hand or use a GUI • No GUI in real world for moving the object • Remote users may not see GUI [Park00] • Use of the interface may be difficult • Collision detection • Placement • Immersive interfaces have to re-invent GUI technology

  7. Previous Work (1/3) • Many areas of previous work • Widget sets for VR • PDAs as input devices • Floating menus and heads-up displays • Virtual User Interface (VUI) • Originally developed at Iowa State University [Heath98] • 3D immersive user interface • May be an open source release from Fuel Tech

  8. Previous Work (2/3) • Java Interface to Virtual Environment (JAIVE) • Used Java interface on handheld computer [Hill00] • Communicated with C++ applications • Interface could be extended from C++ using JAIVE protocol

  9. Previous Work (3/3) • PDA in a CAVE™ system • Kent Watsen, Naval Postgraduate School • Experimented with use of Palm PDA in CAVE [Watsen99] • 3DI Group • Doug Bowman, Virginia Tech • Investigating differences in system interaction techniques between VE displays • Active on 3DUI mailing list

  10. Tweek • Framework for the development of cross-platform, cross-language, scalable GUIs • GUIs communicate with remote applications • GUIs can be use on the desktop, on a PDA, or in an immersive VR environment • Features • GUI panels are dynamically loaded components • Panels use standard 2D GUI “widgets” • Dynamic interface update and extension • Multi-language capabilities

  11. Goals • Allow GUI scalability • Same GUI runs on the desktop, a PDA, and a projection-based VR system • Must provide an easily extendable framework for cross-platform, cross-device GUIs • Interact with VR applications written in any programming language • Fully separate the GUI from the application • Reuse existing GUI technology • We do not want to re-invent scrollbars, menus, etc. • Avoid cluttering immersive space

  12. Same GUI, Multiple Uses (1/2) • PDA in Projection System • Offers force feedback • High-resolution display • Back-lit display is needed • Widely available • A person with a PDA can walk into the VR system and get the GUI dynamically

  13. Same GUI, Multiple Uses (2/2) • Desktop simulation • Standard GUI interface used to control VR software in simulator mode • Immersive 3D space • VRJ VNC incorporates 2D GUI into 3D visualization • Modeled after 3Dwm (www.3dwm.org) use of VNC and other VR VNC viewers

  14. Design Overview (1/3) • Distributed Model/View/Controller

  15. Design Overview (2/3) • Observer design pattern • Simplified MVC: subject, observer • VR application is the subject • Maintains user-defined state information • Per-application customizations • GUI is the observer • Provides a view of subject’s state • GUI interactions manipulate application state • Benefits • Distinct separation of application and GUI • Allows multiple GUIs to observe the same application

  16. Design Overview (3/3) • Two-way communication between application and GUI • Application can send data to the GUI • GUI flexibility • A GUI can load any component it needs to make interaction more effective • Not limited to direct manipulation of application state

  17. Implementation Overview • Language tools • C++ VR applications • Java-based GUI composed of JavaBeans • Beans can be “pushed” to the GUI while the user is in the virtual space • Dynamic extension of interface • CORBA communication between C++ and Java

  18. Example Tweek Interface • Multi-purpose tool • Navigation • Map visualization • Data visualization • Information storage to spreadsheet-friendly format

  19. VRJ VNC: Embedding 2D GUIs in 3D Space • Embed any 2D GUI in 3D world • Uses VNC • Can be added to existing VR Juggler applications

  20. VRJ VNC: Presentations in VR • Bring PowerPoint presentation into the 3D environment

  21. VRJ VNC: Embedding Tweek • Tweek Java GUI is just another window on a desktop • Set VNC desktop size to give the desired embedded frame size • Interaction with the GUI is translated to the desktop via VNC

  22. Future Work • Use Java Applets to allow Tweek to be used from a web browser • Make use of a familiar interface • Easy access to application GUI for remote demonstrations

  23. Online Information • 3DUI • http://www.mic.atr.co.jp/~poup/3dui.html • Tweek • http://www.vrjuggler.org/tweek/ • Contains publications, guides, and programmer references

  24. References • [Park00] Park, K. et al. “Lessons Learned from Employing Multiple Perspectives in a Collaborative Virtual Environment for Visualizing Scientific Data”. In Proceedings of ACM Conference on Collaborative Virtual Environments, San Francisco, California, September 10–12, 2000. • [Heath98] Heath, D. “Virtual User Interface (VUI): A Windowing System for VR”. In Proceedings of 2nd Immersive Projection Technology Workshop, Ames, Iowa, May 11–12, 1998. • [Hill00] Hill, L., Usability of 2D Palmtop Interaction Device in Immersive Virtual Environment. Master’s thesis, Iowa State University, Ames, IA, 2000. • [Watsen99] Watsen, K. “A Handheld Computer as an Interaction Device to a Virtual Environment”, In Proceedings of 3rd Immersive Projection Technology Workshop, Stuttgart, Germany, May 10–11, 1999.

More Related