1 / 14

Università degli Studi di Milano

Università degli Studi di Milano. A M ultimodal I nteractive S ystem to create and explore graph structures. Cristian Bernareggi , Loredana Parasiliti Provenza (joint work with C. Comaschi, A. Marcante, P. Mussio,

sumana
Download Presentation

Università degli Studi di Milano

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Università degli Studi di Milano A Multimodal Interactive System to create and explore graph structures Cristian Bernareggi, Loredana Parasiliti Provenza (joint work with C. Comaschi, A. Marcante, P. Mussio, S.Vanzi)

  2. N H H H The problem • Scientific reasoning widely employs spatial representations of concepts having an underlying graph structure. Different contexts: • Automata theory • Chemistry • Visually impaired or blind people meet with difficulties to perceive and reason on such representations.

  3. Related work • Tactile & audio-tactile systems for exploring 2D diagrams [Way&Barnek 1997], [Vanderheiden 1989] • Haptic and audio systems for exploring line graph (e.g. function diagrams): • to analyse and manipulate bar graphs [McGookin &Brewster 2006] • to compare multiple graphs [McGookin &Brewster 2006] • TeDUB: an audio-haptic system for exploring 2D UML diagrams [Petrie et al. 2002]

  4. Our goal • To enable visually impaired or blind people • to perceive and reasonon graph structures • to manipulate them • to communicate their reasoning on graph structure and experiences also with sighted people

  5. Our solution • Creation of a virtual environment for producing multimodal experiences of graph structures through a direct manipulation paradigm

  6. The MIS: a first experience • To explore the virtual space, shaped as a 3D discretegrid, where the user can experience: • Haptic modalities: the space is an haptic grid (when the user moves from one element to the next one s/he perceives the overcoming of a gentle threshold); • Audio modalities: the virtual space informs the user on her/his position within the grid • Visual modalities: the virtual space is a visual grid (a proxy represents the position of the haptic device in the space)

  7. The MIS: a second experience • To create, explore and manipulate graph structures. Visually impaired or blind users are able to • perceive a graph structure through haptic and auditive signals • manage and interact with the graph structure through direct manipulation operations Visual feedback supports communication on graph structures among sighted and not-sightedusers

  8. How to interact with the MIS MIS to create and explore graph structures FORCE FEEDBACK

  9. The current prototype A user aims at creating a graph structure that represents the ammonia molecular structure NH3

  10. System development and evaluation • The current prototype has been developed as a third step in a usability life cycle Evaluations performed with 2 types of blind users: • experts • not-experts in computer science and in the use of haptic device;

  11. Evaluation experiments • A first experiment with 2 expert blind users to assess: • the representation of the virtual space to better support the perception and navigation of the space and graph structures • the movement in the virtual space and within the graph structure • A second experiment (*) with 5 not-expert blind users to assess • the movement in the virtual space and graph nodes perception • How easily graph nodes can be found in the grid. • How useful speech messages are. (*) OBSERVED VARIABLES: the number of errors, the execution time for each considered tasks

  12. Some evaluation results • Virtual space representation & perception as: • Grid consisting of haptic points and lines better support the user than spheres and cylinders in navigating the 3D virtual space • Graph manipulation and perception by not-experts blind users • Users learned in few seconds (10-15 secs) how to move in the virtual space and place nodes • Node search: nodes inserted by the user found in about (20-25) sec, pre-existing nodes found in a longer time (45-60 sec); • Vibrational effect was rather misleading.

  13. Future Work • The system needs further enhancements: • To support more advanced exploration strategies based on guided movements (e.g. forces) rich audio descriptions and visual feedback • further evaluation experiments with both sighted and not-sighted users

  14. References • Way, T. P., and Barner, K.E. Automatic Visual to Tactile Translation, Part I: Human Factors, Access Methods and Image Manipulation. IEEE Trans. on Rehabilitation Engineering, 5, 1 (1997). • Vanderheiden, G. C. Nonvisual alternative display techniques for output from graphics-based computers. Journal of Visual Impairment and Blindness 83, 8 (1989), 383-390. • McGookin, D. K., and Brewster, S.A. MultiVis: Improving Access to Visualisations for Visually Impaired People. In extended Proc. CHI 2006, ACM Press (2006). • McGookin, D. K. and Brewster, S.A. 2006. Graph builder: Constructing non-visual visualizations. In ICAD 2006 (London, UK). • McGookin, D.K. and Brewster, S.A. 2006. Contextual Audio in Haptic Graph Browsing. In Proc. ICAD 2006 (London, UK), ICAD, 91-94 • Petrie, H., Schlieder, C., Blenkhorn, P., Evans, G., King, A., O’Neill, A.-M., Ioannidis, G.T., Gallagher, B., Crombie, D., Mager, R., and Alafaci, M. TeDUB: a system for presenting and exploring technical drawings for blind people. In K. Miesenberger, J. Klaus and W. Zagler (Eds.), LNCS 239: Computers Helping People with Special Needs. Heidelberg, Springer Verlag, 2002.

More Related