1 / 16

A Cross-modal Electronic Travel Aid Device

A Cross-modal Electronic Travel Aid Device. F. Fontana, A. Fusiello, M. Gobbi, V. Murino, D. Rocchesso, L. Sartor, A. Panuccio. Università di Verona Dipartimento di Informatica. Overview. Motivations The system Visual analisys Auditory display Work in progress. Motivations.

warren-johnson
Download Presentation

A Cross-modal Electronic Travel Aid Device

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Cross-modal Electronic Travel Aid Device F. Fontana, A. Fusiello, M. Gobbi, V. Murino, D. Rocchesso, L. Sartor, A. Panuccio. Università di Verona Dipartimento di Informatica

  2. Overview • Motivations • The system • Visual analisys • Auditory display • Work in progress Mobile HCI 2002

  3. Motivations • Humans use several senses simultaneously to explore and experience the environment. • Multimodal displays can enhance the user experience and the sense of engagement. • The redundancy of our sensory system can be exploited in order to choose the display that is the most convenient for a given application. • Our interest: exploring new ways of transferring information across different sensorial modalities, especially in the context of interactive systems. Mobile HCI 2002

  4. Cross-modality: methods to perform analysis tasks using one modality, and synthesis (display) tasks using another modality. • Examples: give auditory display to visual information, visualize auditory scenes, and sonify haptic sensations. • Design problem: how to render a given percept using a certain modality, the latter possibly being not the most obvious for the stimulus at hand. Mobile HCI 2002

  5. A cross-modality instance • Auditory display of visual information • More effective HC interfaces • Aid devices for visually impaired people • Auditory display needs sonification of the (visual) objects being displayed. • Sonification is the acoustic analogous of visualization. Mobile HCI 2002

  6. The system • The blind person is equipped with a stereo camera and earphones. • He/she uses a laser pointer as a cane. • The system computes the 3D position of the laser spot and sonifies this piece of information Mobile HCI 2002

  7. Visual analisys • Track the position of the laser spot in two images • Compute the 3D position of the laser spot by triangulation Mobile HCI 2002

  8. Stereopsis: the concept The same 3D point projects onto two different pixels: the difference is the disparity, which is related to the depth of the point. Mobile HCI 2002

  9. Laser spot tracking • Red band-pass filter • Brightness normalization • Temporal averaging • Thresholding • Size filter • Epipolar constraint • Kalman filter Mobile HCI 2002

  10. Auditory display • Sonification: the most important visual objects are associated to sounds that “label” such objects. • Visual objects are: • The laser spot • Disparity blobs • For disparity blobs we sonify their area, using pitch and loudness. Mobile HCI 2002

  11. Spatialization: sounds are positioned in a virtual scenario, and displayed in 3D • Relative angular position • Relative distance Mobile HCI 2002

  12. Spatial attributes • Distance is conveyed by processing sounds as if they were located inside a tube • Angular position: • Interaural delay • Head diffraction (low pass) Mobile HCI 2002

  13. The prototype Mobile HCI 2002

  14. Work in progress • Empirical evaluation • Porting to PocketPC • Sketch based interface • Other sonification modes (global) • Looking for other interesting applications of cross-modality Mobile HCI 2002

  15. Questions? Mobile HCI 2002

  16. Brown and Duda model • Interaural distance  time delay • Head diffraction  low pass filter on the opposite hear • Brown&Duda, IEEE Tr. Speech and Audio, 6(5) 1998. Mobile HCI 2002

More Related