1 / 15

Multimodal Interfaces in a Ubiquitous Computing Environment

Multimodal Interfaces in a Ubiquitous Computing Environment. 3 rd UK-Ubinet Workshop —————— 9 th – 11 th February 2005 —————— Fausto. J. Sainz Salces, Dr. David Llewellyn-Jones, Prof. Madjid Merabti —————— School of Computing and Mathematical Statistics Liverpool John Moores University

chesna
Download Presentation

Multimodal Interfaces in a Ubiquitous Computing Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimodal Interfaces in a Ubiquitous Computing Environment 3rd UK-Ubinet Workshop —————— 9th – 11th February 2005 —————— Fausto. J. Sainz Salces, Dr. David Llewellyn-Jones, Prof. Madjid Merabti —————— School of Computing and Mathematical Statistics Liverpool John Moores University James Parsons Building Byrom Street Liverpool, L3 3AF, UK {cmsfsain, D.Llewellyn-Jones, M.Merabti}@livjm.ac.uk http://www.cms.livjm.ac.uk/PUCsec/

  2. Disappearing Computers – Disappearing Interfaces • The Ubiquitous Computing paradigm presents difficult user interface problems • Enforces a minimal interface • Requires increased usability • Current user interfaces tend to concentrate on visual display techniques • Need to explore alternative methods for transferring information and allowing control

  3. Harnessing the Senses • We felt the most appropriate sense for information presentation after vision was sound • Can alert the user without requiring their specific attention • Ambient interface ideal in a pervasive environment • Considerable work exists in the field of audio interfaces • Earcons: tones or sequences of tones as a basis for building messages [1,2] • Auditory icons: everyday sounds that convey information about events by analogy with everyday sound producing events [3] • Voice: instructions are read either as a recorded or simulated voice • Our work looks at the use of earcons in a Ubiquitous Computing Environment [1] M. Blattner, Sumikawa, D. & Greenberg, R., "Earcons and icons: Their structure and common design principles," Human Computer Interaction, vol. 4(1), pp. 11-44, 1989. [2] Brewster, S., Wright, P.C., Edwards A.D.N. (1994). “A detailed investigation into the effectiveness of earcons,” in Auditory Display. Sonification, audification and auditory display., vol. XVIII, G. Kramer, Ed. Santa Fe, U.S.A.: Addison-Wesley, 1994, pp. 471-498. [3] W. W. Gaver, "Using and creating auditory icons.," in Auditory Display. Sonification, audification and auditory display., vol. XVIII, G. Kramer, Ed. Santa Fe, U.S.A.: Addison-Wesley, 1994, pp. 417-446.

  4. Multimodality • The use of several means to present information simultaneously • Combine the use of visual icons and earcons • Makes sure the message arrives at the receiver • Redundant info • Multiple channels (also increases accessibility) • More natural presentation method • Undertook experiments to compare devices used to control household appliances • Small handheld device with a restricted form factor • Large screen ‘fixed’ laptop device • Both devices were used with various combinationsof visual, audio and multimodal interfaces

  5. Interface Design • We compared people’s perceptions and ability to use each device using 6 permutations of the interface • Visual – Four device control • Earcons – Six device control • Multimodal

  6. Design Process • An initial design stage was used to establish effective designs for each of the interfaces modalities • Human constraints • Machine constraints • Cognitive knowledge • User preference • An iterative design method was used

  7. Motif 1 Motif 2 Hob on Earcon Design • Each earcon is comprised of two parts • The first motif represents the household appliance • The second motif represents the new state of the device (e.g. “on” or “off”) Hob off

  8. Experiments • Experiments were undertaken individually with two groups of users • 20–33 year olds • 60–84 year olds • The intention was to observe a broad range of users, both relatively proficient users and computer novices

  9. Results: Comparing Large & Small • Overall, users made favourable comments about the smaller handheld devices • The most common remark concerned the positive usability aspect of the palmtop“Small portable, clear, handy able to carry in one hand” • However, people cited the small screen as being a problem • The larger screen was seen as a positive aspect of the laptop device, but this was countered by its fixed nature • Interestingly, the only comment concerning security, was in favour of the palmtop

  10. Results: Device Ranking • The comments are echoed by the preference ranking that subjects gave for the devices Ranked Highest Ranked Lowest 60 60 50 50 40 40 Percent Percent 30 30 20 20 10 10 0 0 Screen Audio Screen Visual Palmtop Visual Screen Audio Screen Visual Screen Multimodal Palmtop Audio Palmtop Visual Palmtop Multimodal Palmtop Multimodal

  11. Audio Visual Multimodal Mean 48.73 23.04 24.07 Mode 30 5 & 10 5 Std. Deviation 25.229 22.140 23.576 Audio Visual Multimodal 14 25 30 12 25 20 10 20 15 Frequency Frequency 8 Frequency 15 6 10 10 4 5 5 2 0 0 100 0 100 100 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 0 5 10 5 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 5 10 15 20 25 30 35 40 45 50 60 70 75 80 85 Results: NASA TLX • Mental Demand measurement“How much mental demand and perceptual activity was required (e.g. thinking, deciding, calculating, remembering, looking, searching, etc.)? Was the task easy or demanding, simple or complex, exacting or forgiving?”

  12. Audio Visual Multimodal Mean 40.00 21.91 20.59 Mode 5 5 5 Std. Deviation 29.593 23.964 24.468 Visual Multimodal Audio 40 14 30 12 30 10 20 Frequency Frequency 8 Frequency 20 6 10 4 10 2 0 0 0 100 100 100 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 80 90 95 0 5 10 15 20 25 30 35 40 45 50 55 60 70 80 85 90 95 0 5 25 40 45 20 30 35 50 70 75 80 95 55 60 65 85 90 10 15 100 Results: NASA TLX • Performance measurements“How successful do you think you were in accomplishing the goals of the task set by the experimenter? How satisfied were you with your performance in accomplishing these goals?”

  13. Audio Visual Multimodal Mean 37.06 22.25 18.28 Mode 5 5 5 Std. Deviation 28.773 22.763 18.912 Audio Visual Multimodal 20 40 40 15 30 30 Frequency Frequency Frequency 10 20 20 10 5 10 0 0 0 100 100 100 0 5 10 15 20 25 000 35 40 45 50 00 65 70 75 80 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 0 5 10 15 20 25 30 35 40 45 50 60 70 80 85 90 Results: NASA TLX • Frustration measurement“How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified, content, relaxed and complacent did you feel during the task?”

  14. Conclusions • Users generally found the handheld device to be better than the larger screen laptop • Many of the benefits were described in terms that might apply in a Ubiquitous Computing environment • Multimodal designs were preferred • Earcons can be used to augment a limited user interface in a beneficial way • Earcons alone performed significantly worse than visual icons alone • These results applied across both groups of subjects • Further results concerning reaction times are in the process of being analysed

  15. Multimodal Interfaces in a Ubiquitous Computing Environment 3rd UK-Ubinet Workshop —————— 9th – 11th February 2005 —————— Fausto. J. Sainz Salces, Dr. David Llewellyn-Jones, Prof. Madjid Merabti —————— School of Computing and Mathematical Statistics Liverpool John Moores University James Parsons Building Byrom Street Liverpool, L3 3AF, UK {cmsfsain, D.Llewellyn-Jones, M.Merabti}@livjm.ac.uk http://www.cms.livjm.ac.uk/PUCsec/

More Related