160 likes | 364 Views
Multisensory & Multimedia. Overview. 5 senses: Vision, hearing, touch, smell & taste Connection with computers “sensory deprived and physically limited” Multimedia experience: multisensory Interviews: Hiroshi Ishii (Associate professor of Media Arts and Sciences @ MIT)
E N D
Multisensory & Multimedia
Overview • 5 senses: • Vision, hearing, touch, smell & taste • Connection with computers “sensory deprived and physically limited” • Multimedia experience: multisensory • Interviews: • Hiroshi Ishii (Associate professor of Media Arts and Sciences @ MIT) • Durrell Bishop (Teacher of Product Design @ Royal College of Art in London) • Joy Mountford (Member of Human Interface Group @ Apple) • Bill Gaver (Professor of Interaction Research @ Royal College of Art in London)
Hiroshi Ishii • Interaction requires 2 key components: Controls & representation of results • Graphical UI: representation pixels on screen • control remote: mouse, keyboard,… • Tangible UI: representation & control: tangible & coupled • GUI TUI: • Painted bits tainted bits by giving physical form to digital information • Multisensory multimedia interactions: fun & help us do things better
Tangible User Interfaces: example “The reacTable, is a state-of-the-art multi-user electro-acoustic music instrument with a tabletop tangible user interface.”
Tangible User Interfaces • http://tangible.media.mit.edu/projects/ [open] • Hiroshi Ishii’s latest project: Jabberstamp
Tangible User Interfaces • Control & representation of results: in a more physical form compared to GUI’s • Directly grab & manipulate results • Abacus: feel, manipulate, touch information
Tangible User Interfaces • musicBottles: • weather forecast bottle
Durrell Bishop • “Things should be themselves”: physical or virtual objects should be self-evident • Consumer electronics: universally black boxes of standard size • Labels say what a button does: merely an action on a word • Monopoly vs. VCR
Durrell Bishop • TUI: Marble answering machine
Durrell Bishop • Picking up tagged objects physical pointers
Joy Mountford • Goal of HIG @ Apple: design consistent interface for all users • 1980’s: Spreadsheets, word processing • Why not include pictures, sounds, movies? QuickTime • Designers design for themselves instead of for audience: Hand controller dilemma • Navigable movies (became QuickTime VR) • Object Maker: 3D object scanner: almost all viewpoints around objects •
Joy Mountford • Beyond video: audio & music • Soundscapes team @ Interval Research: Bead Box
Bill Gaver • Designing sound: SonicFinder (Apple) • Auditory representations of real-world objects • Enhanced experience examples: • - dragging a file • - copying a file sounds like pouring water • Properties of a file determines it’s sound, for instance: size
Equator project • History tablecloth
Equator project • - Key table/picture frame (capturing emotions)
Equator project • Drift table