Locative Media

1. Locative Media Lalya Gaye Ubiquitous Computing course IT-University in Göteborg 31 November 2007

2. Introduction Locative Media Lecture Aims and scope • Overview of the field • Technology overview • Discussion of design and prototyping approaches • Design issues: focus on sustainability in locative media

3. IntroductionLecture Content Ubiquitous computing: recap Ubicomp technologies Locative Media: definition and origins Themes, projects and related design issues Characteristics, challenges and design opportunities Technologies available to the general public Sustainable Design?

4. Ubiquitous ComputingRecap

5. Ubiquitous Computing Recap • Mark Weiser’s vision (1991) • disappearing computer • everyday world literally used as interface “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

6. Ubiquitous Computing Recap • The computer: calculator -> information system -> interactive -> pc -> mobile, integrated, networked • Levels of interaction: electrical -> symbolic -> textual -> visual -> social, tangible • Evolution of the user interface: from immersing the user in the computer’s world to computing increasingly adapting to the user’s world and skills. • Ubicomp = opposite of virtual reality: embedded reality.

7. Ubiquitous Computing Recap • Evolution of computer-human interaction: • more of the human’s everyday world and everyday skills in computing • computers an increased part of our everyday life • requiring less specialised knowledge to operate them • relying increasingly on user’s everyday skills • smaller computers • from one computer for many user, to many computers

8. Ubiquitous Computing Recap • Designing ubicomp systems: Focus on the interaction between user & technology (as opposed to form and function), on what experience the user gets from it, on what added-value ubicomp brings to his/her life. • Follow needs and requirements but also entice new behaviours?

9. Ubiquitous Computing Recap • Enhance people’s activities by making computing available at hand, when and where needed (including when the users are mobile) • Computing naturally blending into everyday settings, vanishes into the background • The physical and social world around us as digitally augmented and distributed interface • Manipulating digital data = manipulating entities in the physical world • Literally build on people’s everyday use of the physical and social world, in situation and in real time. • Peripheral awareness • Greenfield: “information processing dissolving into behaviour” • IT + everyday life as design material (f. ex. I/O Brush)

10. Ubiquitous Computing Recap Implementing the ubicomp vision: • Many interconnected computers per person • Mobile devices combined with computers embedded in the environment (e.g. post-hoc augmentation of everyday objects with sensors and networked communication) • With awareness of physical & social context + each other -> Mapping the digital world to the physical one -> User interface: tangible and embedded in the real world

11. Ubiquitous Computing Recap Implementing the ubicomp vision: • Distributed interface: networking mobile devices and embedded computers (sensors, processors, etc) -> flexible and seamless integrated whole -> e.g. any display or input device can become one’s own (user mobility) • Interaction in context and in real time (f.ex. tracking things and people -> relevant information and interaction opportunity to the right person at the right time)

12. Ubiquitous Computing Recap Types of systems: • “walk-up-pop-up” • wearables • ambient displays • intelligent work environments • augmented, interconnected everyday objects • etc Media cup, TecO

13. Ubiquitous ComputingTechnologies

14. Ubiquitous Computing Technologies • Ubiquitous Computing (Weiser): computing interweaved in everyday life, “where the action is” (Dourish) • context awareness • embedded sensor networks • global positioning • wearable computing • augmented & mixed-reality • ad hoc and p2p user networks

15. Ubiquitous Computing Technologies * Embedded sensor networks • Sensors: - in everyday environments - on people - on artefacts • Sensor fusion: combining different data and placements to gather context

16. Ubiquitous Computing Technologies * Context-aware computing • “computer-based devices [that] reach out into the real world through sensors” [Gellerson]. • “A system is context-aware if it uses context to provide relevant information and/or services to the user, where relevancy depends on the user’s task.” [Dey & Abowd, 1999].

17. Ubiquitous Computing Technologies * Context-aware computing • Enables computing to run into the background and adapt to changes of context in order to present appropriate behaviour to specific situations. • “presentation of information and services to a user” • “automatic execution of a service” depending on context appropriateness • or “tagging of context to information for later retrieval” [Dey].

18. Ubiquitous Computing Technologies * Context-aware computing Gellersen et al.

19. Ubiquitous Computing Technologies * Context-aware computing Gellersen et al.

20. Ubiquitous Computing Technologies * Tangible computing • Input, data, output and networking contained and accessed within the same tangible artefact • Paper, cups, pens, umbrellas or specially designed artefacts • Tangible objects as active entities that respond to the environment, to user manipulation and people’s activities in general • Building on the users’ cognitive abilities

21. Ubiquitous Computing Technologies * Social computing • Incorporating understandings of the social world into interactive systems • Social traces left by people on objects or places • Mobile social networks between co-located acquaintances • enhancing user awareness by providing them information about others and their activity

22. Ubiquitous Computing Technologies * Augmented reality • Superimposing a digital world upon the real one • User experiences both as co-existing parts of the same reality • User is able to interact with their combination in real time • Interfaces: • 3D computer graphics seen through transparent head-mounted displays or augmented glasses • Spatialised audio cues heard through headphones

23. Ubiquitous Computing Technologies * Augmented reality • Mixed-reality:digital world not directly overlaid on the physical one but still presented as part of the same reality, f.ex. • with both realities displayed on the screen of hand-held device)

24. Ubiquitous Computing Technologies * Wearable computing • Computing incorporated into clothing • Make use of body-related information or interaction forms to control processes : - body movements- biometrics • Embedded displays (e.g. glasses)

25. Ubiquitous Computing Technologies * Platforms: • Smart-Its • Smart Dust • Pin & Play • Tiny OS • etc

26. Ubiquitous Computing Technologies • Smart-Its: • sensors: sound, light, acceleration (2d), pressure • core board: context-recognition, communication interface (RF)

27. Locative Media:Background

28. Locative Media Background • Typical contexts of use for ubicomp: home, office work, cafeterias, grad-students research labs, etc • Locative media = media with sense of place • New media + urban aesthetic practices + community uses of public space + contextual art + mobile, ubiquitous and geographical technologies • City, public spaces • Ubiquitous computing in public space: Minority Report dystopia (video: 44:20) vs. current creative uses and appropriations of public space?

29. Locatived Media Background Urban aesthetic practices • Mobility as creative act • Creative use of public space • Walking: • aboriginal walkabouts • situationist dérive, psycho-geography

30. Locative Media Background Urban aesthetic practices • Mobility as creative act • Creative use of public space • Graffiti • Reclaim the Streets • Urban sports: • skateboarding • parkour (video) -> urban space as resource for aesthetic movements

31. Themes and Projects

32. Locative Media Projects Themes • Pervasive Gaming: the world as a game-board • Space annotation: media with a specific position in space • Location awareness & GPS-enabled locative media • Mobile music & locative audio • Radio pirates • Social spaces • etc

33. Locative Media Projects Pervasive Gaming Locative Media Projects Pervasive Gaming • The world as game-board • Botfighters and Pirates! • Backseat Gaming (video) • Can You See Me Now? (video) • iPerG • ... Can You See Me Know? Blast Theory + Equator

34. Locative Media Projects Space Annotation • Media with a specific position in space • User-authored social cues • Virtual: Geonotes (video)Urban Tapestries(animations) • Physical: Yellow Arrow (video)Grafedia Grafedia, grafedia.net Yellow Arrow, Count Media

35. Locative Media Projects GPS & Positioning Hundekopf, knifeandfork • GPS-drawing • Non-linear narratives: Hundekopf (video)

36. Locative Media Projects GPS & Positioning • Tracking and mapping paths • Biomapping (video), Drift, Net_Derive (video)... Biomapping, Christian Nold Drift, Teri Rueb

37. Locative Media Projects Mobile Music and Locative Audio • Audio space annotation • Mobile music sharing/listening: • distributed • ad hoc • sound walks • Mobile music making: • situated • collaborative • Wearable audio

38. Locative Media Projects Mobile Music and Locative Audio • Audio space annotation • Hear&There • (Rozier, MIT Medialab, 1999) • Tacticle Sound Garden [TSG] (video) • (Mark Shepard, Buffalo Univ. 2004-06) • Tejp / Audio tags • (PLAY & FAL, 2003-04)

39. Locative Media Projects Mobile Music and Locative Audio • Audio space annotation • Audio Bombing (video) • (Fleming et al., 2007) • Sonic Graffiti (video) • (C-Y Lee, 2007)

40. Locative Media Projects Mobile Music and Locative Audio • Audio space annotation • [Murmur] (murmur.ca)

41. Locative Media Projects Mobile Music and Locative Audio • Sound walks • Electric walks (Christina Kubisch) • Drift (Rueb) • 34n118w (Knowlton, Spellman, 2005) • Craving (Garnicnig, Haider, 2007) • Seven Mile Boots (Beloff et al., 2003-04) • The Case at Kulturhuset • (Knifeandfork, 2004) • Riot! (Mobile Bristol, Hewlett Packard)

42. Locative Media Projects Mobile Music and Locative Audio • Distributed and located music • Location 33 (Carter & Liu, USC, 2005)

43. Locative Media Projects Mobile Music and Locative Audio • Mobile music sharing • SoundPryer (Mattias Östergren, Interactive Institute, 2001) • TunA • (Arianna Bassoli et al., • Medialab Europe, 2002)

44. Locative Media Projects Mobile Music and Locative Audio • Mobile music sharing • Bass Station • (Mark Argo & Ahmi Wolf, 2003) • Push!Music(Håkansson et al., 2005)

45. Locative Media Projects Mobile Music and Locative Audio • Situated music making • Sonic City (video) • (Gaye et al., FAL & PLAY, 2002-04) • Sound Lens(Toshio Iwai, Tokyo Univ.) • Solarcoustics: CONNECT (Barnard, ITP/NYU, 2005)

46. Locative Media Projects Mobile Music and Locative Audio • Situated music making • Sound Mapping (video) • (Mott et al., Reverberant, 1997) • Sonic Interface • (Akitsugu Maebayashi, 1999) • Warbike • (McCallum, 2005-06) • Skatesonic (video) (van Toder, 2006)

47. Locative Media Projects Mobile Music and Locative Audio • Collaborative mobile music making • ImprovE (video) • (Wideberg & Hasan, 2006) • CosTune • (Nishimoto et al., ATR, 2001) • Malleable Mobile Music • (Atau Tanaka, Sony CSL, 2004)

48. Locative Media Projects Mobile Music and Locative Audio • Collaborative mobile music making • China Gates (Clay, Majoe, 2006) • Sequencer404 (Hatcher, Jimison et al., 2006) • Cellphonia (Bull et al, 2006)

49. Locative Media Projects Mobile Music and Locative Audio • Wearable audio • Nomadic Radio (Shawney, MIT Medialab, 1998) • Sonic Fabric (Alice Santaro, 2002)

50. Locative Media Projects Mobile Music and Locative Audio • Wearable audio • ”Personal instruments” • (Krzysztof Wodiczko, 1969) • (Chelle Hugues, RCA/CRD, 2000)