Real-Time Cities: an Introduction to Urban Cybernetics Harvard Design School: SCI 0646900

1. Student Wenlig Li Exercise #2: Case Studies in Data Analysis (5 Ways to Make a Story out of Numbers) Real-Time Cities: an Introduction to Urban Cybernetics Harvard Design School: SCI 0646900 Spring 2014

2. Augmented Reality. Augmented reality technology, defined by levmanovich as the “overlaying of dynamic and context-specific information over the visual field of a user” 1 |The table is the score 2 | Hidden Worlds 3 |Messadi Voce concert 4 | The manual input 5 | VIDEOPLACE

3. 1 | The table is the score On the table are a variety of dark rubber and felt objects. The table is also a dry-erase surface and can be scribbled on with conventional whiteboard markers. Note the real-time video projection, from overhead, of various augmented-reality (AR) information layers: a grid representing subdivisions of time and pitch; a “Current-Time Indicator,” which scans the table lengthwise; and glowing haloes around the physical objects, indicating successful detection. Project Video: http://www.flong.com/texts/publications/pub_icmc06/

4. 1 | The table is the score

5. 1 | Sensing the City Which of the 10 possible categories of scenarios the Project belongs to? VI Mediator between fleeting encounters Which sensing technology/ types of sensors are deployed to impliment spatial gaze? IR camera How the performance of the piece/instalation is conditioned? Graphic information processing

6. 1 | Sensing the City Which actuation technology is used to provide for dynamism of the piece? Projection What connectivity technology is used to connect different parts of the informatically driven system at question? processing Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? No

7. 1 | Sensing the City What is the extreme vision of the scenario? Each object or person enter a space or plaza will be casted with an outline of light, as if on the stage. Why is the project significant? It relates presence to high lightening directly. What do you identify as a poetic aspect to the logic of operation of the system at question? Every presence in the space are given attention and respect.

8. 1 | Sensing the City What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? If installed in street crossing or plaza, it might interfere with privacy? What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Good How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re-appropriated for the different context? To a street corner, plaza, or a stage.

9. 1 | Sensing the City How the system will deal with nodal or total failure in operation? Answer… … What is the system’s logic of operation and information flow/architecture? Answer… … What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer… …

10. 2 | Hidden Worlds Hidden Worlds is an interactive audiovisual installation, or, alternatively, an augmented-reality speech-visualization system. Participants in Hidden Worlds are able to “see” each others’ voices, which are made visible in the form of animated graphic figurations that appear to emerge from the participants’ mouths while they speak. Project Video: http://www.flong.com/texts/publications/pub_icmc06/

11. 1 | The table is the score

12. 1 | Sensing the City Which of the 10 possible categories of scenarios the Project belongs to? VI Mediator between fleeting encounters Which sensing technology/ types of sensors are deployed to impliment spatial gaze? Acoustic. How the performance of the piece/instalation is conditioned? In the installation, visitors wear special see-through data glasses, which register and superimpose stereoscopic 3D graphics into the real world. When one of the users speaks or sings, colorful abstract forms appear (through the goggles) to emerge from his or her mouth. The graphics representing these utterances assume a variety of shapes and behaviors that are tightly coupled to the unique qualities of the vocalist’s volume, pitch and timbre.

13. 1 | Sensing the City Which actuation technology is used to provide for dynamism of the piece? To accomplish this, we use headsets consisting of a home-brewed combination of an Ascension Flock of Birds 6-degree-of-freedom position tracker; Sony I-Glasses which had been modified (with additional view-holes and custom half-silvered optics) to be semitransparent, and a miniature AKG vocal microphone mounted within the bridge of the Sony glasses. What connectivity technology is used to connect different parts of the informatically driven system at question? Visitor utterances are segmented, analyzed and then represented graphically as noodle-like “sound-gestures.” We employ a variety of speech analyses to parameterize the shapes of the soundgestures, but the most perceptually salient of these are duration (mapped to the visual length of sound-gestures) and volume (mapped to changes in the sound-gestures’ diameter). Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? Yes, the length relates to the time duration, the width relates to the volume of the sound

14. 1 | Sensing the City What is the extreme vision of the scenario? 6 persons shout at each other Why is the project significant? Answer… … What do you identify as a poetic aspect to the logic of operation of the system at question? Answer… …

15. 1 | Sensing the City What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer… … What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? mediem How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re-appropriated for the different context? It can be used in observation of bees’ routes and location in a box, if scaled down

16. 1 | Sensing the City How the system will deal with nodal or total failure in operation? Answer… … What is the system’s logic of operation and information flow/architecture? Gathering acoustic data and then generate visual image What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Transform data between accoustic and visual.

17. 3 |Messadi Voce concert Messa di Voce, a concert performance in which the speech, shouts and songs produced by two vocalists are augmented in real-time by custom interactive visualization software. Created in collaboration with Joan La Barbara and JaapBlonk—two singer/composers known for their experimental vocal techniques—the performance touches on themes of abstract communication, synaesthetic relationships, cartoon language, and writing and scoring systems, within the context of a sophisticated and playful audiovisual narrative. Project Video: http://www.flong.com/texts/publications/pub_icmc06/

18. 1 | The table is the score

19. 1 | The table is the score

20. 1 | Sensing the City Which of the 10 possible categories of scenarios the Project belongs to? III performing arts Which sensing technology/ types of sensors are deployed to impliment spatial gaze? Answer… … How the performance of the piece/instalation is conditioned? Answer… …

21. 1 | Sensing the City Which actuation technology is used to provide for dynamism of the piece? Answer… … What connectivity technology is used to connect different parts of the informatically driven system at question? Answer… … Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? Answer… …

22. 1 | Sensing the City What is the extreme vision of the scenario? Answer… … Why is the project significant? Answer… … What do you identify as a poetic aspect to the logic of operation of the system at question? Answer… …

23. 1 | Sensing the City What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer… … What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Answer… … How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re-appropriated for the different context? Answer… …

24. 1 | Sensing the City How the system will deal with nodal or total failure in operation? Answer… … What is the system’s logic of operation and information flow/architecture? Answer… … What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer… …

25. 4 | The manual input intended to probe the expressive possibilities of unencumbered hand gestures. The concert consists of a suite of custom vision based software systems, described below, which are performed on a combination of a traditional analog overhead projector, and a modern DLP video projector. In our setup, the analog and digital projectors are registered and aligned such that their projections overlap, resulting in an unusual quality of hybridized, dynamic light. Project Video: http://www.flong.com/texts/publications/pub_icmc06/

26. 1 | The table is the score

27. 1 | Sensing the City Which of the 10 possible categories of scenarios the Project belongs to? V. One interact while others watch Which sensing technology/ types of sensors are deployed to impliment spatial gaze? camera How the performance of the piece/instalation is conditioned?

28. 1 | Sensing the City Which actuation technology is used to provide for dynamism of the piece? In our setup, the analog and digital projectors are registered and aligned such that their projections overlap, resulting in an unusual quality of hybridized, dynamic light. What connectivity technology is used to connect different parts of the informatically driven system at question? Answer… … Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? No.

29. 1 | Sensing the City What is the extreme vision of the scenario? Why is the project significant? It input figure as a data manually What do you identify as a poetic aspect to the logic of operation of the system at question? Answer… …

30. 1 | Sensing the City What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer… … What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Answer… … How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re-appropriated for the different context? Applied in stage setting and design where the silhouette of the dancer can be operated

31. 1 | Sensing the City How the system will deal with nodal or total failure in operation? Answer… … What is the system’s logic of operation and information flow/architecture? Answer… … What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer… …

32. 2 | VIDEOPLACE Myron Krueger’s influential VIDEOPLACE system, created more than thirty years ago, was one of the first interactive artworks to make use of computer vision as a means for capturing and incorporating the gestural expressions of its users. By detecting and tracking the tips of participants’ fingers, for example, Krueger enabled participants to create virtual synthetic drawings in mid-air; to adjust the shape of a virtual Bezier curve; and to experience the fiction of picking up and holding the entire body of a networked cohort in one’s hands. Project Video:https://www.youtube.com/watch?v=dmmxVA5xhuo /

33. 1 | The table is the score

34. 1 | Sensing the City Which of the 10 possible categories of scenarios the Project belongs to? III performance art Which sensing technology/ types of sensors are deployed to impliment spatial gaze? camera How the performance of the piece/instalation is conditioned? Answer… …

35. 1 | Sensing the City Which actuation technology is used to provide for dynamism of the piece? the computational augmentation of hand silhouettes could be used to prompt both narrative and abstract forms of creative visual play What connectivity technology is used to connect different parts of the informatically driven system at question? Answer… … Is the system operates based on a memory of the past and/or anticipation of the future through recognition of past patterns? Answer… …

36. 1 | Sensing the City What is the extreme vision of the scenario? Answer… … Why is the project significant? one of the first interactive artworks to make use of computer vision as a means for capturing and incorporating the gestural expressions of its users. What do you identify as a poetic aspect to the logic of operation of the system at question? Interactive between human and in-human

37. 1 | Sensing the City What challenges are inherent in deployment of the project or maintaining the integrity or level of operation of the system in the long run? Answer… … What is your evaluation of the design of the physical manifestation of the system in terms of its esthetics? Answer… … How can the system scale up or down to fit a different context of operation? How can its logic of operation or technology be re-appropriated for the different context? Answer… …

38. 1 | Sensing the City How the system will deal with nodal or total failure in operation? Answer… … What is the system’s logic of operation and information flow/architecture? Answer… … What is the nature of actuation explored by the system? Does it explore generation of change that is registred through other senses beyond vision? Answer… …