1 / 26

Advisor: Prof. Francesco Tisato Tutor: Prof. Carla Simone September 23th 2009

University of Milano-Bicocca, Doctoral School of Science Ph.D. Program in Informatics, Cycle XXIV Department of Informatics, Systems and Communication

jana
Download Presentation

Advisor: Prof. Francesco Tisato Tutor: Prof. Carla Simone September 23th 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. University of Milano-Bicocca, Doctoral School of Science Ph.D. Program in Informatics, Cycle XXIV Department of Informatics, Systems and Communication Diego BerniniPh.D. research status and thesis hypothesisSoftware Architectures forResponsive Environments:the case of Interactive Art & Design Advisor: Prof. Francesco Tisato Tutor: Prof. Carla Simone September 23th 2009

  2. Outline • Reference scenario • Research topic, status and thesis hypothesis • State of the art • Plan of the activities • Miscellanea D. Bernini, Ph.D. research status

  3. Software Architecture • Having a sound software architecture is a key aspect of ICT systems • In different classes of ICT systems relevant high-level concepts are not turned into proper elements in their software architecture • need of proper architectural abstractions • examples: time and space in surveillance systems and Geographic Information Systems (Micucci et al., 2006) (Micucci et al., 2009) • This leads to several issues • intermix of conceptual, design aspects and implementation details • strong dependency on specific technologies D. Bernini, Ph.D. research status

  4. Responsive environments • Real physical spaces equipped with ICT technologies capable of sensing and responding accordingly to entities that inhabit them (Silva and Vasconselos 2007) (Pinhanez and Bobick 2003) • Different aims • From more concrete … • Automation of offices, shops, houses (Elrod et al., 1993) • …to more aesthetics • Interactive Art & Design (Bullivant 2007) (Bullivant 2006) • Various related research areas, different characterizations • From Software Engineering, Ubiquitous (Pervasive) Computing • to Human-Computer Interaction, Ambient Intelligence D. Bernini, Ph.D. research status

  5. ..in Interactive Art • ICT-supported interactive art installations • Art Installations: artworks where space is central • “art that appropriates space to its own artistic ends” (Bestor 1996) • Interactive Art installations: the viewer become an active player • ICT-supported: some interactions are achieved by ICT • For itself: they are integrant part of the installation • For the exhibition: valorization of traditional artworks/installations • Interactive museums and galleries (Studio Azzurro, www.studioazzurro.com) (Rafael Lozano Hemmer, http://www.lozano-hemmer.com) D. Bernini, Ph.D. research status

  6. .. in Interactive Design • Various classes of interactive small-scale architectural & design interventions (Castle 2007) (Bullivant 2007) (Bullivant 2006) • Interactivity: the environment enhance the user experience • Interactive Architecture & Design, urban-open & closed spaces (Interactive tunnel, Vito Acconci Studio and CSAI, (Bandini et al 2008)) (Digital Pavillion, ONL [Oosterhuis_Lénárd] design office http://www.oosterhuis.nl) D. Bernini, Ph.D. research status

  7. High-level concepts: examples • Time • real-time responsiveness: the interactions may require strong time constraints • temporal input analysis and time-driven output • Space • locating objects in multiple spaces (e.g. sensor network space vs. environment topology) • Multi-modality • Multiple heterogeneous sensing and responding • Sensing: • Strict physical environmental properties • Pressure, temperature, humidity, lighting condition, noise levels • Entity properties • Presence, identification, motion, contact, physiological • Responding: audio and visual effects, commanding of specific devices D. Bernini, Ph.D. research status

  8. Current practice Various development tools, but no explicit treatment of the previous high level concepts! low level of abstraction Hardware computing devices: “low level” programming E.g. Arduino, input, output & computing device, C programming oriented to specific technologies Different visual programming language & tools, but too specific E.g. Cycling’74 Max, oriented to audio and image processing … consequently, they hardly address issues like: Management of complexity: do not add complexity to the inherent complexity of the desiderata interactions Integration: how to integrate new and heterogeneous technologies Scalability: from few homogenous sensors to a lot of heterogeneous sensors September 23th 2009 D. Bernini, Ph.D. research status 8

  9. Research topic Filling the gap between low-level, technology oriented aspects and high-level concepts of the previous response environments What are suitable architectural abstractions for the development of responsive environments in Interactive Art & Design? (Trifonova 2008) (Edmonds et al., 2005) (Machin 2002) September 23th 2009 D. Bernini, Ph.D. research status 9

  10. What I Have Done Characterization of a thesis hypothesis Investigation of the state of the art Identification of case studies and collaborations (in progress) GAS (“Grandi Attrezzature Scientifiche”) project (UniMiB-DISCo): location aware, multimodal applications my overall collaboration: architectural abstractions for the integration of heterogeneous components they allow the establishment of communication flows based on space and time my current specific task: design and implementation of a supporting platform (in progress) September 23th 2009 D. Bernini, Ph.D. research status 10

  11. Thesis hypothesis (what) • Devising effective architectural abstractions for responsive environments in Interactive Art and Design that: • Goal 1: abstract from the specific technologies (hardware-input/output and computing devices- and software- specific programming languages & components-) • Goal 2: go beyond ad-hoc approaches, fulfillment of real-time and multimodal requirements in an effective, robust and scalable way • Goal 3: are usable by artists & designers, not only technologists • Goal 4: are applicable to other purposes, e.g. Video-surveillance, Domotics • Design and development of experimental applications based on the proposed abstractions • One for Interactive Art & Design, another for a different context D. Bernini, Ph.D. research status

  12. Thesis hypothesis (how and who) • Core approach: Software Architectures (Prof. Tisato) enhanced with other disciplinary perspectives • Ambient Intelligence • Interaction Design/Human-Computer Interaction • Collaborative work and Knowledge Management • Prof. Tisato, Prof. Archetti, Prof. Bandini, Prof. De Michelis, Prof. Schettini, Prof. Simone... • Identified case studies, collaborations and abroad period • A specific workgroup among CSAI center, DISCo-ITIS and DISCo-SAL labs • Studio Azzurro (ref. Prof. De Michelis) • GAS project • Abroad period: acquiring knowledge & skills in interactivity vs. Art & Design vs. ICT • e.g. The Mixed Reality Lab, University of Nottingham (ref. Prof. De Michelis) D. Bernini, Ph.D. research status

  13. Deployment centralized- federated highly distributed Physical environment IN DEVICES OUT DEVICES State of Art: a reference schema Different conceptual models are adopted: from Data flow models to Agents ICT System organization: from (purely) reactive, stateful/reflexive, to emergent behaviors • It does NOT imply a centralized design organization and/or deployment Responsive environment Output flow Input flow ICT System Computing & Communication D. Bernini, Ph.D. research status

  14. State of the Art: enabling technologies • Input: sensors, vision & tracking systems, tangible & wearable devices • Values of strict physical environmental properties and entity properties, a survey in (Cook and Das 2007) • Common technologies: • Wireless sensor networks, a survey in (Yick et al., 2008) • Cameras, image detection and tracking systems • Object tracking, human motion capture ad analysis, surveys in (Yilmaz et al., 2006) (Moeslund et al., 2006) • Face recognition (Colombo et al., 2007) and other systems: DISCo-Imaging and Vision Lab • Tangible interfaces and wearable devices (Harrison and Hudson 2008) • Output: lights, projectors, displays, speakers, sound synthesizers and other specific devices • Some common protocols: DMX (lights), MIDI (sound), Power Line Communication (PLC) and X10 (general electronic devices) D. Bernini, Ph.D. research status

  15. State of the Art: tools & programming languages • Arduino • “an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software” • http://www.arduino.cc/ • Cycling ’74 Max • Visual programming language, http://www.cycling74.com • System as computational graph of objects (patches) representing input/output devices, processing and data containers • Objects: self-contained program, they communicate by messages • Data flow oriented model, high extensible • MSP package: audio synthesis • Jitter package: video processing • Support for execution in local networks • Others: Pure Data (PD), vvvv, Quartz Composer • Core-model similar to MAX, but different focus D. Bernini, Ph.D. research status

  16. State of the Art: architectural abstractions • Software Components • (Heineman and Council 2001), (Shaw and Garlan 1996), component styles (Garlan 1996) (Maier and Rechtin 2000) • Specific component typologies: components with ports and connectors (Caflisch et al., 2005), (Senthil et al., 2009), targeted to responsive environments: context components (Jacquet et al., 2005), e-Gadgets (Markopoulos et al., 2004) • Time and space awareness • time (De Paoli and Tisato 1995), (Micucci et al., 2006), space (Micucci et al., 2009), GAS project • Emergent conceptual and architectural models • Multi-Agent models, Cellular Automata (CA) models, suitable for emergent and lifelike behaviors • (Bandini et al., 2008), (Silva and Vasconcelos 2007), (Sparacino et al., 2001) D. Bernini, Ph.D. research status

  17. State of the art: conceptual frameworks Time and space time, space and audience vs. interactivity (Benford et al., 2009) Multi-modality multi-modality categories (Bongers and Veer 2007) typologies of sensing (Benford et al., 2005) human movement representation and responsive environments (Loke and Robertson 2009) September 23th 2009 D. Bernini, Ph.D. research status 17

  18. Plan of Activities • Phase 1 (investigation) - 12 months • Investigation of the state of the art (completed) • Identification of case studies & collaborations (currently in progress) • Phase 2 (research) – 12 months • Devising and development of the architectural abstractions, by generalizing the analysis of the identified case studies • Phase 3 (experiment) – 8 months • Design and development of experimental applications based on the proposed abstractions • Phase 4 (research) - 4 months • Refinement of the architectural abstractions on the basis of the previous experimental applications D. Bernini, Ph.D. research status

  19. Wishful thinking • How artists/designers would like to conceive the interactivity, the audience/user participation, independently from available technologies? • From collaborations with artists/designers! • This is NOT A GOAL of the thesis D. Bernini, Ph.D. research status

  20. Target Conferences and Journals • Conferences (and related Symposia) • IEEE/IFIP Conference on Software Architecture (WICSA) and European Conference on Software Architecture (ECSA) • ACM Special Interest Group on Graphics and Interactive Techniques (SIGGRAPH) conferences • ACM’s Creativity & Cognition conferences • ACM’s International conference on Digital Interactive Media in Entertainment and Arts • ACM Special Interest Group for Computer–Human Interaction (SIGCHI) conferences • ACM International Conference on Multimedia • International Conference on Tangible and Embedded Interaction (TEI) • Journals • ACM Transactions on Computer Systems (TOCS) • Pervasive and Mobile Computing, Elsevier (PMC) • ACM Transactions on Computer-Human Interaction (TOCHI) • ACM Transactions on Multimedia, Applications, and Computing (TOMCCAP) D. Bernini, Ph.D. research status

  21. Courses • Attended courses: • “Knowledge Management: conoscenza, condivisione, tecnologia”, Prof. Simone • “Interaction Design”, Prof. De Michelis • Others • “Laboratorio di Comunicazione” (from QUASI_SI Ph.D. Program) • “Comunicazione della Scienza” (Doctoral School of Science) • “Gestione dei progetti di ricerca” (Doctoral School of Science) • English course (DISCo), ongoing • Next courses: • 2/3 internal courses • “Aspetti epistemologici dell’informatica”, DISCo, Prof. Bandini • Enabling technologies: courses of Prof. Bisiani & Schettini? • 2/1 external courses/Ph.D. schools D. Bernini, Ph.D. research status

  22. Other collaborations F.I.R.B. - Integrated Systems for Emergency project two papers related to my Master’s degree thesis Bernini, D., Toscani, D., and Frigerio, M. A software architecture for the deployment of executable transformation models. Proceedings of the 2009 International Conference on Wireless Communications and Mobile Computing: Connecting the World Wirelessly, ACM (2009), 47-51. Toscani, D., Frigerio, M., and Bernini, D. Dynamic update of data analysis models in emergency systems. Proceedings of the 2009 International Conference on Wireless Communications and Mobile Computing: Connecting the World Wirelessly, ACM (2009), 37-41. workshop participations and talks Leipzig, 21-24 June 2009, International Workshop on Advanced Topics in Mobile Computing for Emergency Management: Communication and Computing Platforms (MCEM 2009), presentation of (Bernini et al., 2009) Rome, 3-4 April 2009, InSyEme internal workshop, talk “From Application Requirements to Application Architecture” September 23th 2009 D. Bernini, Ph.D. research status 22

  23. References (1) • Overview of Responsive Environments, Interactive Art & Design • Bullivant, L. Responsive Environments: architecture, art and design. Victoria and Albert Museum, 2006. • Bullivant, L. 4dSocial: Interactive Design Environments. John Wiley & Sons, 2007. • Castle, H. Editorial. Architectural Design 77, 4 (2007), 4-5. • Elrod, S., Hall, G., Costanza, R., Dixon, M., and Rivières, J.D. Responsive office environments. Commun. ACM 36, 7 (1993), 84-85. • Edmonds, E.A., Weakley, A., Candy, L., Fell, M., Knott, R., and Pauletto, S. The studio as laboratory: combining creative practice and digital technology research. Int. J. Hum.-Comput. Stud. 63, 4-5 (2005), 452-481. • Machin, C.H.C. Digital Artworks: Bridging the Technology Gap. Proceedings of the 20th UK conference on Eurographics, IEEE Computer Society (2002), 16. • Trifonova, A., Jaccheri, L., and Bergaust, K. Software engineering issues in interactive installation art. International Journal of Arts and Technology 1, 1 (2008), 43-65. D. Bernini, Ph.D. research status

  24. References (2) • Enabling technologies, tools & programming languages • Bestor, C. MAX as an overall control mechanism for multidiscipline installation art. Computers & Mathematics with Applications 32, 1 (1996), 11-16. • Cook, D.J. and Das, S.K. How smart are our environments? An updated look at the state of the art. Pervasive Mob. Comput. 3, 2 (2007), 53-73. • Colombo, A., Cusano, C., and Schettini, R. Face^3 a 2D+3D Robust Face Recognition System. Proceedings of the 14th International Conference on Image Analysis and Processing, IEEE Computer Society (2007), 393-398. • Harrison, C. and Hudson, S.E. Scratch input: creating large, inexpensive, unpowered and mobile finger input surfaces. Proceedings of the 21st annual ACM symposium on User interface software and technology, ACM (2008), 205-208. • Pinhanez, C.S. and Bobick, A.F. Interval scripts: a programming paradigm for interactive environments and agents. Personal Ubiquitous Comput. 7, 1 (2003), 1-21. • Moeslund, T.B., Hilton, A., and Krüger, V. A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104, 2 (2006), 90-126. • Yick, J., Mukherjee, B., and Ghosal, D. Wireless sensor network survey. Comput. Netw. 52, 12 (2008), 2292-2330. • Yilmaz, A., Javed, O., and Shah, M. Object tracking: A survey. ACM Comput. Surv. 38, 4 (2006), 13. D. Bernini, Ph.D. research status

  25. References (3) Software architectures and architectural abstractions Caflisch, L., Savigni, A., Schettini, R., and Tisato, F. A software architecture for real-time, embedded monitoring systems. Advanced Video and Signal Based Surveillance, 2005. AVSS 2005. IEEE Conference on, (2005), 540-545. De Paoli, F. and Tisato, F. Architectural Abstractions for Real-Time Software. Proceedings of the Second Asia Pacific Software Engineering Conference, IEEE Computer Society (1995), 199. Heineman, G.T. e Councill, W.T., cur. Component-based software engineering: putting the pieces together. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2001. Garlan, D. and Shaw, M. An Introduction to Software Architecture. Carnegie Mellon University, 1994. Jacquet, C., Bourda, Y., and Bellik, Y. An architecture for ambient computing. Intelligent Environments, 2005. The IEE International Workshop on (Ref. No. 2005/11059), (2005), 47-54. Markopoulos, P., Mavrommati, I., and Kameas, A. End-User Configuration of Ambient Intelligence Environments: Feasibility from a User Perspective. In Ambient Intelligence. 2004, 243-254. Maier, M.W. and Rechtin, E. The art of systems architecting (2nd ed.). CRC Press, Inc., 2000. Micucci, D., Oldani, M. and Tisato, F. Time-Aware Multi Agent Systems. In Proocedings of Multiagent Systems and Software Architecture (MASSA). D. Weyns and T. Holvoet (Eds.), 2006, 71-78. Micucci, D., Tisato, F., and Adorni, M. Engineering spatial concepts. Knowl. Eng. Rev. 24, 1 (2009), 77-93. Senthil, R., Kushwaha, D.S., and Misra, A.K. An improved component model for component based software engineering. SIGSOFT Softw. Eng. Notes 32, 4 (2007), 9. Shaw, M. e Garlan, D. Software architecture: perspectives on an emerging discipline. Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1996. September 23th 2009 D. Bernini, Ph.D. research status 25

  26. References (4) Emergent conceptual and architectural models Bandini, S., Bonomi, A., Vizzari, G., et al. A CA-Based Approach to Self-Organized Adaptive Environments: The Case of an Illumination Facility. Proceedings of the 2008 Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems Workshops - Volume 00, IEEE Computer Society (2008), 1-6. Silva, F.S.C.D. e Vasconcelos, W.W. MANAGING RESPONSIVE ENVIRONMENTS WITH SOFTWARE AGENTS. Appl. Artif. Intell. 21, 4-5 (2007), 469-488. Sparacino, F., Davenport, G., and Pentland, A. Media in performance: interactive spaces for dance, theater, circus, and museum exhibits. IBM Syst. J. 39, 3-4 (2000), 510, 479. Conceptual frameworks Benford, S., Giannachi, G., Koleva, B., and Rodden, T. From interaction to trajectories: designing coherent journeys through user experiences. Proceedings of the 27th international conference on Human factors in computing systems, ACM (2009), 709-718. Benford, S., Schnädelbach, H., Koleva, B., et al. Expected, sensed, and desired: A framework for designing sensing-based interaction. ACM Trans. Comput.-Hum. Interact. 12, 1 (2005), 3-30. Bongers, B. and Veer, G.C. Towards a Multimodal Interaction Space: categorisation and applications. Personal Ubiquitous Comput. 11, 8 (2007), 609-619. Loke, L. and Robertson, T. Design representations of moving bodies for interactive, motion-sensing spaces. Int. J. Hum.-Comput. Stud. 67, 4 (2009), 394-410. September 23th 2009 D. Bernini, Ph.D. research status 26

More Related