Design of software intensive installation art

1. Design of software intensive installation art Letizia Jaccheri Department of Computer and Information Science, Norwegian University of Science and Technology (NTNU) letizia@idi.ntnu.no

2. Outline • About me • Briefly software engineering and art • About you, your study program and this course • Software engineering issues in interactive installation art • Architecture: A case study • Other examples • Conclusions

3. About me (education and positions) • Master in Computer Science 1988 from the University of Pisa • Programmer from 1988 to 1990, C, C++, e-mail system • PhD 1994 from Politecnico di Torino, Software process modelling • In Norway since 1997, • professor since 2002

4. About me (research, teaching, and interests) • Software engineering • Process, quality, object orientation • Multidisciplinary • Software engineering issues in Art • Software engineering Education • Dissemination • Cuore & Computer Mondadori 2004 • Kjærelighet og Computer Tapir 2006 • www.letiziajaccheri.splinder.com and other blogs

5. Outline • About me • Briefly software engineering and art • …

6. Outline • About me • Briefly software engineering and art • About you, your study program and this course

7. About you • Use 15 min. to discuss the following issues with your mate: • What has this topic to do with: • Q1. This course ”Advanced Topics in Software Design”? • Q2. other courses, like Software engineering? • Q3. New media art and installation art in Amsterdam and Nederlands • write down some words for each question

8. Outline • About me • Briefly software engineering and art • About you, your study program and this course • Software engineering issues in interactive installation art

9. An art installation as a software system (1) software system output input Business goals: -more money -more efficient

10. An art installation as a software system (2) Art installation output input Artistic goals: -reflection -emotion

11. Software is used in Interactive Installation Art • for controlling the numerous sensors that are capturing environmental parameters; • for incorporating the captured parameters into music/light that is played by the sculpture; • for maintaining an archive of the music generated in the past; • for presenting online of the artwork, the artist, the music archive and the software itself

12. SE issues • Requirements • Architecture • Validation • Process • Tools • Open source software • SART: TOWARDS INNOVATION AT THE INTERSECTION OF SOFTWARE ENGINEERING AND ART, paper available at http://prosjekt.idi.ntnu.no/sart/

13. Requirements • Commercial (general purpose) software does not always satisfy artists’ needs • custom software development is needed • In Art projects, requirements are often difficult to capture, vague and frequently changing

14. Software architecture • Artists often require freedom to experiment without the need to learn programming in depth • additional software level or domain specific languages and tools might be a good solution • Reuse of software/components is often preferred

15. Validation • Evaluation with final users is sometimes possible only in final stage (i.e. when the artworks is deployed) • User interactions in unexpected/unpredicted ways is not always considered a bad thing by the artists

16. Process • A multidisciplinary team can involve artists (painters, composers, sculptors, etc.), constructors, hardware designers, electrical engineers, software engineers, programmers, art curators, etc. • Process models and project management • Agile method, human-centered design and evolutionary prototyping are reported to be successful • Risk assessment is important (emergent technologies)

17. Tools • Development environments and tools • CASE tools are rarely used by artists • Max/MSP, Flash (e.g. to make storyboards) are preferred

18. OSS and open source ideology • Roles in OSS projects • Communities of interest and practice • Continuation of support

19. Case study: Improsculpt architecture • Improscultp started in the late 90’s by artist Øyvind Brandtsegg • At NTNU 4 student projects (1 x 250 hours; 3 x 500 hours; 1 x 1000 hours) • Thor Arne Gald Semb and Audun Småge (2006) Software Architecture of the Algorithmic Music System ImproSculpt. Software Engineering Master Thesis

20. Research method: action research solutions & new knowledge Practice Improsculpt … Research Theory SE: QA’s ATAM 4+1 views MVC PAC Python & Csound Etc. Music: Algor. Comp, etc. knowledge needs

21. Flyndre Artists: Øyvind Brandtsegg and Nils Aas Location: StraumenInderøy, Norway

22. Improsculpt (1) Brandtsegg’s original system, as was handed to us in March 2006, consisted of a single directory of numerous files of Python source code, Csound effects and instruments, MIDI files, and audio files.

23. Lecture for Vrije University, Amsterdam, 29th October 2007

24. Improsculpt (requirements) The system must handle audio input, processing and output with very low latency. The system must run well on today’s market mid-range computers.

25. Improsculpt (requirements) The system must have a modular object-oriented design that is easy to understand. Changes made to the system should be as local as possible (little to no ramifications for other parts of the system). The system must be easily expandable with new processing modules. Csound changes should not affect ImproSculpt drastically, or vice versa

26. Improsculpt (requirements) As the system will be used in live performances, it is of great importance that the system produces no fatal errors of such nature that the program halts or crashes (i.e. failures). The system must be able to handle “wrong” or illogical usage. That is, it must be able to handle unexpected input from the user without crashing. The system must be able to catch internal errors in the software in such a manner that the system never terminates because of these. The system must have an error rate of under 1 per 100 time units, where one time unit corresponds to 10 seconds interaction with the system. An error is an action from the system that differs from the expected behavior. The errors mentioned here are non-critical, in that they do not cause the system to halt, but may produce wrong or unexpected output.

27. Quality attributes 1. Performance 2. Modifiability 3. Availability 4. Usability 5. Testability 6. Security 7. Safety

28. Exercise (15 min.) • Which are the important quality attributes for Improscultp? • Choose 1 QA and specify it

29. Improsculpt (performance) The system must handle audio input, processing and output with very low latency. The system must run well on today’s market mid-range computers.

30. Improsculpt (modifiability) The system must have a modular object-oriented design that is easy to understand. Changes made to the system should be as local as possible (little to no ramifications for other parts of the system). The system must be easily expandable with new processing modules. Csound changes should not affect ImproSculpt drastically, or vice versa

31. Improsculpt (availability) As the system will be used in live performances, it is of great importance that the system produces no fatal errors of such nature that the program halts or crashes (i.e. failures). The system must be able to handle “wrong” or illogical usage. That is, it must be able to handle unexpected input from the user without crashing. The system must be able to catch internal errors in the software in such a manner that the system never terminates because of these. The system must have an error rate of under 1 per 100 time units, where one time unit corresponds to 10 seconds interaction with the system. An error is an action from the system that differs from the expected behavior. The errors mentioned here are non-critical, in that they do not cause the system to halt, but may produce wrong or unexpected output.

34. Tests at the end of each iteration • Numer of imports • 1. iteration 73 • 2. iteration 59 • 3. iteration 46 • 4. iteration 42 • Max number files in a package • 19, 15, 10, 8 • File rename ramifications

37. Location: Blussvoll skole,Trondheim, Norway Sonic Onyx Artist: Samir M’kadmi

38. Sonic Onyx • The goal of the project is the creation of an interactive 3D sculpture that is placed in the court yard of a Secondary School in Trondheim (Norway). The sculpture is about four meters high and the diameter of the “space” is about seven meters. The globe on the top is 1.75 m. The ‘legs’ are metal and static, while the globe is changing its color and intensity. The color depends on the surrounding temperature. The intensity of the light changes according to the speed of the wind. In this way the sculpture varies constantly its appearance.

39. Sonic Onyx • Additionally, sound is played all the time, which attracts the pupils (12-14 years old) and other spectators. Although the sculpture’s appearance itself is very important in its aesthetical manifestation it provides another important functionality – the possibility to interact with its sound system. The spectators are able to go inside the sculpture to experience the 3-dimensional sound. While near the “space” the spectators are able to interact with the system through their mobile phones.

40. Sonic Onyx • The communication is via Bluetooth technology and accepts messages and different files sent by the pupils. In real-time the sent content are processed by the software system and the interaction influences the sound produced by the sculpture . This includes conversion of images and text into sound and combining it with the background music. Additionally, the sound is played by speakers integrated in the sculpture. The documents that are played are archived and played again in a random way when there is no traffic.

41. Sonic Onyx • Several issues have to be faced for the creation of the sculpture. Apart from the physical creation of the space, the hardware and software have to be designed and developed. The work includes the planning and programming of the communication technology, evaluation of the equipment that is needed and planning of the sound part.

42. Software engineering and Art • New • Other interdisciplinary fields (e.g. health, automotive) have been addressed in SE research, we focus on Art • Relevant • Contemporary Art is often highly software dependent (e.g. new media art, interactive art installations) • Emerging industries (e.g. Entertainment - game industry, music, web) often bring artists and software developers together

43. Software engineering and Art • Useful • In Education – raising awareness, increasing the tolerance, etc. • Foster Innovation and Creativity – Artists tend to use newest technologies in innovative ways • Rigor • Use existing SE theories and methods, including empirical studies • Identify particularities of the domain -> tailor the existing SE theories, methods and tools to the specific needs of Art

44. Important issues in SArt • Documentation • Architecture and Design • Maintenance • Testing • Open source and Intellectual property rights

45. THANK YOU! Letizia Jaccheri with Anna Trifonova, Salah U. Ahmed and Department of Computer and Information Science, Norwegian University of Science and Technology {trifonova, salah, letizia}@idi.ntnu.no

46. Installations and music (acknowledges) • Installations • Artist Øyvind Brandtsegg • Artist Samir M’kadmi • Music from • SuperCollider http://www.audiosynth.com/ • Csound http://csounds.com/ • Illustrations • Processing http://www.processing.org/ • …