Music Research Concept Paper incl. Colour Organ

1. Music ResearchConcept Paperincl. Colour Organ Akiko Abe Graphic Design and New Media

2. Research Targets

3. Things to investigate • How minimal/conceptual music composers make music. After looking at some basic parts of music theory, I want to understand the way they do. And hopefully, want to try for myself. • Also how they are different comparing with the traditional music and composers’ context. How they broke the rules if there have been any. • Also listen to romantic music for comparison and investigate why it is romantic, as I also often work in romantic motives. • In parallel with the investigation of them, watch again the films they contribute their music to. Make notes again. It must be different to the previous impression and understanding. If time allows, try games too.

4. Things to investigate • What does it mean that music has mathematical aspects? Is my mathematical knowledge helpful to the research and making musical elements within my work? (You can’t just say it it mathematical because we do some calculation when composing time of each notes, triads or harmonies. Those are just arithmetic!) • Make a colour organ (not necessarily made like an “organ”) and enjoy playing, composing, and sharing with other people. Try to watch the reaction. • Feedback this to the project Blue. Current keywords are: • Motif: Hope, Beginning, Change • Visual: Blue, Dawn, Light, Open sky and things in the sky, Water, Slow change

5. Research Paths

6. How to proceed investigation • [R1] Reading Books Discuss with teachers for inside topics. If the book is too difficult, ask help. • Experimental Music - Cage and Beyond (Nyman) To understand the context of minimal music and major composers in the field • Sounds in Space Sounds in Time (Vella) To learn and develop musical thinking and making in a creative way • Music and Mathematics (Fauvel, Flood and Wilson) To investigate the relationship between music and mathematics • The Third Ear (Berendt) Because I am not sure if I have the third ear, though I do have the third eye • 大作曲家たちの履歴書(CVs of great composers) （三枝 Saegusa） [Japanese] To look through some major composers’ profiles for background knowledge • Point and Line to Plane (Kandinsky) Because I adore this book For others, also approach to poems. Reading voice is my favourite sound source.

7. How to proceed investigation • [R1] Case Study • Music とりあえず聴く。聴く、聴く、聴く。そして聴く!!! 耳の限界に挑戦。 • Experimental/Conceptual/Minimal music <--> In parallel with films • Nyman, Cage, Eno, Takemitsu etc. • Romantic music <--> in parallel with films • Beethoven, Wagner, Tchaikovsky, R. Strauss, Debussy, Preisner etc. • Satie, looking at Klee (Satie is cited in Jarman’s “Blue” too) • Ballet music, looking at Joseph Cornell or de Gas • Opera/Theatre music, looking at David Hockney (go theatre if possible) • Holst, reading Greek myths • Poems, reading aloud • Hymns, gospells in a church. Read a sutra. Crystal ball music while meditating • Collect environmental sounds: water, wind, something burning, breaths..

8. How to proceed investigation • [R1] Case Study(continued) • Films Check specially those in which music is powerful in evoking certain feelings to the audience or giving impact to its storytelling. As this is connected to films survey in Primary Research, I choose works from those in the research. • Nyman flicks (Greenaway films, The Piano etc.) • Preisner’s works in Keislowski films (Preisner is influenced by Sibelius) • Ennio Morricone’s works in various films • Animation works: Fischinger, McLaren etc. • Horror movies • Watching CFs, music PVs • [R1] Making notes This time, because the works vary and extensive, all the notes will be taken in one format based on my work structure model. I am now formatting this.

9. How to proceed investigation • [R2] Making Colour Organ This work is supposed to be made with Flash based on the transplant of my previous work [am] from Director’s lingo codes. Details are in later pages. This activity will be done by Easter break. (if be late, until the end of the break) • Design visual presentation • Design musical interaction • Define colour-sound-key (or other devices) mapping • Design backend architecture • Obtain the sound source • Work in actual coding • [Proj.Blue] Working on actual sound/musical elements for visual works This is the temporary goal of this research but I am not sure if I can reach, for I extended the research very much. Constantly watched by tutors and if cannot handle for myself, ask help. Saving time and scceleration by learning efficiently is the key.

10. Colour Organ Project

11. Visual PresentationAny better idea?

12. Searching for interesting visual… A little bored of such visual… wondering what should be best…

13. Searching for interesting visual… I need some twist… currently almost no idea. Please help.

14. Case Study - Piano - as Image Media (Toshio Iwai) Iwai’s work is a transformation of drawing into sound scapes, rather than a colour organ which is similar to a real musical instrument. It produces various simple-shaped figures which are coloured according to their relevant pitches sent from the console and went through the dot grid screen. As seen in the work, it stands on the paradigm that one (or more) octave(s) are distributed to rainbow colours, probably uniformly. I am personally still in wonder if colours should be shared by every pitch equally, without a bias.

15. Case Study - Hammond Flower (Amit Pitaru) This is more like a graphic equalizer. Its sound is similar to that of a hammond organ but is ambient and beautiful, depending on the user’s interaction. The user can slide each bar and create the overall texture of the organ’s sound, according to the author, Pitaru. And we can see the polygon placed in the centre of the screen changing its shape. Hammond Flower: http://www.pitaru.com/

16. Case Study - O - [am] This is also rainbow-coloured and similar in the concept as Iwai’s work. The weaving starts from the centre and spirally goes outwards. The difference from the previous two cases is that it has its trace of playing music as a woven tapestry, although it lacks information of each note’s duration. It has three tones; accordion, trumpet and piano, which are different in its colour tones in the screen. The switching key position is fixed but hidden. So is every colour-sound key. The weaving visual is made inspired by simple meias weaving and very popular Japanese girls’ toy “Lily-yarn,” which almost all Japanese around my age have seen it at least once in their childhood. リリアン(Lily-yarn) アンデミルミル (Andemirumiru) A fancy knitting product by Hamanaka, based on Lily-yarn

17. Case Study - O - [am] if the key mapping is known… If you play it by your visual curiosity, tapestries may be like them. If you play it by your musical curiosity, then tapestries are.. Each sound x 4 for one octave Edel Weiss (Accordion) Chiisana Konomi (Trumpet) Originally the aria in “La Jolie Fille de Perth” by Bizet

18. Colour and Sound

19. Historical Colour-Sound Mapping Blue does not seem like being attached to a specific colour as stably as red - C, green - F/Fsharp or magenta - A. Where is blue? Where should it be? Or it should be everywhere? Nowhere?

20. O - [am] Colour-Sound-Key Mapping • Basically mapped like a piano • Piano’s white keys are distributed onto two lower rows of the keyboard and the upper first row of keys are for piano’s black keys, which makes the touch of playing quite different to that of a real piano • Right hand keys and left hand keys are same sets and independent to each other • Tone change keys for each hand change the tone of the sound based on the toggle: • Right: Accordion -> Piano -> Trumpet -> Accordion -> … (repeat) • Left: Trumpet -> Accordion -> Piano -> Trumpet -> … (repeat) And it changes the colour tones as well (next page) Tone: Trumpet

21. O - [am] Colour-Sound-Key Mapping • This colour mapping is too blue-dominant. But the paradigm of what colour for how much has no rule originally. Tone: Accordion Tone: Piano

22. O - [am] Colour-Sound Mapping Alternative I would rather do this!

23. Musical InteractionAny better idea?

24. Key Mapping Alternatives: 2-octave Piano • I thought of this mapping when making [am], but discarded because it was not my aim to make it like a piano • The upper two rows for the right hand, the lower two for the left • It is a serious imitation of a piano keyboard but both hands will confuse each other because the space is very narrow • Can expect fast-response interaction owing to its simplicity, but still the outputting sounds will have some difficulty if too much keys are pressed at one time

25. Key Mapping Alternatives: Guitar (or String-based) • This mapping is based on how we play a guitar • Right hand is for up/down strokes or arpeggio with the base shortcut and each string: 1,2,…, 6, while left hand for bunch of chords • Need to prepare triad sounds in advance for strokes, as well as single sounds source for single string pickings, which may cause a delay in response. Especially stocks for choking should vary in combination and large in data size • Yet might be very interesting in terms of interaction design. Similar kind of toy guitar (shaped like electric guitar) exists in Japan

26. Is there any other better mapping? • Not always have to follow a piano just because it is a “keyboard” • For example, xylophone is also interesting to imitate, realising fast-changing triads • Dependent on the sound source, the instrument can be a rhythm instrument too • The instrument could be something like a environmental sounds juke box with video pieces • If there is other controllable input device, it gives more interaction space • Must remember that Flash can only have two audio tracks. In order to play more sounds, multiple sound source must be given a priori, in either/both track

27. Interesting cases beyond a keyboard bye bye Basically, development in interface using other devices costs high and needs advanced engineering knowledge and experience, therefore, it is often a collaboration with a manufacturing company. • ARS Electronica (probably similar works exist by previous nominees) http://www.aec.at/en/index.asp • TENORI-ON by Toshio Iwai and YAMAHA http://www.global.yamaha.com/design/tenori-on/ • Maywa Denki (see ART -> TSUKUBA series) http://www.maywadenki.com/ Maybe many of Music or New Media Department in various universities are doing similar projects. But I don’t know who is doing what, especially in England, outside Japan. High-IQ projects in science • Hyperinstruments project led by Tod Machover, at MIT Media Lab http://www.media.mit.edu/hyperins/ • Sony CSL (seems that they are doing musical stuff in France, but this lab. is more famous for development in intuitive user interfaces.) http://www.csl.sony.co.jp/

28. Appendix

29. Colour-Sound Mapping: Sound Tapestry (v3) Sound Tapestry is one of my past works, which shares the weaving algorithm partially and the sound set with O - [am]. It was a school project using Director, which we newly learned, and I made it with the inspiration given by early 80s video games. Tone: Accordion Tone: Piano Tone: Trumpet

30. Sound Tapestry: Visual Samples In Sound Tapestry, the player can choose instruments for each hand separately before playing, and colour weaving is done from the most left upper corner of the screen. The sound source is the same set as [am]’s. [am] has its breakthru on the cross point of a creative motivation towards a toy for celebrating children and the reflection that this sound set suits better for healthy visuals rather than the video-game atmosphere. But this time, I really want to renew the sound set.

31. Colour me any colour of you! Please give me any suggestion about this research project and if you need, please use this keyboard map.