Nature-inspired Design

1. Lecture 11: Music Nature-inspired Design Thomas Miconi* http://www.cs.bham.ac.uk/~adc/nid.html *Original authors: Julian Miller, Xin Yao, Thomas Miconi

2. Sources • Variations: Composing with Genetic Algorithms, Bruce Jacob • GenJam: A Genetic Algorithm for Generating Jazz Solos, John A. Biles • Living Melodies, Palle Dahlstedt and Mats Nordahl • Cellular Automata for Music Generation, Eduardo Reck Miranda

3. Nature Inspired Design • 1: Reducing design to a search problem: well-defined solution space (should be small and possess exploitable regularities - use a priori knowledge) with a fitness function (provided by the task at hand) • 2: Using a nature-inspired method to search through this space

4. How to generate music ? • Music is a very ill-defined problem: the fitness function is “interest to humans” • Many rules, but no precise criteria for music “quality” • OTOH, possibility to exploit human tendency to see (and hear) patterns • Works best in either highly restricted problems (choral harmonisation) or in free, open styles (jazz, modern music)

5. 2 Approaches • 1: Generate complex data, transform it through complex operations, evaluate the result (interactively or automatically) and refine the output (real search problem) • 2: Generate complex data, transform it through complex operators, and hope for the best (exploitation of human “pattern sense”)

6. Algorithmic music generation • Mozart's “dice game” (1793): randomly rearrange pre-written pieces of music according to dice throws • Hiller and Isaacson's Illiac Suite (1957): First computer composition • Many possibilities (extracting patterns from existing music, using data from chaotic systems, applying known rules of music composition, etc.)

7. Evolving Music: Variations • Bruce Jacob's Variations (1994 - present) • 3 modules: • Composer (hard-coded) generates individual motifs and phrases, using complex variation methods, without much attention to musicality • Ear assesses the musical suitability of Composer production, accepting or rejecting new motives • Arranger arranges phrases into larger movements

8. Evolving music: Variations • Ear module: defines acceptable chords and acceptable transitions between chords. • Evolved interactively, based on “concordance of opinion” with a human operator over a set of fixed music motives • Each 12-bits gene represents an acceptable chord, each pair of adjacent genes in the genome represent an acceptable transition

9. Results • Produces interesting, though rather intellectual, music • Compositions are largely unconstrained, but musicality is maintained • Results could be compared with contemplative forms of contemporary music (e.g. Duruflé)

10. Evolving jazz solos: GenJam • John “Al” Biles' GenJam (1993-present) generates jazz solo lines to be played on pre-defined chord progressions • Two types of individuals: measures and phrases (groups of measures) • Evolution occurs interactively; human operator evaluates the music as it is being played, entering 'g' or 'b' (good / bad)

11. Chromosome representation 0: rest 1-14: new notes (pitch) 15: hold Fitness

12. From numbers to notes: Chord/Scale Maps • For any song, the user must provide a Chord Progression file (1 chord for each half-measure) • A chord/scale map indicates acceptable notes for each chord • Pitch values are translated into actual notes by looking up the pitch value in this map • A given solo may thus be adapted to any song / chord progression file

13. Evolution • Human fitness function: 'g' or 'b', applies both to current phrase and current measure (with a small 'lag') --> joint evolution of phrases and measures • Generational GA with 50% survival, crossover and complex mutation operators • Ensemble evolution: evolving the whole population, not just one single measure/phrase

14. Mutation operators

15. “Jamming” mode • “Chasing chorus” with a human player: take turns, develop variations on each other's ideas • GenJam listens to the human player, applies mutation operators to it and plays the resulting melody • No evaluation (and thus no evolution); dependent on the quality of mutation operators

16. Results • Produces interesting, enjoyable music • Fluid musical progression, despite occasional “un-natural” passages • Obviously dependent on restrictions in playable notes and on the “cleverness” of genetic operators (GenJam can “improvise” without evolution at all)

17. Music evolving: Living Melodieshttp://www.ituniv.se/~palle/projects/living-melodies/index.html • Artificial world containing coevolving communicating creatures. The sounds produced by the entire system defines the musical material • Music is emergent from rules defined by authors – no human evaluation • Creatures in their world need to make music to find mates

18. Results • Results show some coherence (not totally chaotic, balance of randomness and patterns) • Alternance of “moods” • Some (few) passages could be mistaken for actual man-made music • Obviously dependent on transcriptional choices (notes, volume, tempo...) • Cross between Shostakovich and John Cage

19. Environment • Grid world, toroidal or enclosed • Creatures can walk, rest, turn (left/right), sing • Lattice word contain food, creatures must eat to compensate energy cost of living, moving, singing (total energy is kept constant) • Neighbouring creatures reproduce if they have enough energy and enough “listening pleasure”

20. Genomes • Creatures have two genomes: a procedural genome (what to do) and a sound genome (what sounds do I like) • Procedural genome: programme composed of actions, looping and branching (IF) instruction • Sound genome : list of creatures' favourite notes, in descending order of pleasure

21. Procedural Genome • Describes both creature movement and sound production • Instruction set: • Walk, Turn, Rest, Sing, Loop, IF (loops and IF can only nest once), plus macro “Turn and follow sound if listening pleasure sufficient” • IF statements can compare values of a number of inputs (neighbouring creatures, energy, age, listening pleasure, listening direction...)

22. Sound Genome • Sound genome gives the list of notes that this creature likes, in descending order • Listening_pleasurenew= Listening_pleasureold * k + ((L-P+1)/L)2, where k < 1, L is length of sound genome and P position of currently heard sound in the genome (if found) • First note of sound genome is sound played on a SING instruction

23. Lots of parameters

24. Sound mapping • Instruction SING in procedural genome leads to singing the first note in sound genome • The note can be interpreted as an absolute note, or an interval chosen from C# (above middle C) to C • Intervals can be defined in two modes, global interval mode or local mode – direction given by current direction of motion • Volume of note usually equal to current listening pleasure of singing creature

25. Music with Cellular Automata • CAMUS (Cellular Automata for MUSic generation – Miranda, 1999-present) • Uses the Game of Life to generate music • X,Y coordinates of each active cell define a triad of three notes (two intervals – origin of fundamental note is not mentioned !) • State of neighbour cells is used to obtain a rythmic pattern for the triad

26. Chord generation

27. Results • Surprisingly coherent (though amount of human “fiddling” in rythm and orchestration unclear) • Has been performed in public – audience has been 'tricked' • Lack of documentation (Choice of basic note for each triad ? Time progression ?)

28. MCLD Evolutionary Sound • Interactive evolution of rythmic patterns • Each individual corresponds to a sound • Sounds are played in loop • The user guides evolution in real time • Very little documentation available

29. See Also... • Tim Blackwell's Swarm Music (using data from a particle swarm to generate music – very chaotic) • Isle Ex system (using demographics data from a CA to generate sounds – interesting) • Evolutionary Music Bibliography : Many links and references http://www.it.rit.edu/~jab/EvoMusic/EvoMusBib.html