AGATE

1. AGATE Adaptive, Generative Audio Tonal Environment

2. Why do we want this? • We want music because it can support the emotional experience of the player. • But, repetitious or emotionally inappropriate music can distract the player from their actual emotional experience.

3. The Nature of audio repetition • Identical audio repetition = bad (usually). • The more traditionally “composed” music is, the less it bears repetition. • Usual solutions? • Buy MUCH music • Hope no one notices until it is too late

4. Money For Music • Music = $1,500 / minute... • Minimal coverage for 30-hour game: • 30 minutes music = $45,000

5. MMO Money For Music • MMO gameplay can easily exceed 1000 hours. • Heavy usage: 7000 hours or more! • Full coverage: all new music all the time = 420,000 minutes of music • 420,000 minutes of music =$630,000,000.

6. Forget Games, Invade A Country!

7. ANSWER:GENERATIVE MUSIC • Think of it as wind chimes: Never repeating but always familiar. • Not good for highly-structured, intensely-composed music (“cinematic”). • Very good at loosely-structured static mood music (“ambient”). • VERY well-suited to MMO game environments.

8. Adaptive Generative Audio Tonal Environment Composer Music Sounds provides Audio Output provides Create • Density • Pitch • Tempo • Randomized • Sound selection • etc. Music Data plays AGATE informs • Combat • Time of day • Weather • Location • Anything Game Data High-level script from in-house audio lead controls

9. What else? • Sequences • Keymapping • Totally awesome lasers • Etc.

10. AGATE = World Music • Good for the background; the primary voice and spirit of your game world. • Important: Vary the delivery of the music: • Fade your world music in and out. • Bring in bits of more traditionally-composed music to accentuate specific things.

11. SO EASY AND LIGHT • One programmer • One audio designer • Several weeks of less-than-100% man-hours * two people, mostly research and design-iteration • System = < 10kbytes • CPU usage is so small it is difficult to measure

12. CHRISTOPHER MAYERContract Programmer

13. Adaptive Generative Audio Tonal Environment Composer Music Sounds provides Audio Output provides create • Density • Pitch • Tempo • Randomized • Sound selection • etc. Music Data plays AGATE informs • Combat • Time of day • Weather • Location • Anything Game Data High-level script from in-house audio lead controls

14. WHERE DID THE TIME GO?

15. 1st - FMOD Events, AGATE 1, Game Hooks, Editor 2 months part time 2nd - XACT, AGATE 2, Editor 2 weeks part time 3rd - FMOD Ex, AGATE 3, Editor 5 hours

16. SAMPLE MOOD FILE Soundbank: C:\Soundbanks\TestBank.fsb Tempo: 360 Volume: .5 GuitarG3 5 64 1 Seq 1 1 -1 1 -12 12 952 ViolaG3 20 16 2 Seq .5 1 -1 1 -12 12 952 PianoG3 35 4 4 Chord .5 1 -1 1 -12 12 952 Snare 100 1 8 Chord .5 1 -1 1 -12 12 FFF

17. void main(int argc, char *argv[]) { audioInit(); agateLoadMood(argv[1]); while (!_kbhit()) { agateLoop(); audioLoop(); } agateUnloadMood(); audioExit(); }

18. void audioInit(void); void audioLoop(void); void audioExit(void); void audioLoadSoundbank(const char *file); void audioPlay(int index, float frequency, float volume, float pan); int audioGetIndex(const char *soundname);

19. typedef struct { int soundIndex; int probability; // 0 to 100 int nNotes; // 0 to ? int beat; // 1 to ? bool chord; // chord or sequence int nQueued; // 0 to nNotes float volMin, volMax; // 0.0 to +1.0 float panMin, panMax; // -1.0 to +1.0 int pitchMin, pitchMax; // -12 to +12 int pitchScale; // 12 bits int *pitchList; // cents int pitchListSize; } AGATE_SOUND; void agateLoadMood(const char *filename); void agateUnloadMood(void); void agateLoop(void);

20. void agatePlay(int i) { float frequency = (float)_mood[i].pitchList[rand() % _mood[i].pitchListSize]; float volume = randFloat(_mood[i].volMin, _mood[i].volMax); float pan = randFloat(_mood[i].panMin, _mood[i].panMax); audioPlay(_mood[i].soundIndex, frequency, volume, pan); }

21. void agateLoop(void) { for (unsigned int i=0; i<_mood.size(); i++) { if (_beat % _mood[i].beat) continue; if (rand()%100 < _mood[i].probability) if (_mood[i].chord) for (int j=0; j<(rand()%_mood[i].nNotes)+1; j++) agatePlay(i); else if (_mood[i].nQueued == 0) for (int j=0; j<(rand()%_mood[i].nNotes)+1; j++) _mood[i].nQueued++; if (_mood[i].nQueued) { _mood[i].nQueued--; agatePlay(i); } } _beat++; Sleep(60000/_tempo); }

22. Who would compose music with this? • Composer should be: • Comfortable with non-linear, non-traditional music and methods • Technically savvy (?) • Games-oriented • Familiar with ambient music • Familiar with generative music • Enjoys helping to design new technology

23. JIM HEDGESIndependent Composerand sound-designerSan Francisco, CA

24. How is this different from other forms of adaptive music? • “Vertical” approach: Layering and cross fading tracks • “Horizontal” approach: Starting and stopping cues • Often the two are combined • Both approaches tend to rely on through composed, long “stems”

25. How is this different from other forms of adaptive music? • AGATE uses smaller elements, from short clips to individual notes • It combines these elements both horizontally and vertically i.e it starts/stops and layers elements • It combines pre-determined elements with probability and randomization

26. Why use middleware intended for sound design? • This kind of music has much in common with sound design approaches and practices • Avoiding repetition and producing variety while retaining a recognizable identity • Collections of small elements, recombined at run time using randomization and probability • AGATE leverages these strengths of sound design middleware for creating music

27. Musical examples: Interlocking rhythm

28. Musical examples: Interlocking rhythm

29. Musical examples: Interlocking rhythm

30. Musical examples: Interlocking rhythm

31. Musical examples: Interlocking rhythm

32. Musical Examples: Timbre

37. Musical Examples: Melody Prime

38. Musical Examples: Melody

39. Musical Examples: Melody Prime

40. Musical Examples: Melody Up 4 Prime

41. Musical Examples: Melody Up 4 Prime Down 4

42. Musical Examples: Melody Up 4 Prime Down 4

43. Musical Examples: Stingers

44. Conclusion: Impressions • Forces composer to think primarily in terms of timbre, density and orchestration • A more “sound design” approach to composition • Learn how to work with semi-randomness • Be comfortable adapting one's approach during the compositional process

45. Conclusion: Impressions • “Each thing you add modifies the whole set of things that went before and you suddenly find yourself at a place that you couldn't possibly have conceived of, a place that's strange and curious to you. That sense of mystery, learning to live with it and make use of it, is extremely important.” - Brian Eno

46. HOW TO GET APPROVAL: EXPLAIN WHAT PROBLEMS YOUR AUDIO TECH WILL SOLVE

47. What this can do for the player and for your company: • It will make your players much less likely to turn off the music or even all the sound. • Informal poll: Everquest guild of 100+ people: 92% said they had music turned off, 74% said they had ALL sound off!

48. What this can do for the player and for your company: • It will make the game directly, viscerally pleasurable to play • Better reviews • Better word-of-mouth • Infinite minutes of music in finite storage and RAM • Large coverage with very reasonable amount of work from composer

49. What this can do for the player and for your company: • Increased sales for games with in-game purchase model. See Julian Treasure’s “Sound Business” for extensive research references.

50. AGATE: YOU LIKE IT HERE Or perhaps, “You like it hear”