Composing, Producing and Implementing an Interactive Music Soundtrack for a Video Game

1. Composing, Producing and Implementing an Interactive Music Soundtrack for a Video Game Chuck Doud Music Director, SCEA

2. Introduction • Choosing a composer • Defining what you want the music to do • Relationships between team members • Planning • Creative, organizational and technical issues • Compositional techniques • Demonstration of the adaptive music

3. Disclaimer: An interactive soundtrack isn’t appropriate for every project

4. Choosing a composer As a producer you can be successful in choosing a composer who may not have the necessary experience, or being that composer be successful if: • There are people involved in the process who have experience • Time and Project budgets reflect their involvement

5. Choosing a composer (cont) • Factor a new layer into the development process • More people may need to be involved • More time and money may need to be spent

6. Choosing a composer (cont) Get the composer involved at a very early stage of the project and KEEP THEM INVOLVED Ideally at first playable No substitute for actually getting some music in the game and hearing it in context with the game and the other sound elements

7. Important Questions In addition to picking a style of music, you’ll also need to decide the role that the music is going to play Will the music be an active participant or will its function be to create mood but most effective when it draws little attention to itself? After these discussion take place you can then expand the questioning to specific applications. Mainly…. What stimuli will the music respond to? and How will it respond?

8. Planning • Come up with a plan at the very early stages of the project • Visit and re-evaluate this plan throughout the project to accommodate: • Gameplay situations evolving or changing • Level/Characters being cut or changed • Ideas that sounded good in theory may prove to not work in application • Music milestones need to be more comprehensive than just approval and delivery dates • Responsibility should be shared between the producer and the composer for maintaining and updating this plan

9. Work Closely and Update • Make sure the composer has a lot of contact with the game • Keep them up-to-date on changes that will affect the music • Composers: Be proactive about inquiring about the state of the game

10. Technical Considerations • Midi • Streamed • Combination

11. Pros and Cons Midi Pros • Extremely flexible • Can accommodate many situations • Provide a lot of variation • Extremely powerful • No substantial data bandwidth requirements

12. Pros and Cons (cont) Midi Cons • Fidelity may be compromised • Depends on available Sound Ram • Style of music • May be more time consuming to implement • Requires knowledge of specific tools • May be a limitation on number of voices available

13. Pros and Cons (cont) Multiple Streams Pros • Available Sound Ram has minimal impact (only need buffers) • No console/pc midi experience required • Style or caliber of music production isn’t compromised • Full resolution/spectrum • Full orchestral score w/live orchestra • Vocal tracks • Bands

14. Pros and Cons (cont) Multiple Streams Cons • Not as flexible or powerful as midi • Limited number of simultaneous tracks • Longer reaction time to events • Limited scope of interactivity • Not as many options available • Parts interacting • Situations being addressed • Require a data bandwidth that may not be available

15. Communication • Resource division (sfx & music) • Producer should facilitate communication • Both composers and sfx creators benefit from hearing other sound elements • Dialog may facilitate more cohesive overall sound

16. Specifics:What will the music respond to? The next step is to identify what game stimuli will influence the music. For Kri we decided on: Number of enemies Enemy proximity Player under attack Weapon status/stealth Location Special characters (bosses)

17. Specifics:How will the music respond?

18. Communication about how the music should respond to combat was between: • Composer(s) / Interactive Music Programmer and Game Director / Producer

19. Questions regarding response to combat were: • What are the combat situations like? (Hand-to-hand, style of weaponry, attack scenarios) • How often do they occur? • How many enemies will there be? (Minimum/maximum, will they attack at same time?) • How will this change throughout the game? (Number of enemies increasing, harder to kill, etc) • What do you want the music to do in response? (Be more active, drop out to only percussion, etc)

20. COMBAT TRACKS 1-2 Enemies 3+ Enemies 6+ Enemies

21. Visual of Level Music Ambient Combat

22. Ambient and Combat Segment Lengths Ambient Ambient Combat Combat Combat

23. AMBIENT TRACKS Responding to Combat Status Primary Tracks Stealth/Weapon Status Only Responds to Location Location Based(Temple chants)

24. Communication regarding the location based music was: • Composer/Interactive Programmer to Level Designer/Producer: • How often do you anticipate the player being in the temple areas? • What happens when they are there? • Do you want instantaneous feedback or can the temple music /chanting creep in and out? • Is there a limit on how long they can linger there?

25. Communication regarding the location based music was: • Composer to SFX Creator: • Are you planning anything for this situation? • If so, how can our ideas compliment each other?

26. Communication regarding the location based music was: • Composer/Interactive Programmer to Lead Engineer: • When the player enters the Temple Area(s), set Register 5 == 1 When they leave the Temple grounds set Register 5 == 0. • I’ll take care of the rest.

27. Communication Throughout The Project • Each round of questions further refines both the composition and its technical implementation. • If any of the previous criteria were to change it could have a profound effect on the adaptive score.

28. Communication Throughout The Project • What if the team decided that instead of the combat music responding to the number of enemies, they wanted the music to build based on how long the player was in a combat situation? • This could effect everything from the arrangement, interactive messaging and the composition. • Everyone shares responsibility for communicating, but as the Producer/Game Director it is ultimately your responsibility.

29. COMPOSITION Objectives: • Accommodate all the game stimuli that needs feedback • Make it musical

30. COMPOSITION Ambient tracks • No constant rhythmic parts • Variation in each instrument throughout • Avoid strong repetitive themes • Identify parts that will drop out in Stealth Mode • Identify parts that will drop out in each strata of combat music

31. COMPOSITION Combat tracks • Three different layers, each capable of playing independently of the others (and still sound good).-Need a lot of variety throughout each layer • Stagger introduction and exit of parts within each layer. • Heavy focus on rhythmic elements.

32. COMPOSITION Stealth tracks Be able to exist on their own without the other ambient tracks. Weapon tracks Work with the ambient tracks and possibly the lower strata of combat tracks.

33. COMPOSITION Overall • Be aware of harmonic elements in all the tracks to avoid conflicting parts. • Take advantage of odd time signatures (11/8, 5/4, etc). • Human ear is always looking for patterns. • Don’t have to avoid them completely, but set against a time signature other that 4/4 helps to disguise it. • Be sure that the music works independent of the game state. • Arrangement has enough variety in all possible iterations to play for a long period of time without something to trigger a change. • No combat, fighting only 1 enemy, etc.

34. COMPOSITION Correlation between the amount of interactivity built into the music and the compositional control you need to give up.

35. COMPOSITION Testing the Music • Tweak it as much as you can before you submit it to the team. • Evaluate it after it’s in the game to hear it in context and confirm registers being set. • Feedback can be given on style, dynamics and arrangements.

36. Vacuum = No Bueno

37. Silence! Please? At least sometimes?

38. Silence! Please? At least sometimes?

39. Variation • Great idea to introduce themes, and have them reoccur throughout the game or a level, but don’t hardcode them so they always appear exactly the same at regular intervals. • Add variation to the melodies, change instruments, use only partial phrases. • Apply this idea to every aspect of the music. • Experiment with odd or changing time signatures.

40. Experiment • Share ideas that work • Don’t limit yourself to what you’ve already heard.

41. Examples Ambient Combat