Dynamically Generated Music Tracks

1. Dynamically Generated Music Tracks • A New Approach to the Modern Guitar Hero

2. Our Team • Michael Smith (myself) • Dylan Barrett • Karen Tipping

3. Currently • Current play along music games allow: • Instrumental controllers (guitar, drums) • Different difficulty levels • Variety of pre-made songs

4. Examples • “Guitar Hero” - Activision • “Rock Band” - Harmonix • “Frets of Fire” - Open Source Project

5. Watch

6. Music Tracks • All the current games function the same. • Tracks pre-coded by hand. • Can only play songs that have been composed. • Composed tracks are ‘perfect’.

7. Downside • Limited song selection • Song ownership

8. Alternative • Dynamically create music tracks • For song of players choice

9. Imagine A New Game • When a user plays the game: • Can play any song they own • At a desired difficulty level • Can play any instrument in song • If they have the add-on for it

10. Instrument Add-On • Each instrument fundamentally different from others. • Different handlers for different functionality. • Similar to device drivers. • Ex: Guitar add-on lets user play any guitar track found in any song

11. So.... • User will choose a song • User will choose a difficulty setting • User will choose an instrument • Game will generate a track for that song • For that instrument • At that difficulty setting

12. Project Scope • Described game very complex • Only have a single semester • Narrow down the focus to create an effective Proof of Concept demonstration • Take advantage of tools already available to save time

13. Existing Framework • Open Source “Frets of Fire” • Designed for guitar • Playable track stored as MIDI file • MIDI file basically a musical XML

14. Our Goal • Create a tool that: • Takes a Guitar only song • Analyzes the song • Outputs a playable track • In Frets of Fire format • At a selected difficulty setting

15. Data Gathering • Need location and values of musical notes • Similar to creating a music visualization • Tool called Analyze from The Echo Nest • Returns XML file with comprehensive data about a submitted song • the.echonest.com/analyze/

16. Analyze Demo • Music visualizer created by Anita Lillie of MIT using this tool shows its power.

17. Data Recording • Parse the XML to: • locate notes • find song structure (solo’s, chorus, etc...) • Create a Musical Data Structure to record the information • Linked List • Node contains Note and Timing

18. Track Generation • Range of notes need to be compressed • Need to fit range of controller • Ex: Current Guitar games have 5 notes • Not all notes should be put into track • Ex: Hendrix vs Average Player • Variations between difficulty levels

19. Difficulty Levels • Easy (least note density, more repetitive) • Find common repeated note structures • Fewest notes needed for that structure • Medium • Greater density, less repetitive notes • Hard • Solo’s, high note density, least predictable • For harder levels, add in more of

20. How to Decide • Structural analysis on the song • A Suffix Tree can find common sub-sequences/patterns of notes • Use a set of Heuristics to make decisions • Evaluate note density, timing, etc... • Heuristic values can be adjusted • Player response = better gameplay

21. First Stage Tasks • Timeline: 2 - 3 weeks • Reverse Engineer FoF MIDI file format • Use javax.sound.midi to create MIDI files • Simple music clips to understand Analyze • Create java functions to extract the data

22. Second Stage Tasks • Timeline: 3 - 4 weeks • Define Musical Structure • Create a set of plausible Heuristics • Consult with School of Music

23. Third Stage Tasks • Timeline: 1 - 2 weeks • Generate FoF formatted songs • using all previously created tools

24. Fourth Stage Tasks • Timeline: remaining time • Test code for errors • Run game to tweak heuristics for better play

25. Questions?