1 / 16

Automatic synchronisation between audio and score musical description layers

Automatic synchronisation between audio and score musical description layers. Antonello D’Aguanno, Giancarlo Vercellesi Laboratorio di Informatica Musicale Università degli Studi di Milano Speaker: Antonello D’Aguanno. Introduction.

honorato-nguyen
Download Presentation

Automatic synchronisation between audio and score musical description layers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automatic synchronisation between audio and score musical description layers Antonello D’Aguanno, Giancarlo Vercellesi Laboratorio di Informatica Musicale Università degli Studi di Milano Speaker: Antonello D’Aguanno

  2. Introduction Contemporary digital music archives consist of huge collections of heterogeneous documents The heterogeneity of music information makes retrieval hard to accomplish Synchronization is one of the open problems in Music Information Retrieval (MIR) Automatic synchronisation between audio and score musical description layers 2 /15

  3. Synchronisation Stream Audio Synchronized score Synchronization Score Synchronisation means that for a given event in a music score we can determine the timing of the corresponding audio events Automatic synchronisation between audio and score musical description layers 3 /15

  4. Related Works Many algorithms have been proposed in literature that deal with synchronisation All the algorithms proposed use MIDI score representation and PCM audio Automatic synchronisation between audio and score musical description layers 4 /15

  5. Related Works The unsolved problems of synchronisation are not limited to finding a suitable algorithm How can this result generalised to use it in other applications? No answers in literature… MX can solve the problem Automatic synchronisation between audio and score musical description layers 5 /15

  6. The linking structure in MX Notational Layer Spine Sub-Layer Performance Layer Audio Layer Automatic synchronisation between audio and score musical description layers 6 /15

  7. The COMSI algorithm Three phases: Score Analysis Audio Analysis Recursive Decisional Matching Automatic synchronisation between audio and score musical description layers 7 /15

  8. The three phases Score Analysis: The MX score is read in order to extract all relevant musical events Audio Analysis: The PCM audio signal is analysed to identify all possible musical notes Recursive Decisional Matching: Relates the event at the score level with the same event at the audio level Automatic synchronisation between audio and score musical description layers 8 /15

  9. MX score analysis Observation: COMSI Algorithm can manage conveniently even MIDI score For each measure, only the notes with strong accent are selected For every strong accent, the verticalisation of the score is computed Automatic synchronisation between audio and score musical description layers 9 /15

  10. Audio Analysis • The audio signal is filtered with a notch filter centred on the frequency of the note examined • The audio signal in time-windows is split at 100ms • For each note, a possible attack-time is the audio segment which has an energy value above a threshold, obtained from the average energy of the filtered signal Automatic synchronisation between audio and score musical description layers 10 /15

  11. Execution Events The execution events are all the audio events having an energy above the threshold The set of these execution events is named pseudo-score Automatic synchronisation between audio and score musical description layers 11 /15

  12. The Decisional Matching Measure Synchronisation Recursive research of attack-time related to the first event of each measure in the score Musical Event Synchronization Sequential research of each musical event contained in a measure Automatic synchronisation between audio and score musical description layers 12 /15

  13. Results Automatic synchronisation between audio and score musical description layers 13 /15

  14. Conclusions In this work we have described algorithm dedicated to score and audio alignment using the MX / IEEE P1599 format This algorithm allows alignment of an MX score and its execution, coded in PCM format It produces an output for the MX Spine that contains synchronisation between notes and audio signal Automatic synchronisation between audio and score musical description layers 14 /15

  15. Future works • A new test infrastructure is being developed, which will use the MX capabilities • More tests in a shorter time! • Will take into account every musical event Automatic synchronisation between audio and score musical description layers 15 /15

  16. Thanks! daguanno@dico.unimi.it Titolo della presentazione

More Related