Lyric alignment in popular songs

1. Lyric alignment in popular songs Luong Minh Thang

2. Outline • Project description & techniques • Some important knowledge • Base-line system

3. Outline • Project description & techniques • Some important knowledge • Base-line system

4. Project description • Given: • textual transcription of lyrics • acoustic musical signal of a song • Purpose: find the timestamps for the beginning and ending points for each line of the song.

5. Techniques to be investigated • A repetition based technique for detecting self-similarities in both the audio and the text • Dynamic programming (also known as dynamic time warping) will be employed to align the repetition analysis of both media together to produce an alignment

6. Outline • Project description & techniques • Some important knowledge • Base-line system

7. C C# D … B C C# D…B C C# D…B C C# D…B Tone heights 1 octaves with 12 semitones 1 octaves with 12 semitones 12-dimensional chroma vector Chroma vectors 0.3 0.5 0.1 0.7

8. Dynamic time warping algorithm • Actually, dynamic programming

9. Word similarity using phoneme dictionary • Using CMU phoneme dictionary • Each word is decomposed into phoneme sequence • ALIGNMENT: AH0 L AY1 N M AH0 N T • ALGORITHMS: AE1 L G ER0 IH2 DH AH0 M Z • Similarity of 2 words is the similarity of the 2 phoneme sequences

10. Outline • Project description & techniques • Some important knowledge • Base-line system

11. Chroma-vector calculations Text processing Simplification Phoneme dictionary Mapping Aligning Base-line system: overview Musical signal input Text input Music notation sequence Word sequence Word self-similarity matrix Symbol Self-similarity matrix

12. Chroma-vector calculations Text processing Simplification Phoneme dictionary Mapping Aligning Base-line system: overview Musical signal input Text input Music notation sequence Word sequence Word self-similarity matrix Symbol Self-similarity matrix

13. Chroma-vector calculations Text processing Simplification Phoneme dictionary Mapping Aligning Base-line system: overview Musical signal input Text input Music notation sequence Word sequence Word self-similarity matrix Symbol Self-similarity matrix

14. Chroma-vector calculations Text processing Simplification Phoneme dictionary Mapping Aligning Base-line system: overview Musical signal input Text input Music notation sequence Word sequence Word self-similarity matrix Symbol Self-similarity matrix

15. Chroma-vector calculations Text processing Simplification Phoneme dictionary Mapping Aligning Base-line system: overview Musical signal input Text input Music notation sequence Word sequence Word self-similarity matrix Symbol Self-similarity matrix

16. Thank you !