Real-Time Rhythmic Transcription of MIDI Signals for Audio-Visual Applications

1. Rhythmic Transcription of MIDI Signals Carmine Casciato MUMT 611 Thursday, February 10, 2005

2. Uses of Rhythmic Transcription • Automatic scoring • Improvisation • Score following • Triggering of audio/visual components • Performance • Audio classification and retrieval • Genre classification • Ethnomusicology considerations • Sample database management

3. MIDI Signals • Unidirectional message stream at 3.125KHz • System Real Time Messages provide Timing Tick message • A simplification of acoustic signals • No noise, masking effects • Easily retrieve note onsets, offsets, velocities, pitches • However, no knowledge of acoustic properties of sound

4. Difficulties in Rhythmic Transcription • Expressive performance vs mechanical performance • Inexact performance of notes • Syncopations • Silences • Grace notes • Robustness of beat tracker • Can the tracker recover from incorrect beat induction? • Real time implementation • (Dixon 2001)

5. Human Limits of Rhythmic Perception • Two note onsets are deemed synchronous when played within 40ms of each other, 70 ms for > two notes • Piano and orchestral performances exhibit note onset asynchronicity of 30-50ms • Note onset differences of 50ms to 2s give rhythmic information • (Dixon 2001)

6. Evaluation Criteria for Beat Trackers • Informally - click track of reported beats added to signal • Visually marking the reporting beats • Comparing reported vs known, correct beats • (Dixon 2001)

7. Definitions • Beat - “perceived pulses which are approximately equally spaced and define the rate at which notes in a piece are played” • meterical, score , performance level • tempo - beats per minute • Inter-onset Intervals (IOI) - time intervals between note onsets • (Dixon 2001)

8. Approaches - Probabilistic Frameworks • Cemgil et al (2000) - Bayesian framework, using a tempogram (wavelet) and a 10th order Kalman Filter to estimate tempo, which is a hidden state variable • Takeda et al (2002) - Hidden Markov models for fluctuating note lengths and note sequences, estimating both rhythms and tempo • Raphael (2002) - tempo and rhythm

9. Approaches - Oscillators • Period and phase that adjusts itself to synchronize to IOI input • Dannenberg and Allen (1990) - weighted IOIs and credibility evaluation based on past input • Meudic (2002) - real time implementation of Dixon • Induce several beats and attempt to propagate them through the signal (agents), then choose the best • Pardo (2004) - Oscillator, compared to Cemgil using same corpus

10. Pardo 2004 - Oscillatory Design • Is a Kalman Filter (Cemgil) or oscillator better for online tempo tracking? • Performance as time series of weights, W, over T time steps • Weight of time step with no note onsets = 0, increased proportional to # of note onsets • 100ms is minimum IOI allowed, minimum beat period

11. Pardo 2004 • Uses weighted average of last 20 beat periods, with one parameter varying degrees of smoothing • A correction parameter varies how far the period and phase of the next predicted beat is changed according to known information • A window size parameter affects how many periods may affect the current prediction • Chose 5000 random values of these three parameters, ran each triplet on 99 performances of Cemgil corpora

12. Cemgil MIDI/Piano Corpora • Four pro jazz, four pro classical, three amateur piano players • Yesterday and Michelle, fast, slow and normal, captured on a Yamaha Diskclavier • Available at www.nici.kun.nl/mmm/

13. Pardo 2004 - Error Measurement (Pardo 2004) • After finding best parameters values for Michelle corpus, applied same values to analysis of Yesterday corpus • Compared to Cemgil using that paper’s defined error metric, which takes into account both phase and period errors, to come up with a score

14. Comparison of Approaches (Pardo 2004) • Oscillator somewhat better than tempogram alone, • Somewhat worse than tempogram plus Kalman, yet fall within standard deviation (bracketed numbers) of Kalman scores

15. Other Considerations • Stylistic information • Training of tracker • Musical importance of note • Duration • Pitch • Velocity

16. Bibliography • Allen, P., and R. Dannenberg. 1990. Tracking musical beats in real time. In Proceedings of the International Computer Music Conference 1990: 140–3. • Dixon, S. 2001. Automatic extraction of tempo and beat from expressive performances. Journal of New Music Research 30 (1): 39–58. • Meudic, B. 2002. A causal algorithm for beat-tracking. In Proceedings of Conference on Understanding and Creating Music. • Pardo, B. 2004. Tempo tracking with a single oscillator. In Proceedings of the International Conference on Music Information Retrieval2004. • Raphael, C. 2002. A hybrid graphical model for rhythmic parsing. Artificial Intelligence 137: 217–38. • Takeda, H., T. Nishimoto, and S. Sagayama. 2002. Automatic rhythm transcription from multiphonic MIDI signals. In Proceedings of the International Conference on Music Information Retrieval 2003.