Rule induction for expressive music performance modeling ISMIR Graduate School - October 2004

1. ahazan@iua.upf.es Rule induction for expressive music performance modelingISMIR Graduate School - October 2004 Amaury Hazan

2. Outline • Expressive Performance Modeling • Model Descriptors • Classical ML techniques • ILP Techniques • Results • Further Work

3. Expressive Performance Modeling • A jazz musician alterates strongly the score of the excerpt he performs • Example: Body And Soul

4. Expressive Performance Modeling • For each note of a score, given a contextual description, predict expressive transformations. • 2 Aims: understand and generate

5. 2. Model Descriptors • Current Note Descriptors • Note Duration • Note Metrical Strength (within a bar) • Local Context Descriptors • Previous and next note relative duration • Previous and next note relative pitch • Melodic Context Descriptors (ILP) • Narmour I/R representation • Considering note Neighbors • Performance Related: Tempo deviation

6. Model Descriptors Duration Meter Strength

7. Model Descriptors Next note relative pitch and duration Previous note relative pitch and duration Duration Meter Strength

8. Model Descriptors: ILP Next note relative pitch and duration Previous note relative pitch and duration Duration Meter Strength Melodic Context

9. Model Descriptors: ILP Next note relative pitch and duration Previous note relative pitch and duration Duration Meter Strength Neighbors local description Melodic Context

10. Training Set • Body And Soul, Once I Loved, Like Someone in Love • 11 tempo deviations for each song • 5 slow • 1 nominal • 5 fast • Resulting Dataset: 1936 notes

11. 3. Classical ML Techniques • Classification • Naive Bayes • Lazy Learning • Induction Trees • Boosting, Bagging • Regression • Model Trees • Support Vector Machines

12. Classical ML Techniques: Target Classes • Duration Ratio, Onset Deviation, Energy • Classification • Discretization: • Duration Ratio: 9 classes • Onset Deviation: 7 classes • Energy: 3 classes • Regression

13. 4. ILP Techniques • First-order logic rule induction • Use of extensive background knowledge • One can induce either very general theories (knowledge discovery) or generative models • ILP: Set of rules and Tree-based strategies

14. ILP: Overview • Given: • Set of positive examples • Set of negative examples • Background Knowledge • Induce a set of rules: • That explains the positive examples • That does not cover the negative examples • That uses background knowledge concepts

15. ILP: Set of Rules and Tree-based Theory • Set of rules • Sequential covering of the examples • Each rule is induced independently of the others (covering loop) • Tree-based Theory • First Order generalization of induction trees • Each full path of the tree is a rule

16. Rule 2 Rule 1 Delegating Exceptions : Uncovered Examples • Positives left: No rule apply to these examples 270 examples are not covered by any rule in the model

17. Rule 2 Rule 1 Delegating Exceptions : Intersecting Rules • Rule conflicts: Intersecting rules assign a different class to the same example 128 examples are covered by more than two intersecting rules

18. ILP: Target Classes • Note to note transformations • Duration ratio with the score note (shorten, same, lengthen) • Onset deviation with the score note (advance, same, delay) • Energy of the performed note (soft, medium, loud) • Melody alterations • Ornamentation • Consolidation • None

19. 4. Results • ILP: Rules with significant coverage • Classical ML techniques • Solution synthesis

20. Results: ILP • Rules with significant coverage: S-1: [Pos cover = 12 Neg cover = 1] stretch(A,B,C,lengthen):- succ(C,D),melo(A,B,D,4,-1,-1,0,-1,1,nominal). ``Lengthen a note at an offbeat position if its successor is a quarter between two shorter notes, the former one being at the same pitch, the next one being lower, at a nominal tempo''

21. Results: ILP S-2: [Pos cover = 21 Neg cover = 1] stretch(A, B, C, shorten) :- succ(C, D), succ(D, E), context(A, E, [nargroup(p, 1)|F]), context(A, C, [nargroup(p, 2)|F]). ``Shorten a note n if it belongs to a P Narmour group in second position and if note n+2 belongs to a P Narmour group in first position''

22. Results: ILP A-1: [Pos cover = 232 Neg cover = 0] alteration(A, B, C, none) :- :- context(A, C, [nargroup(p, 2)|D]). ``Do not perform alteration of a note if it belongs to a P Narmour group in second position''

23. Results: ILP A-2: [Pos cover = 8 Neg cover = 0] alteration(A, B, C, ornamentation) :- succ(C, D), context(A, D, [nargroup(d, 2)|E]), member(nargroup(ip, 1), E), context(A, C, [nargroup(vr, 3)|F]). ``Ornamentate a note if it belongs to a VR Narmour group in third position and if its successor belongs to D Narmour group in second position and to IP Narmour group in first position''

24. Results: Classical ML techniques Duration Model

25. Results: Solution Synthesis • Melody transformation (Model Tree)

26. 5. Further Work • Fuzzy Logic • Intra note Expressive Descriptors • Solution Synthesis • Structure Level Knowledge • Players Interactions

27. Intra note Expressive Description • High Level Intra note classes e.g. : attack(A, soft). sustain(A, vibrato).

28. Solution Synthesis • Midi + controls: • Amplitude • Pitch • Audio synthesis • A/S/R construction

29. Structure Level Knowledge • Performance note description considering • Local context • Melodic context • Song structure (e.g. ABABCC)

30. Players Interactions • Considering other instruments scores • Chords/Tonality representation • Band model: Considering each player's expressive transformations