melodic features and retrieval l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Melodic Features and Retrieval PowerPoint Presentation
Download Presentation
Melodic Features and Retrieval

Loading in 2 Seconds...

play fullscreen
1 / 30

Melodic Features and Retrieval - PowerPoint PPT Presentation


  • 145 Views
  • Uploaded on

Melodic Features and Retrieval. ISMIR Graduate School, Barcelona 2004 Musicology 3-4 Frans Wiering, ICS, Utrecht University. Outline. yesterday’s assignment demo: MIR outside academia (7:20; 44:10) one-dimensional melody retrieval Gestalt view of melody advanced melody retrieval

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Melodic Features and Retrieval' - thurman


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
melodic features and retrieval

Melodic Features and Retrieval

ISMIR Graduate School, Barcelona 2004

Musicology 3-4

Frans Wiering, ICS, Utrecht University

outline
Outline
  • yesterday’s assignment
  • demo: MIR outside academia (7:20; 44:10)
  • one-dimensional melody retrieval
  • Gestalt view of melody
  • advanced melody retrieval
  • assignment
one dimensional melody retrieval
one-dimensional melody retrieval
  • common assumption is (was?) pitch-only retrieval is sufficient
    • e.g. CCGGAAGGFFEEDDEC
    • mechanisms for fuzzy matching
  • variants
    • interval (distance between 2 pitches)
    • pitch-contour
      • same/up/down (Parson’s Code)
      • RURURDRDRDRDRUD
  • examples:
    • www.musipedia.com (Rainer Typke)
    • www.themefinder.org (CCARH)
results from musipedia
Results from Musipedia
  • query is ranked 3
  • other hits are very unlikely
    • unfortunately no notation/sound available
  • Haydn: evident false positive
    • why?
themefinder
Themefinder
  • Several 1-dimensional search options, e.g.
    • pitch
    • interval
    • contour
    • rhythm
  • wildcards
  • each theme stored as a number of strings
  • matching by regular expressions
  • ca. 40.000 themes
    • Barlow and Morgenstern (1948)
    • ESAC encodings
    • Lincoln, 16th Century Motet (DARMS project)
results from themefinder
results from Themefinder

Query: +m2 +M2 P1 -M2 -m2 -M2

  • Example from Byrd & Crawford (2001)
  • other hits
    • not as far-fetched as musipedia’s
    • different rhythm
    • different meter
    • still not very similar
  • is this what people have in mind?
nice one we ve just discovered
Nice one we’ve just discovered
  • www.tuneteller.com
  • Pitch-only search of MIDI on the internet
  • many more MIR systems in Rainer Typke’s survey. URL is in your mailbox
why pitch only retrieval is unsatisfactory
Why pitch-only retrieval is unsatisfactory
  • information contribution of other 3 parameters (estimate for Western music; Byrd & Crawford 2001)
    • pitch: 50%
    • rhythm: 40%
    • timbre + dynamics: 10%
  • melodic confounds (Selfridge-Field 1998):
    • rests
    • repeated notes
    • grace notes, ornamentation
    • Mozart example
why pitch only retrieval is unsatisfactory9
Why pitch-only retrieval is unsatisfactory
  • information contribution of other 3 parameters (estimate for Western music; Byrd & Crawford 2001)
    • pitch: 50%
    • rhythm: 40%
    • timbre + dynamics: 10%
  • melodic confounds (Selfridge-Field 1998):
    • rests
    • repeated notes
    • grace notes, ornamentation
    • Mozart example
gestalt and melody
Gestalt and melody
  • melody: coherent succession of pitches
    • from New Harvard Dictionary of Music
  • coherence important for similarity: creates musical meaning
    • bottom-up (pitches and durations)
    • top-down: segmenting, Gestalt
  • Gestalt theory of perception
    • late 19th/early 20th century, Germany, later US
    • perception of wholes rather than parts
    • explanations: Gestalt principles of grouping
    • application in visual and musical domain
low level gestalt principles
Low-level Gestalt principles
  • Snyder mentions:
    • proximity
      • rhythmic
      • intervallic
    • similarity
      • duration
      • articulation
    • continuity
      • melodic
  • these produce closure of wholes
  • Example: Beethoven 5th symphony: beginning 1st movement
    • also illustrates high-level principles

from Snyder (2001)

low level gestalt principles12
Low-level Gestalt principles
  • Snyder mentions:
    • proximity
      • rhythmic
      • intervallic
    • similarity
      • duration
      • articulation
    • continuity
      • melodic
  • these produce closure of wholes
  • Example: Beethoven
    • also illustrates high-level principles

from Snyder (2001)

high level gestalt principles
High-level Gestalt principles
  • parallellism
    • very strong in Mozart, Ah vous, second half of melody
  • intensification
    • important organisational principle in variations and improvisations
    • Mozart’s last variation

from Snyder (2001)

application in analysis and retrieval
Application in analysis and retrieval
  • Gestalt reduces memory overload: we can ignore the details
  • Analytical: Schering (1911)
    • 14th century Italian songs
    • basic melodic shape
    • might be nice for retrieval
  • Problem with Gestalt principles:
    • many different formulations
    • overlap; no rules for conflict
    • intuitive, cannot be successfully formalized

from New Grove, Music analysis

the cognitive interpretation chunking
The cognitive interpretation: chunking
  • what creates a boundary
    • interval leap
    • long duration
    • tonality (stable chords)
    • etc
  • Example of quantification: Melucci & Orio (2004)
    • using local boundary detection (Cambouropoulos 1997)
      • apply weight to intervals and durations
      • boundary after maximum
    • chunks forther processed for indexing
organising chunks
Organising chunks
  • STM problem: max. 5-7 different elements
    • very short span
  • solution: hierarchical grouping
  • melody schemas
    • contours of melody
      • cf. Schering ex.
    • examples: axial, arch, gap-fill
    • Mozart begins with gap-fill
  • next level: form
    • A-B-A

from Snyder (2001)

mental model of a song

Ah, vous dirai-je maman

melody level

analysis

synthesis

phrase level

A

A

B

chunk level

subchunk level

mental model of a song
  • analysis: from ear to LTM
    • (sub) chunks created by similarity and continuity
      • a lot of parallellism
    • boundaries by leaps and harmony
      • chunks may have a harmonic aspect too (I, V, V->I)
  • synthesis: from LTM to focus of attention
    • recollection
      • using general characteristics of phrases and chunks
    • performance
      • notes are reconstitued through some musical grammar
problems of melody retrieval
Problems of melody retrieval
  • People remember high-level concepts, not notes
    • often confused with poor performance abilities
    • theme-intensive music (fugues) stimulate formation of such concepts
  • melodic variability and change
    • transposition
    • augmentation/diminution
    • ornamentation
    • variation
    • compositional processes: inversion, retrograde
  • other factors
    • polyphony
    • harmony
set based approaches to melody retrieval in polyphony
Set-based approaches to melody retrieval in polyphony
  • General idea:
    • compare note sets: find supersets, calculate distance
    • usually take rhythm and pitch into account
    • hopefully more tolerant agains some of the problems of melodic variety
  • Clausen, Engelbrecht, Meyer, Schmidt (2000):
    • PROMS
    • matches onset times; wildcards
    • elegant indexing
  • Lemström, Mäkinen, Ukkonen, Turkia (several articles, 2003-4)
    • C-Brahms
    • algorithms for matching line segments
      • P1: onsets
      • P2: partial match onset times
      • P3: common shared time
    • attention to time complexity
  • Typke, Veltkamp, Wiering (2003-2004)
    • Orpheus system
earth mover s distance
Earth Mover’s Distance
  • The Earth Mover’s Distance (EMD) measures similarity by calculating a minimum flow that would match two set of weighted points. One set emits weight, the other one receives weight
  • Y. Rubner (1998); S. Cohen (1999)
application to music
Application to music
  • represent notes as weighted point sets in 2-dimensional space (pitch, time)
  • weight represents duration
    • other possibilities contour/metric position etc
  • other possible application:

pitch event + acoustic feature(s)?

here, the ‘earth’ is only moved along the temporal axis

another example
Another example
  • interesting properties
    • tolerant against melodic confounds
    • suitable for polyphony
    • continuous
    • partial matching
  • disadvantage
    • triangle inequality doesn’t hold
    • less suitable for indexing:

after alignment, the ‘earth’ is moved both along the temporal axis and along the pitch axis

matching polyphony with the emd
Matching polyphony with the EMD
  • EMD’s partial matching property is essential
  • MIDI example used as query for RISM database
  • gross errors in playing are ironed out
proportional transportation distance ptd
Proportional Transportation Distance (PTD)
  • Giannopoulos & Veltkamp (2002)
  • EMD, weigths of sets normalised to 1
  • suitable for indexing
    • triangle inequality holds
  • no partial matching
test on rism a ii26
Test on RISM A/II
  • only hits with approximately same length
  • need 4 queries to find all known items
false positive emd
False positive (EMD)
  • problems arise when length and/or number of notes differs considerably
segmenting
Segmenting
  • overlapping segments of 6-9 consecutive notes
  • not musical units
  • search results are combined
  • better Recall-Precision averages
example of new search
Example of new search

http://teuge.labs.cs.uu.nl/Rntt.cgi/mir/mir.cgi

concluding remarks about melodic retrieval
Concluding remarks about melodic retrieval
  • lots of creativity go into melody; difficult to give rules
    • not a ‘basic musical structure’ (Temperley 2001)
  • essential to use multiple features
    • pitch, rhythm
    • harmony
  • segmentation
    • finding perceptually relevant chunks is not easy
    • finding complete melodies may be harder
    • arbitrary segments may also work
  • indexing strategies for melody
  • melodic change over time
  • several projects have tentative results for polyphony
    • gut feeling: false positives are big issue
    • notion of salience (Byrd and Crawford)