Music notation music representation and intelligence
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

Music Notation, Music Representation, AND Intelligence PowerPoint PPT Presentation


  • 103 Views
  • Uploaded on
  • Presentation posted in: General

Music Notation, Music Representation, AND Intelligence. Donald Byrd School of Music, Indiana University 3 February 2005 minor rev. 5 February. Overview. Music representations: from abstract to concrete (notation) Try to assume just the right things: e.g., no knowledge of music or notation

Download Presentation

Music Notation, Music Representation, AND Intelligence

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Music notation music representation and intelligence

Music Notation,Music Representation,ANDIntelligence

Donald Byrd

School of Music, Indiana University

3 February 2005

minor rev. 5 February


Overview

Overview

  • Music representations: from abstract to concrete (notation)

  • Try to assume just the right things: e.g., no knowledge of music or notation

  • For motivation, focus on real-world content-based music-IR situations

  • Organization of the Talk

    I. Motivation: Why is this Important and/or Interesting?

    II. Representation and Semantics

    III. Music Notation, Representation, and Intelligence

    IV. Conclusions


You are here

You Are Here

I. Motivation: Why is this Important and/or Interesting?

II. Representation and Semantics

III. Music Notation, Representation, and Intelligence

IV. Conclusions


Audio to audio music retrieval

Audio-to-Audio Music “Retrieval”

  • “Shazam - just hit 2580 on your mobile phone and identify music”

  • Query:

  • Match:

  • Fantastically impressive to many people

  • Have they solved all the problems of music IR? No, (almost) none!

  • Reason: intended signal & match are identical => no time warping, let alone higher-level problems (perception/cognition)


Similarity scale for content based music ir

Similarity Scale for Content-Based Music IR

  • Relationship categories describing what’s in common between items whose similarity is to be evaluated (from closest to most distant)

    • For material in notation form, distinctions among (1), (2), and (3) don’t apply: it’s just “Same music, arrangement”

1. Same music, arrangement, performance, & recording (Shazam)

2. Same music, arrangement, performance; different recording

3. Same music, arrangement; different performance, recording

4. Same music, different arrangement; or different but closely-related music, e.g., simpler variations (Mozart, etc.), minor revs. (OMRAS, etc.)

5. Different & less closely-related music: freer variations (Schumann, etc.), extensive revisions (AI)

6. Music in same genre, etc. (AI?)

7. Music influenced by other music (AI!)


Omras polyphonic audio music ir a task that needs note representation

OMRAS Polyphonic Audio Music IR: A Task that Needs Note Representation

  • Started with recordings of Bach preludes and fugues

  • Did polyphonic (several notes at once) music recognition

    • Polyphonic audio -> events is an open research problem

    • Converted results to MIDI, used as queries against database of c. 3000 pieces in MIDI form

    • One of worst-sounding cases: Prelude in G Major from Well-Tempered Clavier, Book I

    • Outcome: the actual piece was ranked 1st!

  • Models built from notation database, but note data only

  • Query (audio -> MIDI -> audio)

  • Match (original audio recording)


Basic representations of music audio

Basic Representations of Music & Audio

Audio (e.g., CD, MP3): like speech

Time-stamped Events (e.g., MIDI file): like unformatted text

Music Notation (sheet music): like HTML text


Basic representations of music audio1

Basic Representations of Music & Audio

AudioTime-stamped EventsMusic Notation

Common examplesCD, MP3 fileStandard MIDI FileSheet music

UnitSample Event Note, clef, lyric, etc.

Explicit structurenone little (partial voicing much (complete

information) voicing information)

  • Converting to form with less explicit structure (to left): moderately difficult

  • Converting to form with more explicit structure (to right): very difficult

  • MIDI = Musical Instrument Digital Interface: simple, very standard low-bandwidth protocol (from early 1980’s)


Music ir problems that needs more structure

Music-IR Problems that Needs More Structure

  • Joan Public’s problem: find a song, given some of the melody and some lyrics

    • Needs notes and text (lyrics)

    • Common question for music librarians, esp. in public libraries

  • Musicologist’s problem: authorship/origin of works in manuscripts

    • Full symbolic data is important, even “insignificant” details of notation (John Howard)


You are here1

You Are Here

I. Motivation: Why is this Important and/or Interesting?

II. Representation and Semantics

III. Music Notation, Representation, and Intelligence

IV. Conclusions


Representation from abstract to concrete

Representation, from Abstract to Concrete

  • Cf. Basic Representations of Music & Audio

  • Abstract: represention: semantics only

  • Intermediate: syntax (mapping rules)?

  • Concrete

    • for use by computers: encoding

    • for use by humans: if visual, notation (involves graphics and/or typography)

  • Analogous to knowledge representation vs. data structure


Semantics in music

Semantics in Music

  • Denotation (explicit, well-defined)...

  • vs. Connotation (implicit, ill-defined)

  • In text

    • Two “definitions” of pig:

      • 1. Ugh! Dirty, evil-smelling creatures, wallowing in filthy sties! (Hayakawa)

      • 2. Mammal with short legs, cloven hoofs, bristly hair, and a cartilaginous snout used for digging (Amer. Heritage)

    • Prose is “mostly” denotation

    • Poetry is art => connotation much more important

  • Music is always art, & only connotation!

  • Major issue for content-based music IR


From representation to notation

From Representation to Notation

  • Choosing a representation inevitably introduces bias

  • Given a representation, choosing notation inevitably introduces more bias

  • Important to consider the purpose (R. Davis et al; Wiggins et al)

  • For huge body of important music, we have no choice: notation is CMN (Conventional Music Notation)!

    • Really “CWMN” (W = Western)

    • Alternative for some music: tablature (guitar, lute, etc.)

    • CMN is among the most successful notations ever...

    • but enormously complex and subtle


Notation says much about representation

Notation Says Much about Representation

  • CMN standard for Western music after c.1650

  • Evolved for “classical” music, but heavily used for very wide range (pop, jazz, folk, etc.)

  • Composers/arrangers/transcribers have pushed it hard => reveals things about music representation in general

  • Will concentrate on notation (CMN)


You are here2

You Are Here

I. Motivation: Why is this Important and/or Interesting?

II. Representation and Semantics

III. Music Notation, Representation, and Intelligence

IV. Conclusions


How to read music cmn without really trying the basics

How to Read Music (CMN) Without Really Trying: The Basics

  • Four basic parameters of a musical note

    1. Pitch: how high or low sound is

    2. Duration: how long the note lasts

    3. Loudness: perceptual analog of amplitude

    4. Timbre or tone quality

  • Above in decreasing order of importance for most Western music

  • Principles of CMN (& e 1)

    1. Pitch on vertical axis: clef gives offset (“zero”)

    2a. Duration indicated by note/rest shapes

    2b. Start times (sum of durations in the voice) on horizontal axis

    3. Loudness indicated by signs like p , mf , etc.

    4. Timbre indicated with words like “violin”, “horn”, “pizzicato”


Why is musical information hard to handle

Why is Musical Information Hard to Handle?

1. Units of meaning: not clear anything in music is analogous to words (all representations)

2.Polyphony: “parallel” independent voices, something like characters in a play (all representations)

3.Recognizing notes (audio only)

4.Other reasons


Units of meaning problem 1

Units of Meaning (Problem 1)

  • Not clear anything in music is analogous to words

    • No explicit delimiters (like Chinese)

    • Experts don’t agree on “word” boundaries (unlike Chinese)

  • Are notes like words?

  • No. Relative, not absolute, pitch is important

  • Are pitch intervals like words?

  • No. They’re too low level: more like characters

  • Are pitch-interval sequences like words?

  • In some ways, but

    • Ignores note durations

    • Ignores relationships between voices (harmony)

    • Probably little correlation with semantics


Independent voices in music problem 2 e 2

Independent Voices in Music (Problem 2) (& e 2)

J.S. Bach: “St. Anne” Fugue, beginning


Independent voices in text

Independent Voices in Text

MARLENE. What I fancy is a rare steak. Gret?

ISABELLA. I am of course a member of the / Church of England.*

GRET. Potatoes.

MARLENE. *I haven’t been to church for years. / I like Christmas carols.

ISABELLA. Good works matter more than church attendance.

--Caryl Churchill: “Top Girls” (1982), Act 1, Scene 1

Performance (time goes from left to right):

M: What I fancy is a rare steak. Gret? I haven’t been...

I: I am of course a member of the Church of England.

G:Potatoes.


Complex notation multiple voices e 3

Complex Notation: Multiple Voices (& e 3)

  • Multiple voices on a staff rapidly gets worse with more than 2 (Telemann “Liebe, Liebe”):

  • 2 voices in mm. 5-6: not bad: stem direction is enough

  • 3 voices in m. 7: notes must move sideways

  • 4 voices in m. 8: almost unreadable—without color!

  • Still acceptable because specific voice is rarely important


Problems example 1 superficial but interesting

Problems: Example 1 (superficial but interesting)

  • Ravel work has slur with 7 inflection points

  • Impressive, but complexity is purely graphical

  • No big deal in terms of representation

  • …but influence of performance on notation is revealing


Duration and higher level concepts of time

Duration and Higher-Level Concepts of Time

  • Schubert Impromptu(& e 4)

  • Measures: everything between barlines

  • Time signature: 3/4 = 3 quarter notes per measure

  • Triplets: 3 notes in the time normally used by 2

    • General concept is tuplets


Problems example 2 deep

Problems: Example 2 (Deep)

  • Chopin Nocturne has nasty situation (& e 5)

  • One notehead is triplet in one voice, but normal duration in another

  • “Semantics” (execution) well-defined, obvious

    • Note starts 1/16 before barline…

    • But also (2/3)*(1/16) before barline! How to play?

  • Reason: musical necessity

  • Solution for performer: “rubato”

  • Solution for music IR program: ?


Problems example 3 medium

Problems: Example 3 (Medium)

  • Bach: time signature change in middle of measure

  • (& e 6)

  • “Semantics” well-defined and obvious

    • Measure has duration of 18 16ths…

    • But not until the middle of the measure!

  • How does this make sense?

  • Triplets express same relationship as equivalent simple/compound meter

  • Invisible (unmarked) triplets

  • Cf. Bach Prelude: two time signatures at once (& e 7)

  • Reason: avoid clutter


Problem 4 medium

Problem 4 (Medium)

  • Brahms Capriccio (& e 8)

  • Time signature 6/8 => measure lasts 12 16ths

  • A dotted half note always lasts 12 16ths…

  • but here it clearly lasts only 11 16ths!

  • Reason: avoid clutter


Two ways to have two clefs at once

Two Ways to Have Two Clefs at Once

  • Clef gives vertical offset to determine pitch

  • Debussy (& e 9)

    • Bizarrely obvious something odd involving clefs

  • Ravel (& e 10)

    • Only comparing time signature (3/8) and note durations makes it clear both clefs affect whole measure

  • Reason: save space (by avoiding a 3rd staff)


Surprise music notation has meta principles 1

Surprise: Music Notation has Meta-Principles! (1)

1. Maximize readability (intelligibility)

  • Avoid clutter = “Omit Needless Symbols”

  • Try to assume just the right things for audience

  • Audience for CMN is (primarily) performers

  • General principle of any communication

    • Applies to talks as well as music notation!

  • Examples: Schubert, Bach, Brahms


Surprise music notation has meta principles 2

Surprise: Music Notation has Meta-Principles! (2)

2. Minimize space used

  • Save space => fewer page turns (helps performer); also cheaper to print (helps publisher)

  • Squeezing much music into little space is a major factor in complexity of CMN

  • Especially important for music: real-time, hands full

  • Examples: Telemann, Debussy, Ravel


The rules of music notation

The “Rules” of Music Notation

  • Tempting to assume that rules of such an elaborate & successful system as CMN work (self-consistent, reasonably unambiguous, etc.) in every case

  • But (a) “rules” evolved, with no established authority; (b) many of the “rules” are very nebulous

  • In common cases, there's no problem

  • If you try to make every rule as precise as possible, result is certainly not self-consistent

  • Trying to save space makes rules interact; something has to give!


Music notation software and intelligence

Music Notation Software and Intelligence

  • Despite odd notation, really nothing strange going on in almost all of these examples

    • Ravel slur, Debussy & Ravel 2 simultaneous clefs, Bach & Schubert invisible triplets, Brahms “short” dotted-half note, Telemann 4 voices/staff are all simple situations

    • Chopin Nocturne is complex

  • Programmers try to help users by having programs do things “automatically”

  • A good idea if software knows enough to do the right thing “almost all” the time—but no program does!

  • Notation programs convert CMN to performance (MIDI) and vice-versa => requires shallow “semantics”; makes things much harder


You are here3

You Are Here

I. Motivation: Why is this Important and/or Interesting?

II. Representation and Semantics

III. Music Notation, Representation, and Intelligence

IV. Conclusions


Conclusions review 1

Conclusions: Review (1)

  • Representations express Semantics

  • Semantics of Music; Denotation & Connotation

  • Principles of CMN

  • Meta-Principles of CMN

    1. Maximize readability; Omit Needless Symbols

    • Try to assume just the right things for audience

    • General principle of any communication

      2. Minimize space used

    • Save space => fewer page turns, less paper


Conclusions review 2

Conclusions: Review (2)

  • We need CMN or equivalent to solve spectrum of music-IR (and other music-IT) problems

    • But CMN can’t represent everything we want

    • Even when it can, may not, at least explicitly

    • Need high-level intelligence to interpret

    • Solution: unknown

    • Likely to require major funding :-)


Conclusions why is this really important and or interesting

Conclusions: Why is this really Important and/or Interesting?

  • Some problems directly related to other areas of informatics

    • Example: Approximate string matching in bioinformatics

  • Encourages progress on real semantics

    • Connotation is an important part of meaning in everything

    • Can often ignore, but any semantics in arts forces you to deal with connotation

    • Music is at least as quantifiable as any art, so likely to be more tractable than others!


You are here4

You Are Here

The End


  • Login