representation of musical information l.
Skip this Video
Loading SlideShow in 5 Seconds..
Representation of Musical Information PowerPoint Presentation
Download Presentation
Representation of Musical Information

Loading in 2 Seconds...

play fullscreen
1 / 68

Representation of Musical Information - PowerPoint PPT Presentation

  • Uploaded on

Representation of Musical Information Donald Byrd School of Music Indiana University Updated 20 Feb. 2008 Copyright © 2003-08, Donald Byrd Classification: Logician General’s Warning Classification is dangerous to your understanding Almost everything in the real world is messy

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Representation of Musical Information' - adamdaniel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
representation of musical information
Representation of Musical Information

Donald Byrd

School of Music

Indiana University

Updated 20 Feb. 2008

Copyright © 2003-08, Donald Byrd

classification logician general s warning
Classification: Logician General’s Warning
  • Classification is dangerous to your understanding
    • Almost everything in the real world is messy
    • Absolute correlations between characteristics are rare
    • Example: some mammals lay eggs; some are “naked”
    • Example: is the piano a keyboard, a string, or a percussion instrument?
  • People say “an X has characteristics A, B, C…”
  • Usually mean “an X has A, & usually B, C…”
  • Leads to:
    • People who know better claiming absolute correlations
    • Arguments among experts over which characteristic is most fundamental
    • Don changing his mind

31 Jan. 07

dimensions of music representations encodings 1
Dimensions of Music Representations & Encodings (1)


(After Wiggins et al (1993). A Framework for the Evaluation of Music Representation Systems.)

rev. 20 Feb. 07

dimensions of music representations encodings 2
Dimensions of Music Representations & Encodings (2)
  • Expressive completeness
    • How much of all possible music can the representation express?
    • Includes synthesized as well as acoustic sounds!
    • Waveform (=audio) is truly “complete”
    • Exception, sort of: conceptual music
      • E.g., Tom Johnson: Celestial Music for Imaginary Trumpets (notes on 100 ledger lines), Cage: 4’ 33” (of silence), etc.
  • Structural generality
    • How much of structure in any piece of music can the representation express?
    • Music notation with repeat signs, etc. still expresses nowhere near all possible structure

rev. 31 Jan. 07

representation vs encoding
Representation vs. Encoding
  • Representation: what information is conveyed?
    • More abstract (conceptual)
    • Basic = general type of info; specific = exact type
  • Encoding: how is the information conveyed?
    • More concrete: in computer (“bits”)…or on paper (“atoms”)!)
  • One representation can have many encodings
    • “Atoms” example: music notation in printed or Braille form
    • “Bits” example: any kind of text in ASCII vs. Unicode
    • “Bits” example: formatted text in HTML, RTF, .doc

30 Jan. 06

basic representations of music audio
Basic Representations of Music & Audio

Audio (e.g., CD, MP3): like speech

Time-stamped Events (e.g., MIDI file): like unformatted text

Music Notation: like text with complex formatting

27 Jan.

basic representations of music audio7
Basic Representations of Music & Audio

Audio Time-stamped Events Music Notation

Common examples CD, MP3 file Standard MIDI File Sheet music

Unit Sample Event Note, clef, lyric, etc.

Explicit structure none little (partial voicing much (complete

information) voicing information)

Avg. rel. storage 2000 1 10

Convert to left - easy OK job: easy

Convert to right 1 note: pretty easy OK job: fairly hard -

other: hard or very hard

Ideal for music music music

bird/animal sounds

sound effects


27 Jan.

representation example a bit of mozart
Representation Example: a Bit of Mozart

The first few measures of Variation 8 of the “Twinkle” Variations

27 Jan.

in notation form nightingale notelist
In Notation Form: Nightingale Notelist
  • %%Notelist-V2 file='MozartRepresentationEx' partstaves=2 0 startmeas=193
  • C stf=1 type=3
  • C stf=2 type=10
  • K stf=1 KS=3 b
  • K stf=2 KS=3 b
  • T stf=1 num=2 denom=4
  • T stf=2 num=2 denom=4
  • A v=1 npt=1 stf=1 S1 'Variation 8'
  • D stf=1 dType=5
  • N t=0 v=1 npt=1 stf=1 dur=5 dots=0 nn=72 acc=0 eAcc=3 pDur=228 vel=55 ...... appear=1
  • R t=0 v=2 npt=1 stf=2 dur=-1 dots=0 ...... appear=1
  • N t=240 v=1 npt=1 stf=1 dur=5 dots=0 nn=74 acc=0 eAcc=3 pDur=228 vel=55 ...... appear=1
  • N t=480 v=1 npt=1 stf=1 dur=5 dots=0 nn=75 acc=0 eAcc=2 pDur=228 vel=55 ...... appear=1
  • N t=720 v=1 npt=1 stf=1 dur=5 dots=0 nn=77 acc=0 eAcc=3 pDur=228 vel=55 ...... appear=1
  • / t=960 type=1
  • N t=960 v=1 npt=1 stf=1 dur=4 dots=0 nn=79 acc=0 eAcc=3 pDur=456 vel=55 ...... appear=1
  • (etc. File size: 1862 bytes)

27 Jan.

an event form standard midi file file dump
An Event Form: Standard MIDI File (file dump)
  • 0: 4D54 6864 0000 0006 0001 0003 01E0 4D54 MThd.........‡MT
  • 16: 726B 0000 0014 00FF 5103 0B70 C000 FF58 rk......Q..p¿..X
  • 32: 0402 0218 0896 34FF 2F00 4D54 726B 0000 .....ñ4./.MTrk..
  • 48: 0055 00FF 0305 5069 616E 6F00 9048 3881 .U....Piano.êH8Å
  • 64: 6480 4840 0C90 4A38 8164 804A 400C 904B dÄH@.êJ8ÅdÄJ@.êK
  • 80: 3881 6480 4B40 0C90 4D38 8164 804D 400C 8ÅdÄK@.êM8ÅdÄM@.
  • 96: 904F 3883 4880 4F40 1890 4F38 8360 9050 êO8ÉHÄO@.êO8É`êP
  • 112: 3883 4880 4F40 1890 4D38 8330 8050 4018 8ÉHÄO@.êM8É0ÄP@.
  • 128: 804D 400D FF2F 004D 5472 6B00 0000 3200 ÄM@../.MTrk...2.
  • 144: FF03 0550 6961 6E6F 8F00 9041 2B81 6480 ...Pianoè.êA+ÅdÄ
  • 160: 4140 0C90 4330 8164 8043 400C 9044 3181 A@.êC0ÅdÄC@.êD1Å
  • 176: 6480 4440 0C90 4647 8164 8046 4001 FF2F dÄD@.êFGÅdÄF@../
  • 192: 00 .

5 Feb.

an event form standard midi file interpreted
An Event Form: Standard MIDI File (interpreted)
  • Header format=1 ntrks=3 division=480
  • Track #1 start
  • t=0 Tempo microsec/MIDI-qtr=749760
  • t=0 Time sig=2/4 MIDI-clocks/click=24 32nd-notes/24-MIDI-clocks=8
  • t=2868 Meta event, end of track
  • Track end
  • Track #2 start
  • t=0 Meta Text, type=0x03 (Sequence/Track Name) leng=5
  • Text = <Piano>
  • t=0 NOn ch=1 num=72 vel=56
  • t=228 NOff ch=1 num=72 vel=64
  • t=240 NOn ch=1 num=74 vel=56
  • t=468 NOff ch=1 num=74 vel=64
  • (etc. File size: 193 bytes)

27 Jan.

midi musical instrument digital interface 1
MIDI (Musical Instrument Digital Interface) (1)
  • Invented in early 1980’s
    • Dawn of personal computers
    • Designed as simple (& cheap to implement) real-time protocol for communication between synthesizers
    • Low bandwidth: 31.25 Kbps
  • Top bit of byte: 1 = status, 0 = data
    • Numbers usually 7 bits (range 0-127); sometimes 14 or even 21
  • Message types
    • Channel: Channel Voice, Channel Mode
    • System: System Common, System Real-Time, System Exclusive

5 Feb. 06

midi 2
MIDI (2)
  • Important standard Events are mostly Channel Voice msgs
    • Note On: channel (1-16), note number (0-127), on velocity
    • Note Off: channel, note number, off velocity
  • Can change “voice” (really patch!) any time with Program Change msg
  • A way around the 16-channel limit: cables
    • may or may not correspond to a physical cable
    • each cable supports 16 channels independent of others
    • Systems with 4 (=64 channels) or 8 cables (=128) are common
  • MIDI Monitor allows watching MIDI in real time
    • Freeware and open source!

5 Feb. 06

midi sequencers
MIDI Sequencers
  • Record, edit, & play SMFs (Standard MIDI Files)
  • Standard views
    • Piano roll
      • often with velocity, controllers, etc., in parallel
    • Event list
    • Other: Mixer, “Music notation”, etc.
    • Standard editing
  • Adding digital audio
    • Personal computers & software-development tools have gotten more & more powerful
    • => "digital audio sequencers”: audio & MIDI (stored in hybrid encodings)
  • Making results more musical: “Humanize”
    • Timing, etc. isn’t mechanical—but not really musical (yet)

8 Feb. 06

is a midi file a score or a performance
Is a MIDI File a “Score” or a Performance?
  • MIDI files are often used to encode music from notation
  • …but also often used to describe performances!
  • What’s the difference?
    • Timing
    • Dynamics
    • Realizing ornaments, etc.
  • For scores, MIDI files are very limited
    • Max. 16 explicit voices, no spelling info, no slurs, etc.
  • …though not as badly as many assume
    • Can include time sig., key sig., text/lyrics, etc.
  • Cf. “Dimensions of Music Representations & Encodings” graph

15 Feb. 07

another warning terminology 1
Another Warning: Terminology (1)
  • A perilous question: “How many voices does this synthesizer have?”
  • Syllogism
    • Careless and incorrect use of technical terms is dangerous to your learning much
    • Experts often use technical terms carelessly
    • Beginners often use technical terms incorrectly
    • Therefore, your learning much is in danger
  • Somewhat exaggerated, but only somewhat

5 Feb. 06

another warning terminology 2
Another Warning: Terminology (2)
  • Not-too-serious case: “system”
    • Confusion because both standard (common) computer term & standard (not common but useful) music term
  • Serious case: patch, program, timbre, or voice
    • Vocabulary def.: Patch: referring to event-based systems such as MIDI and most synthesizers (particularly hardware synthesizers), a setting that produces a specific timbre, perhaps with additional features. The terms "voice", "timbre", and "program" are all used for the identical concept; all have the potential to cause substantial confusion and should be avoided as much as possible
    • “Patch” is the only unambiguous term of the four
    • …but the official MIDI specification (& almost everything else) talks about “voices” (as in “Channel Voice messages control the instrument’s 16 voices”)
    • …and to change the “voice”, you use a “program change”!

6 Feb. 06

another warning terminology 3
Another Warning: Terminology (3)
  • Some terminology is just plain difficult
  • Example: “Representation” vs. “Encoding”
    • Distinction: 1st is more abstract, 2nd more concrete
    • …but what does that mean?
    • Explaining milk to a blind person: “a white liquid...”
  • Don’s precision involves being very careful with terminology, difficult or not
    • Vocabulary is important source
    • Cf. other sources
    • Contributions are welcome

6 Feb. 06

an event form standard midi file interpreted19
An Event Form: Standard MIDI File (interpreted)
  • Header format=1 ntrks=3 division=480
  • Track #1 start
  • t=0 Tempo microsec/MIDI-qtr=749760
  • t=0 Time sig=2/4 MIDI-clocks/click=24 32nd-notes/24-MIDI-clocks=8
  • t=2868 Meta event, end of track
  • Track end
  • Track #2 start
  • t=0 Meta Text, type=0x03 (Sequence/Track Name) leng=5
  • Text = <Piano>
  • t=0 NOn ch=1 num=72 vel=56
  • t=228 NOff ch=1 num=72 vel=64
  • t=240 NOn ch=1 num=74 vel=56
  • t=468 NOff ch=1 num=74 vel=64
  • (etc. File size: 193 bytes)

27 Jan.

basic and specific representations vs encodings
Basic and Specific Representations vs. Encodings

Basic and Specific Representations (above the line)

Encodings (below the line)

rev. 15 Feb.

rudiments of musical acoustics
Rudiments of Musical Acoustics
  • Need some musical acoustics for almost anything in digital audio
  • Acoustics: branch of physics that studies sound (of any kind)
    • Concepts like frequency & amplitude
    • Besides music, e.g., ultrasonic (for medicine, underwater, etc.)
  • Psychoacoustics: study of how sound is perceived; mostly psychology
    • Concepts like pitch, loudness, timbre
      • Relationship to physical concepts often roughly logarithmic
      • …but only roughly: always more complex than that

rev. 20 Feb. 08

materials for studying audio acoustics
Materials for Studying Audio & Acoustics
  • Musical instrument samples
    • What are interesting sounds really like?
      • Sine waves, etc. are boring! (cf. addsynenv)
      • Sounds of acoustic instruments are “rich”
      • Vary in every way: with pitch, loudness, time
  • Audacity audio editor
    • For Windows, Mac OS 9 and X, Linux
    • Download from
  • Sonic Visualiser is promising new alternative
  • Programs in (e.g.) R
  • Fourier applet
  • addsynenv

10 Sept. 2006

time domain frequency domain 1
Time Domain & Frequency Domain (1)
  • Time domain involves waveforms
  • Frequency domain involves spectra
  • Fourier’s Theorem
    • Any periodic signal can be described exactly as the sum of sinusoids at integral multiples of its fundamental frequency (Fourier analysis)
    • Fourier Transform takes time domain to frequency
    • Inverse Fourier Transform takes freq. domain to time
    • Fourier synthesis is usual kind of additive synthesis
    • Other possibilities: wavelets, Walsh functions
  • Definite-pitched sounds are (more-or-less) periodic

rev. 18 Feb. 08

time domain frequency domain 2
Time Domain & Frequency Domain (2)
  • Sine & cosine in trigonometry
  • Phase in degrees (0 to 360, for ordinary use)…
  • Or radians 0 to 2*pi, for technical use)
  • Phase rarely important for us => say sinusoid
  • “Simple” example of Fourier synthesis: perfect square wave = an infinite number of odd harmonics
  • …but only if they’re all in phase
  • Demo: Fourier applet

31 Jan. 07

time domain frequency domain 3
Time Domain & Frequency Domain (3)
  • But real-world sounds are almost never periodic!
  • True, but definite-pitched notes are “close enough” for Fourier analysis to be useful
    • In reality, usually use a series of short-term Fourier Transforms (STFTs): time-frequency domain
    • Look at spectra of individual notes (from Iowa samples, EBU arpeggios)
  • This is mathematics & physics; perception (psychoacoustics) is different, subtle
    • We perceive musical sound in both domains—sometimes more one, sometimes more the other
    • Phase affects waveform, but maybe not perception

25 Feb. 07

real world musical sounds
Real-World Musical Sounds
  • The “Attack/Sustain/Release” model for notes
    • Attack, Sustain, Release modified from recordings
  • Used in the Kurzweil 250 (1984), etc.
    • Original version had only 2 MB for all samples
    • Piano had diff. samples for 2 loudness levels
    • …and diff. sound for every 4-6 semitones
    • 1-2 sec. per sample for A+S+R
  • How good did the K250 really sound?
    • COUNTDOWN, by Christopher Yavelow
    • “An opera for the nuclear age”
      • “the ‘orchestral accompaniment’ is in reality a Kurzweil-250 digital sampler, synchronized to the baton of the conductor…”

12 Feb. 07

real world musical sounds28
Real-World Musical Sounds
  • Nowadays, can afford “unlimited” sustain
  • …but also need diff. sounds for many (8?) diff. loudness levels (multisampling)
    • “All Together Now”, Electronic Musician, Jan. 2007
  • …and diff. sound for every semitone or two
  • W/ unlimited sustain, takes gigabytes just for piano!

19 Feb. 07

scholars and others beware
Scholars (and others) Beware!
  • Plausible (at the time) assumptions
    • Stomach ulcers can’t be caused by organisms (20th century)
    • Men have more teeth than women (ancient)
  • What you expect & what you see
    • Sponges, dinosaurs, etc.: discuss later
  • What you expect & what you hear
    • Don & the Kurzweil 250 flute sound
    • Don, a famous musician, & K250 handclaps
    • Huron on what he “knew” & learned
      • R. Moog at Kurzweil & piano touch

rev. 20 Sep. 2006

uncompressed audio files are big
Uncompressed Audio Files are Big
  • 1 byte = 8 bits (nearly always)
  • How much data on a CD?
    • CD audio is 44,100 samples/channel/sec. * 2 bytes/sample * 2 channels = 176,400 bytes/sec., or 10.5 MByte/min.
    • CD can store up to 74 min. (or 80) of music
    • 10.5 MByte/min. * 74 min. = 777 MBytes
    • Actually more: also index, error correction data, etc.

22 Sep. 2006

compressed audio lossless lossy
Compressed Audio: Lossless & Lossy
  • Don’t confuse data compression and dynamic-range compression (a.k.a. audio level compression, limiting)
  • Codec = COmpressor/DECompressor
  • Lossless compression
    • Standard methods (LZW: .zip, etc.) don’t do much for audio
    • Audio specific methods
      • MLP used for DVD-Audio
      • Apple & Microsoft Lossless
  • Lossy compression
    • Depends on psychoacoustics (“perceptual coding”)

16 Feb. 06

lossless compression of text
Lossless Compression of Text
  • Lossless compression of a children’s nursery rhyme

Pease porridge hot,

Pease porridge cold,

Pease porridge in the pot,

Nine days old;

Some like it hot,

Some like it cold,

Some like it in the pot,

Nine days old.

    • Diagram from Witten, Moffat, & Bell, Managing Gigabytes, 2nd ed.

12 Feb. 07

psychoacoustics perceptual coding
Psychoacoustics & Perceptual Coding
  • Pohlmann, Ken (2005). Principles of Digital Audio, 5th ed., Chapter 10: Perceptual Coding
  • Rationale: much better data compression
  • Based on physiology of ear and critical bands
    • Not fixed frequency: any sound creates one or more critical bands
  • Masking
    • Depends on relative loudness & frequency
    • Noise is much better than pitched sounds
  • Threshhold of hearing
    • Depends greatly on frequency

22 Sep. 2006

compressed audio lossy compression 1
Compressed Audio: Lossy Compression (1)
  • General method

1. Divide signal into sub-bands by frequency

2. Take advantage of (e.g.):

      • Masking (“shadows”), via amplitude within critical bands
      • Threshhold of audibility (varies w/ frequency)
      • Redundancy among channels
  • MPEG-1 layers I thru III (MP1, 2, 3), AAC get better & better compression via more & more complex techniques
    • “There is probably no limit to the complexity of psychoacoustics.” --Pohlmann, 5th ed.
    • However, there probably is an “asymptotic” limit to compression!
  • Implemented in hardware or software codecs

22 Feb. 06

compressed audio lossy compression 2
Compressed Audio: Lossy Compression (2)
  • Evaluation via critical listening is essential
    • ITU 5-point scale
      • 5 = imperceptible, 4 = perceptible but not annoying, 3 = slightly annoying, 2 = annoying, 1 = very annoying
    • Careful tests: often double-blind, triple-stimulus, hidden reference
      • E.g., ISO qualifying AAC with 31 expert listeners (cf. Hall article)
    • Test materials chosen to stress codecs
      • Common useful tests: glockenspiel, castanets, triangle, harpsichord, speech, trumpet
      • Soulodre’s worst-case tracks: bass clarinet arpeggio, bowed double bass, harpsichord arpeggio, pitch pipe, muted trumpet
  • References: Pohlmann Principles of Digital Audio (on reserve)

17 Feb. 06

hybrid representation compression 1
Hybrid Representation & Compression (1)
  • Events (with “predefined” timbre) take very little space
    • Mozart fragment AIFF (CD-quality audio): 794,166 bytes
    • Mozart fragment MIDI file: 193 bytes
    • Timbre takes same amount of space, regardless of music length!
    • Problem: don’t have exact timbre for any performance
  • CSound, CMusic, etc. have MIDI-like score and software synthesis def. of orchestra

17 Feb. 07

hybrid representation compression 2
Hybrid Representation & Compression (2)
  • Mike Hawley’s approach: find structure in audio; create events & timbre definition
    • Hawley, Michael J. (1990). The Personal Orchestra, or, Audio Data Compression by 10000:1. Usenix Computing Systems Journal 3(2), pp. 289—329.
  • Could hybrid event/audio representation lead to his “audio data compression by a factor of 10,000”?
  • Maybe, but no time soon!

17 Feb. 07

selfridge field on describing musical information
Selfridge-Field on Describing Musical Information
  • Cf. Selfridge-Field, E. (1997). Describing Musical Information.
  • What is Music Representation? (informal use of term!)
    • Codes in Common Use: solfegge (pitch only), CMN, etc.
    • “Representations” for Computer Application: “total”, MIDI
  • Parameters of Musical Information
    • Contexts: sound, notation/graphical, analytic, semantic; gestural?
    • Concentrates on 1st three
  • Processing Order: horizontal or vertical priority
  • Code Categories
    • Sound Related Codes: MIDI and other
    • Music Notation Codes: DARMS, SCORE, Notelist, Braille!?, etc.
    • Musical Data for Analysis: Plaine and Easie, Kern, MuseData, etc.
    • Representations of Musical Patterns and Process
    • Interchange Codes: SMDL, NIFF, etc.; almost obsolete!

30 Jan. 06

review the four parameters of notes
Review: The Four Parameters of Notes
  • Four basic parameters of a definite-pitched musical note

1.pitch: how high or low the sound is: perceptual analog of frequency

2. duration: how long the note lasts

3. loudness: perceptual analog of amplitude

4. timbre or tone quality

  • Above is decreasing order of importance for most Western music
  • …and decreasing order of explicitness in CMN!
review how to read music without really trying
Review: How to Read Music Without Really Trying
  • CMN shows at least six aspects of music:
    • NP1. Pitches (how high or low): on vertical axis
    • NP2. Durations (how long): indicated by note/rest shapes
    • NP3. Loudness: indicated by signs like p , mf , etc.
    • NP4. Timbre (tone quality): indicated with words like “violin”, “pizzicato”, etc.
    • Start times: on horizontal axis
    • Voicing: mostly indicated by staff; in complex cases also shown by stem direction, beams, etc.
  • See “Essentials of Music Reading” musical example.
complex notation selfridge field s fig 1 4
Complex Notation (Selfridge-Field’s Fig. 1-4)
  • Complications on staff 2:
    • Editorial additions (small notes)
    • Instruments sharing notes only some of the time
    • Mixed durations in double stops
    • Multiple voices (divisi notation)
      • Rapidly gets worse with more than 2!

10 Feb.

complex notation selfridge field s fig 1 443
Complex Notation (Selfridge-Field’s Fig. 1-4)
  • Multiple voices rapidly gets worse with more than 2
    • 2 voices in mm. 5-6: not bad: stem direction is enough
    • 3 voices in m. 7: notes must move sideways
    • 4 voices in m. 8: almost unreadable—without color!
    • Acceptable because exact voice is rarely important

rev. 12 Feb.

domains of musical information
Domains of Musical Information
  • Independent graphic and performance info common
    • Cadenzas (classical), swing (jazz), rubato passages (all music)
  • CMN “counterexamples” show importance of independent graphic and logical info
    • Debussy: bass clef below the staff
    • Chopin: noteheads are normal 16ths in one voice, triplets in another
  • Mockingbird (early 1980’s) pioneered three domains:
    • Logical: “ note is a qtr note” (= ESF(Selfridge-Field)’s “notation”)
    • Performance: “ note sounds for 456/480ths of a quarter” (= ESF’s “sound”; also called gestural)
    • Graphic: “ notehead is diamond shaped” (= ESF’s “ notation”)
    • Nightingale and other programs followed
  • SMDL added fourth domain
    • Analytic: for Roman numerals, Schenkerian level, etc. (= ESF’s “analytic”)

1 Feb. 06

communicating about music
Communicating about Music
  • Basic principle of communicating with people: say just what’s necessary
  • Strunk & White: “Omit needless words”
  • Applies to a lecture or a notation

22 Feb. 07

representation from abstract to concrete
Representation, from Abstract to Concrete
  • Cf. Basic Representations of Music & Audio
  • Abstract: represention: semantics only
  • Intermediate: syntax (mapping rules)?
  • Concrete
    • for use by computers: encoding
    • for use by humans: if visual, notation (involves graphics and/or typography)
  • Analogous to knowledge representation vs. data structure
semantics in music
Semantics in Music
  • Denotation (explicit, well-defined)...
  • vs. Connotation (implicit, ill-defined)
  • In text
    • Two “definitions” of pig:
      • 1. Ugh! Dirty, evil-smelling creatures, wallowing in filthy sties! (Hayakawa)
      • 2. Mammal with short legs, cloven hoofs, bristly hair, and a cartilaginous snout used for digging (Amer. Heritage)
    • Prose is “mostly” denotation
    • Poetry is art => connotation much more important
  • Music is always art, & only connotation!
    • What is a musical idea?
  • Major issue for content-based music IR
from representation to notation
From Representation to Notation
  • Choosing a representation inevitably introduces bias
  • Given a representation, choosing notation inevitably introduces more bias
  • Important to consider the purpose (R. Davis et al; Wiggins et al)
  • For huge body of important music, we have no choice: notation is CMN (Conventional Music Notation)!
    • Really “CWMN” (W = Western)
    • Alternative for some music: tablature (guitar, lute, etc.)
    • CMN is among the most successful notations ever...
    • but enormously complex and subtle
review how people find text information
Review: How People Find Text Information
  • What user wants is almost always concepts…
  • But computer can only recognize words
review how computers find text information
Review: How Computers Find Text Information
  • “Stemming, stopping, query expansion” are all tricks to increase precision & recall (avoid false negatives & false positives) due to synonyms, variant forms of words, etc.
notation says much about representation
Notation Says Much about Representation
  • CMN standard for Western music after ca.1650
  • Evolved for “classical” music, but heavily used for very wide range (pop, jazz, folk, etc.)
  • Composers/arrangers/transcribers have pushed it hard => reveals things about music representation in general
  • Will concentrate on notation (CMN)
problems example 1 superficial but interesting
Problems: Example 1 (superficial but interesting)
  • Ravel work has slur with 7 inflection points
  • Impressive, but complexity is purely graphical
  • No big deal in terms of representation
  • …but influence of performance on notation is revealing
duration and higher level concepts of time
Duration and Higher-Level Concepts of Time
  • Schubert Impromptu(& e 4)
  • Measures: everything between barlines
  • Time signature: 3/4 = 3 quarter notes per measure
  • Triplets: 3 notes in the time normally used by 2
    • General concept is tuplets
problems example 2 deep
Problems: Example 2 (Deep)
  • Chopin Nocturne has nasty situation (& e 5)
  • One notehead is triplet in one voice, but normal duration in another
  • “Semantics” (execution) well-defined, obvious
    • Note starts 1/16 before barline…
    • But also (2/3)*(1/16) before barline! How to play?
  • Reason: musical necessity
  • Solution for performer: “rubato”
  • Solution for music IR program: ?
problems example 3 medium
Problems: Example 3 (Medium)
  • Bach: time signature change in middle of measure
  • (& e 6)
  • “Semantics” well-defined and obvious
    • Measure has duration of 18 16ths…
    • But not until the middle of the measure!
  • How does this make sense?
  • Triplets express same relationship as equivalent simple/compound meter
  • Invisible (unmarked) triplets
  • Cf. Bach Prelude: two time signatures at once (& e 7)
  • Reason: avoid clutter
problem 4 medium
Problem 4 (Medium)
  • Brahms Capriccio (& e 8)
  • Time signature 6/8 => measure lasts 12 16ths
  • A dotted half note always lasts 12 16ths…
  • but here it clearly lasts only 11 16ths!
  • Reason: avoid clutter
two ways to have two clefs at once
Two Ways to Have Two Clefs at Once
  • Clef gives vertical offset to determine pitch
  • Debussy (& e 9)
    • Bizarrely obvious something odd involving clefs
  • Ravel (& e 10)
    • Only comparing time signature (3/8) and note durations makes it clear both clefs affect whole measure
  • Reason: save space (by avoiding a 3rd staff)
surprise music notation has meta principles 1
Surprise: Music Notation has Meta-Principles! (1)

1. Maximize readability (intelligibility)

  • Avoid clutter = “Omit Needless Symbols”
  • Try to assume just the right things for audience
  • Audience for CMN is (primarily) performers
  • General principle of any communication
    • Applies to talks as well as music notation!
  • Examples: Schubert, Bach, Brahms
surprise music notation has meta principles 2
Surprise: Music Notation has Meta-Principles! (2)

2. Minimize space used

  • Save space => fewer page turns (helps performer); also cheaper to print (helps publisher)
  • Squeezing much music into little space is a major factor in complexity of CMN
  • Especially important for music: real-time, hands full
  • Examples: Telemann, Debussy, Ravel
the rules of music notation
The “Rules” of Music Notation
  • Tempting to assume that rules of such an elaborate & successful system as CMN work (self-consistent, reasonably unambiguous, etc.) in every case
  • But (a) “rules” evolved, with no established authority; (b) many of the “rules” are very nebulous
  • In common cases, there's no problem
  • If you try to make every rule as precise as possible, result is certainly not self-consistent
  • Trying to save space makes rules interact; something has to give!
music notation software and intelligence
Music Notation Software and Intelligence
  • Despite odd notation, really nothing strange going on in almost all of these examples
    • Ravel slur, Debussy & Ravel 2 simultaneous clefs, Bach & Schubert invisible triplets, Brahms “short” dotted-half note, Telemann 4 voices/staff are all simple situations
    • Chopin Nocturne is complex
  • Programmers try to help users by having programs do things “automatically”
  • A good idea if software knows enough to do the right thing “almost all” the time—but no program does!
  • Notation programs convert CMN to performance (MIDI) and vice-versa => requires shallow “semantics”; makes things much harder
conclusions review 1
Conclusions: Review (1)
  • Representations express Semantics
  • Semantics of Music; Denotation & Connotation
  • Principles of CMN
  • Meta-Principles of CMN

1. Maximize readability; Omit Needless Symbols

      • Try to assume just the right things for audience
      • General principle of any communication

2. Minimize space used

      • Save space => fewer page turns, less paper
conclusions review 2
Conclusions: Review (2)
  • We need CMN or equivalent to solve spectrum of music-IR (and other music-IT) problems
    • But CMN can’t represent everything we want
    • Even when it can, actually may not (esp. explicitly)
    • Need high-level intelligence to interpret
  • Solution: unknown
    • Likely to require major funding :-)
why music ir research is important outside of music
Why Music-IR Research is Important (Outside of Music)
  • Some problems directly related to other areas of informatics
    • Example: Approximate string matching in bioinformatics
  • Encourages progress on real semantics
    • Connotation is an important part of meaning in everything
    • Can often ignore, but any semantics in arts forces you to deal with connotation
    • Music is at least as quantifiable as any art, so likely to be more tractable than others!
mozart variations for piano k 265 on ah vous dirais je maman a k a twinkle
Mozart: Variations for piano, K. 265, on “Ah, vous dirais-je, Maman”, a.k.a. Twinkle