slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Synchronization of musical sound and visual images: Issues of empirical and practical significance in multimedia develop PowerPoint Presentation
Download Presentation
Synchronization of musical sound and visual images: Issues of empirical and practical significance in multimedia develop

Loading in 2 Seconds...

play fullscreen
1 / 33

Synchronization of musical sound and visual images: Issues of empirical and practical significance in multimedia develop - PowerPoint PPT Presentation


  • 82 Views
  • Uploaded on

Synchronization of musical sound and visual images: Issues of empirical and practical significance in multimedia development. Scott D. Lipscomb , Ph.D. Institute for Music Research, University of Texas at San Antonio. Model of Film Music Perception Lipscomb & Kendall (1994).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Synchronization of musical sound and visual images: Issues of empirical and practical significance in multimedia develop' - ronda


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Synchronization of musical sound and visual images:Issues of empirical and practical significance in multimedia development

Scott D. Lipscomb, Ph.D.

Institute for Music Research,

University of Texas at San Antonio

Acoustical Society of America

association judgment

Association Judgment

Referential Meaning

Acoustical Society of America

influential musical parameters
Influential Musical Parameters:
  • clarity of tonal center
  • harmonic complexity
  • dynamic variation
  • tempo
    • absolute rate & fluctuation
  • phrase structure
  • melodic activity

Acoustical Society of America

accent structure alignment

Accent Structure Alignment

Syntactical Meaning

Acoustical Society of America

sources of perceived accent
Sources of Perceived Accent
  • Musical
    • pitch height, loudness, timbre
  • Visual
    • spatial location, shape, color

Acoustical Society of America

alignment possibilities

Consonant

Out-of-Phase

Dissonant

Alignment Possibilities

Acoustical Society of America

method av stimuli
Method - AV Stimuli
  • Experiment One
    • simple single-object animations and pitch sequences (5-sec); created by author
  • Experiment Two
    • excerpts from experimental animations by Norman McLaren (8-sec); “Dots” (1940), “Canon” (1964), & “Synchromy” (1971)
  • Experiment Three
    • excerpts from “Obsession” (25-sec) with musical score by Bernard Herrmann

Acoustical Society of America

control reliability vs ecological validity

Control & Reliability vs.Ecological Validity

Experiments Two & Three

Acoustical Society of America

slide17

Experiment Two

“Dots” by Norman McLaren

Acoustical Society of America

slide18

Experiment Two

“Synchromy” by Norman McLaren

Acoustical Society of America

slide20

Experiment Three

“Obsession”--music by B. Herrmann

Acoustical Society of America

vame effectiveness across all experiments
VAME - EffectivenessAcross All Experiments

Acoustical Society of America

film music perception paradigm revised

Perception

Accent Structure

Alignment

Association

Judgment

No Shift

of

Attentional Focus

Shift

of

Attentional Focus

Film Music Perception Paradigm(revised)

Aural

Stimulus

Visual

Stimulus

Audio-Visual Congruence

Implicit

Processes

No

Yes

Acoustical Society of America

enhancing instructional materials with multimedia

Enhancing Instructional Materials with Multimedia

CD-ROM Companion to the

Handbook of Music Psychology

(D. Hodges, Ed.; IMR Press)

Acoustical Society of America

hyperlinks to sound files
Hyperlinks to Sound Files

“If a pianist plays a low A (110 Hz) on the piano, the resulting vibration does not consist of only a single, periodic vibration at the rate of 110 times per second. Rather, the string vibrates as a whole (110 Hz), in halves (220 Hz), in thirds (330 Hz), in fourths (440 Hz), and so on. Therefore, there is vibrational energy not only at the fundamental frequency of 110 Hz but also at each of these integer multiples [audio examples: a) complex tone made up of 8 partials & b) complex tone built one partial at a time …”

Lipscomb & Hodges, 1996, p. 97

Acoustical Society of America

sounds to attract attention
Sounds to Attract Attention

“If a pianist plays a low A (110 Hz) on the piano, the resulting vibration does not consist of only a single, periodic vibration at the rate of 110 times per second. Rather, the string vibrates as a whole (110 Hz), in halves (220 Hz), in thirds (330 Hz), in fourths (440 Hz), and so on. Therefore, there is vibrational energy not only at the fundamental frequency of 110 Hz but also at each of these integer multiples [audio examples: a) complex tone made up of 8 partials & b) complex tone built one partial at a time …”

Lipscomb & Hodges, 1996, p. 97

Acoustical Society of America

exploratory environments
Exploratory Environments
  • Beating—experimentation with constructive & destructive interference and the critical bandwidth
  • WaveMix—combining partials to create a complex tone
  • Signal—allows student to create a complex signal by manipulating partial amplitudes & phase relationships, hear the result, and view an animation approximating molecular motion

Acoustical Society of America

sound as a cue
Sound as a Cue
  • Computer Guts
    • instructional package designed by Dr. David Sebald (UTSA Division of Music)
      • more examples available online at:

http://www.aim-ed.com/

Acoustical Society of America

conclusions use of multimedia to enhance the learning experience
Conclusions: Use of Multimedia to Enhance the Learning Experience
  • provide additional information through illustration
    • audio examples or animations
  • focus user attention
  • explicit association judgments
  • create exploratory environments
    • allowing students the opportunity to incorporate the highest levels of knowledge (Bloom, 1956)

a work in progress

Acoustical Society of America

slide33

Contact Info:Dr. Scott D. LipscombInstitute for Music Research Division of Music The University of Texas at San Antonio

6900 N. Loop 1604 West

San Antonio, TX 78249

phone: (210) 458-5334

FAX: (210) 458-4381

lipscomb@utsa.edu

http://music.utsa.edu/~lipscomb

Acoustical Society of America