Sonification and data representation l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 19

Sonification and Data Representation PowerPoint PPT Presentation


  • 274 Views
  • Updated On :
  • Presentation posted in: General

Sonification and Data Representation. John E. Bower Department of Music. Sonification. Concept. Tools Implementation. Concept. Sonification is the representation of some data through metaphorical structures of sound Absolute, quantitative data is interpreted qualitatively

Related searches for Sonification and Data Representation

Download Presentation

Sonification and Data Representation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Sonification and data representation l.jpg

Sonification and Data Representation

  • John E. Bower

  • Department of Music


Sonification l.jpg

Sonification

Concept

Tools

Implementation


Concept l.jpg

Concept

  • Sonification is the representation of some data through metaphorical structures of sound

  • Absolute, quantitative data is interpreted qualitatively

  • The ideal is overt, but reticent immediacy and clarity of message

  • Interpreting the realization can be a learned activity

  • A successful sonification should efficiently convey the metrics of the source data without an inappropriate amount of user fatigue


Concept4 l.jpg

Concept

Ambient Device’s Orb


Concept5 l.jpg

Concept

John Hancock Building

Boston, Mass.


Tools l.jpg

Tools

Michael Winslow?


Csound l.jpg

Csound

  • Progeny from the Music-N languages developed by Max Mathews et al at Bell Labs, Princeton, and Stanford Universites in the 1960s

  • Csound evolves in the 1980s under Barry Vercoe at M.I.T.

  • Freeware GNU LGPL

  • Runs on most every platform, from an Amiga to a Sharp Zire

  • Proprietary “assembly” language

  • “Batch” processing paradigm: sound is rendered from a pair of text files, one denoting user-designed “patches”, the other the parameters to inform those patches

  • Most mature computer music app with close to 500 opcodes

  • Not ideal for real-time use


Max msp jmax l.jpg

Max/MSP | jMax

  • Original author: Miller Puckette

  • Initially developed at IRCAM, Paris

  • Available commercially as Max/MSP for Mac/Win32

  • jMax is the open-source (GNU GPL) version implemented in Java with a C-based rendering engine for OSX, Linux, Win32, IRIX

  • Created specifically for real-time use

  • Graphical “programming” paradigm

  • Extensible by external C plugins

  • Well-supported and used by musicians, media artists, clinicians, architects, etc.

  • “Clone” app, PD (Pure Data), by Puckette available as an open-source app for OSX/Linux/IRIX/Win32


Slide9 l.jpg

Max “graphical” patch


Supercollider l.jpg

SuperCollider

  • Original author: James McCartney

  • Originally a commercial app; open-source (GPL) with SuperCollider3

  • Available for OSX/Linux; Win32 port somewhat likely

  • Object-oriented programming language based upon Smalltalk but with C language family syntax

  • Ideal for real-time use

  • Operates on a distributive paradigm (not shared-memory) facillitated by OSC networking protocol

  • Extensible with Smalltalk/Objective C classes, C++ plugins

  • Arguably the most powerful/efficient language to date: “bleeding edge”

  • Documentation? What’s documentation?????


Implementation l.jpg

Implementation

  • Much like a musical composition, elements of a sonification may include:

    • Pitch and register

    • Rhythm and tempo

    • Phrase and cadence

    • Timbre and orchestration

    • “Modality”

    • Spatialisation

  • The sonification may be overtly musical or ambient (musical or naturalistic) in design


Research examples l.jpg

Research: examples

  • Peep, a network “auralizer” by Michael Gilfix

  • First presented in a paper at Usenix, 2000

  • Discrete events are mapped to short, naturalistic sounds; for instance, a user logging in may be represented by a cricket chirping

  • Continuously variable states such as system load are mapped to continuous sounds like water or wind

Peep


Research examples13 l.jpg

Research: examples

  • Conducted by IBM Research Division’s David A. Rabenhorst et. al.

  • System for complementary visualization and sonification of semiconductor models in the FIELDAY program

  • Program examines any two of the electrostatic potential, electron concentration, and hole concentration of the model in complementary windows

  • A 3D cursor navigates one window visually while the sonification generates sound based upon the 3D vector gradients of the chosen scalar field at the same focal point in the solid as displayed in the second, non-focused, window

  • Allows the user to simultaneously concentrate on one data field visually and one aurally

IBM Research


Research examples14 l.jpg

Research: examples

  • The scalar data field is sonified by a musical triad with voice doubling on each individual component (six voices total)

  • Each signed component of the 3D vector gradient maps to both the detuning and stereo location of a note in the triad

  • Rough precision of each vector gradient (x, y, z) is measured by the panning (direction and intensity) of the doubled-tone, fine precision by the intonation difference (up to +- one semitone) between the two notes

  • The three structural components of the triad are assigned sounds with different spectral characteristics to facilitate differentiation between the voices

  • Static vector gradients have their volumes attenuated so they are not as present in the sonification as actively-changing gradients

IBM Research, continued


Research examples15 l.jpg

Research: examples

  • Tools developed at FSU by Myke Gluck for visualization/sonification of GIS data

  • Tools are intended for general users

  • One represents data through melodies that undergo variations in tempo, key, articulation, and embellishment

  • Others utilize:

    • Range depiction (pitch, volume, timbral quality: “brightness”)

    • Multiple tones (octaves represent data classes, particular notes within an octave represent data values)

    • Different musical scales (diatonic, pentatonic, etc.)

    • Different chord qualities (major, minor, etc.)

Spatial Data Mining


Research examples16 l.jpg

Research: examples

  • Collaboration between biologist Mary Anne Clark and artist John Dunn at Texas Wesleyan University

  • Sonification designed with the following:

    • Alpha and beta regions of proteins differentiated by changes in instrumentation and/or pitch

    • Pitch material determined by assigning a fixed pitch to each amino acid, or assigning pitch based on a frequency histogram of the amino acids with the more “consonant” intervals representing the most frequently occurring amino acids

    • Register determined by the water solubility of the amino acids, with the most insoluble getting the lowest pitches, the most soluble the highest

Life Music: Protein Sonification


Research examples17 l.jpg

Research: examples

  • IBM’s Sonnet “visual” programming language for interface design.

    • Possessed features for adding sound to interfaces to monitor data flow and limit GUI clutter.

    • Used internally at IBM. Never materialized commercially?

  • Spatialisation of air traffic data with aural cues for aircraft proximity, rate of approach, various “emergency” situations, etc. (NASA Ames Research Center)

  • Various tools designed to monitor smog conditions in Los Angeles, Yellowstone forest-fire data; monitoring of patient blood oxygen level, blood pressure, and other vitals for operating theater use; sonification of nuclear power plant control room data; stock and other financial trends

Other research


Implementation18 l.jpg

Implementation

  • Short user study of sonification examples, each utilizing a different means of interpreting data

  • Indicate on the sheet your perception of either an increasing, decreasing, or unvarying metric for each example

  • Please rate each example with a “fatigue” level from 1 to 10 as well as a clarity ranking from 1 to 10

  • At the end of the sheet, please indicate if you have demonstrable musical training


Sonification and data representation19 l.jpg

Sonification and Data Representation

  • John E. Bower

  • [email protected]

  • www.stillmovement.org


  • Login