Sounding it out sharing and disseminating audio visual data
Download
1 / 34

Sounding It Out: Sharing and Disseminating Audio-Visual Data - PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on

Sounding It Out: Sharing and Disseminating Audio-Visual Data. Louise Corti and Mus Ahmet UK Data Archive University of Essex, UK. Beyond Numbers: Preserving and Delivering non-numeric collections, IASSIST 2008, CA. Qualitative data . various data types created:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Sounding It Out: Sharing and Disseminating Audio-Visual Data' - kovit


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Sounding it out sharing and disseminating audio visual data
Sounding It Out:Sharing and Disseminating Audio-Visual Data

Louise Corti and Mus Ahmet

UK Data Archive

University of Essex, UK

Beyond Numbers: Preserving and Delivering non-numeric collections, IASSIST 2008, CA


Qualitative data
Qualitative data

  • various data types created:

    • interview/group recordings and transcripts

    • field notes and diaries

    • audio visual sources – still and moving

  • interviews often captured as an audio source though recordings

  • visual data increasingly being captured in the field


Sharing audio visual data
Sharing audio-visual data

Can be a challenge for social research

data :

  • ethical and consent issues

  • quality of recording

  • various proprietary formats

  • storage capacity


What does ukda currently do
What does UKDA currently do?

  • we acquire audio-visual data as part of qualitative collections – but not routinely

  • we deal with both analogue and born digital data

  • we evaluate data and consider ethical issues

  • we do NOT anonymise any audio visual data

  • we process, document and catalogue the data

  • we disseminate or provide links to the data


Key challenges
Key challenges

  • minimising data storage requirements whilst simultaneously maximising audio quality

  • using sustainable open formats for long term preservation

  • using optimal and flexible formats for delivery


Analogue audio data
Analogue audio data

formats we deal with:

  • reel-to-reel tape

  • audio cassette

  • micro cassette (e.g dictaphone)

  • we list the tapes with all the

    metadata we have (later…)

  • we MAY digitise all or selected extracts

  • latter can belabour intensive!


Digitising audio
Digitising audio

  • have digitised a large reel to reel collection (450 interviews each approx. 3 -10 hrs long)

  • have digitised collections on audio-cassette

  • typically we digitise at frequency 48kHz /24 Bits


Reel to reel digitising project
Reel-to-reel digitising project

Our best loved study: The Edwardians

  • conducted by the British Library with tight spec from us – agreed metadata

  • 1203 reels converted by external supplier

  • no filtering or post-processing applied

  • dead air checks and long gaps edited out

  • processing /metadatabase created

  • batches returned every 3 weeks, audit and random listening. MD5 checksum confirmed

  • 2517 .wav files produced

  • HUGE files!!! 2 terabytes of files

  • dissemination solution? Consent agreed, so just technical


Cassette digitising hardware
Cassette Digitising Hardware

Plusdeck 2c is an innovative computer-controlled audiocassette recorder that lets you convert cassette into Wave or MP3 files (allows duplex)

Specifications: Cassette Speed: 4.75 cm/sec

Frequency Response: 20 ~ 18,000 Hz

Separation: 40dB

Signal to Noise Ratio: 55 dB


Analogue to digital conversion software
Analogue to Digital Conversion Software

The Plusdeck 2c is supplied with easy to use software that captures the analogue output from the cassette and converts it to Wave or MP3 files. Post-processing using Audacity


What s the difference between lossy and lossless compession
What’s The Difference Between ‘Lossy’ and ‘Lossless’ Compession?

There are two broad classes of compression algorithms:

Lossy

  • Lossy compression algorithms discard data in order to compress it more than would normally be possible. Examples include JPEG, Vorbis, and MP3 compression

    Lossless

  • Lossless compression algorithms produce compressed data that can be decoded to output that is identical to the original. Zip is a common general-purpose lossless compression format


Audio file formats
Audio File Formats ‘Lossless’ Compession?

  • it is important to distinguish between a file format and a codec

  • a codec encodes and decodes the raw audio data, the data itself is stored in a file with a specific format

  • most audio file formats support only one codec, however, some file formats support multiple codecs, for example AVI

  • there are three major groups of audio file formats

    • uncompressed audio formats:- WAV, AIFF

    • formats with lossless compression: FLAC, Apple

    • formats with lossy compression: MP3, Ogg Vorbis, and AAC


Digital audio data we could deal with
Digital audio data we could deal with ‘Lossless’ Compession?


Preservation formats for audio
Preservation Formats for Audio ‘Lossless’ Compession?

  • preservation formats should, ideally be lossless, non- proprietary and open source, and cost-free to process

  • the only two widely adopted formats that fulfil these criteria are Wave and FLAC


What s so good about flac
What’s so good about FLAC? ‘Lossless’ Compession?

  • FLAC is a file format for audio data compression

  • like MP3 it is a compression format, but it is lossless

  • compression works in a similar way to WinZip

  • does not remove information from the audio stream as lossy compression formats

  • reduction of bandwidth or storage requirements but without sacrificing the integrity of the audio source

  • audio sources encoded to FLAC are typically reduced in size by 50%

  • can be decompressed into an identical copy of the audio data

  • it’s currently the fastest and most widely supported lossless audio codec

  • oh, and its totally free and Open Source!


Dissemination formats for audio
Dissemination Formats for Audio ‘Lossless’ Compession?

  • files disseminated via the internet by streaming or download need to be:

    • a reasonable size

    • in an accessible format

    • free, non-proprietary and open source – wherever possible

  • unfortunately, the universal adoption of the MP3 standard, and lack of native support for other formats means we generally disseminate in MP3 format

  • however, we also encode all audio files to the Ogg Vorbis format and will disseminate in that format if and when it becomes viable


What is an ogg
What is an ‘ogg’? ‘Lossless’ Compession?

  • An ‘ogg’ is often incorrectly understood to refer to an audio file

  • but it is actually wrapper for multimedia content codecs

  • a wrapper is a package which changes the interface to an existing package without substantially increasing its functionality

  • Ogg can be used to wrap lossless audio, compressed video, and various other multimedia codecs

  • xiph.org (http://www.xiph.org)


What is vorbis
What is "vorbis"? ‘Lossless’ Compession?

  • Vorbis is the name of a specific audio compression scheme that's designed to be contained in Ogg

  • for the average user, Vorbis is simply an alternate patent free version of mp3

  • it offers better quality at comparable file size

  • http://www.vorbis.org


Summary audio formats
Summary Audio Formats ‘Lossless’ Compession?


How do we prepare data
How do we prepare data? ‘Lossless’ Compession?

ingest

Preserve an original copy of data

.wav

confirm the data is identical using Flac verify option

Preservation copy

.flac

convert whole file or extracts

Dissemination copy: download and streaming

.mp3


Current ukda audio download
Current UKDA audio download ‘Lossless’ Compession?

  • audio data in mp3 format is available from our web download system like other data

  • download as zip with metadata in spreadsheet (DataList - interview level, NOT file level)

  • is kept behind access control system

  • is a very limited solution and OK for smaller collections but not good for collections of large files


Streaming and file based download
Streaming and file-based download ‘Lossless’ Compession?

  • for our large Edwardians collection we have 450 interviews

    • we created multiple mp3s for each interview

    • too large for download zips

  • will be enabling:

    • selective file-based download

    • streaming of extracts (a taster)

    • whole file streaming

  • already have an online search and visualisation for this textual collection so easy to link in

    ESDS Qualidata Online


Pioneers project
Pioneers project ‘Lossless’ Compession?

  • also have a collection of in depth life stories with classic sociologists who have created data rich projects

  • dedicated website

    • bios

    • key publications

    • datasets created

    • life interview

    • thematic extracts of interviews

  • used as test bed for streaming


DEMO ‘Lossless’ Compession?

http://www.data-archive.ac.uk/qualidata/pioneers/thompson/becoming%20a%20researcher.asp

http://www.data-archive.ac.uk/qualidata/pioneers/thompson/interview.asp


Why did we choose adobe flash player
Why Did We Choose Adobe Flash Player? ‘Lossless’ Compession?

  • Adobe Flash is ‘standard issue’ on nearly every computer

  • no external programs like QuickTime are required

  • visitors won't have to do anything special to play the media

Adobe® Flash® Player is the world's most pervasive software platform and reaches over 98.8% of Internet-enabled desktops in mature markets.

Millward Brown survey, conducted March 2008.

Mature Markets include US, Canada, UK, France, Germany, Japan.


Why wimpy rave
Why Wimpy Rave? ‘Lossless’ Compession?

  • uses Adobe Flash

  • easy to install

  • highly customizable

  • readily & Seamlessly Integrated into websites

  • universal - runs on any basic Web server

  • flexible – runs under ASP, PHP Cold Fusion

  • automatically lists and plays the contents of a folder

  • can run off XML, RSS or Text playlists

  • javaScript Integration - control nearly every aspect of Wimpy Rave via Javascript

  • encrypt File Names- Use URL encrypter or Playlister for Wimpy Rave to hide locations of file names

  • Ecommerce Integration - put file into a shopping cart to allow download

    only $89!


Words of advice
Words of advice ‘Lossless’ Compession?

  • use a dedicated streaming server as very resource intensive


A tiny bit about metadata really
A tiny bit about metadata (really) ‘Lossless’ Compession?

  • always aim for rich metadata

  • collect audio-visual file metadata in a database/XML – do not rely on information embedded in the audio file as lost on conversion

  • UKDA currently does not capture any file level metadata

    file level DDI fields are not utilised in house…but they could be. Are looking at this as very necessary

  • Chosen to use Library of Congress (LoC) metadata based on current best practice

  • Use in-house but logical file naming convention for audio visual data

    • Study numberinterview number 2000int001.rtf

    • Study numberinterview number_audio file 2000int001_1.rtf

    • Study numberinterview number_clip number

    • 2000int001_clip1.rtf


Qudex
QUDeX ‘Lossless’ Compession?

  • XML schema for storing annotated multi media data - version 3 released

  • built to represent core functions of CAQDAS packages

    • coding, classifying, memoing, search and retrieval

    • text, audio-visual data, URIs etc

    • can relate/link files, segments, codes, notes, URIs and analysis

  • significant input from ODaF

    • I

  • hopefully under DDI committee for review/tools development

  • volunteers welcomed and testers needed

  • http://www.data-archive.ac.uk/dext/


Qudex1
QuDEx ‘Lossless’ Compession?

  • vendors have been consulted throughout the QuDex development process

  • QuDEx aims to deliver core functionality while other metadata standards are used as appropriate

  • recommended that all materials and metadata relating to a collection are packaged as ‘complex objects’ using a standard such as METS

  • METS: Metadata Encoding and Transmission Standard

  • used for encoding descriptive, administrative, and structural metadata regarding digital objects, expressed using the XML schema language (Library of Congress maintained)


Audio visual xml
Audio-visual XML ‘Lossless’ Compession?

</segment> <!-- Audio segments -->

- <segment id="segment_4" cdate="2007-11-15T11:56:38" mdate="2007-11-15T11:56:38" label="Audio segment: games" displayLabel="First audio segment" creator="Angad" language="en">

- <audio id="audio_1" src="document_2">

<param id="p_1" clipType="time" clipBegin="00:02:22" clipEnd="00:04:44" />

</audio>

</segment> <!-- Video segments -->

- <segment id="segment_5" cdate="2007-11-15T11:56:38" mdate="2007-11-15T11:56:38" label="Video segment: time pass" displayLabel="First Video segment" creator="Angad" language="en">

- <video id="video_1" src="document_2">

<param id="p_2" clipType="smil" clipBegin="2.75min" clipEnd="5.34min" />

</video>

</segment> <!-- xml based segments -->

- <segment id="segment_6" cdate="2007-11-15T11:56:38" mdate="2007-11-15T11:56:38" label="Xml segment: childrens" displayLabel="First Xml segment" creator="Angad" language="en">

- <xml id="xml_1">

<node id="r1" src="document_4" xPtrExp="xpointer(/root/U[1]/range-to(/root/U[10])" />

</xml>

</segment> <!-- Image segments -->

- <segment id="segment_7" cdate="2007-11-15T11:56:38" mdate="2007-11-15T11:56:38" label="Image segment: son-in-law" displayLabel="First Image segment" creator="Angad" language="en">

- <image id="image_1" src="document_5">

<area id="a_1" coords="12,34,54,67" shape="poly" />

</image>

</segment>


Other solutions
Other solutions ‘Lossless’ Compession?

  • wish to synchronise text and audio

  • other solutions possible – mostly proprietary

  • Fedora/Dspace repository systems – mulit media files can be ingested, previewed and downloaded as part of the whole collection

  • wish to use QuDEx to reference text to audio –visual materials

  • Open source tools that may be explored to create dynamic XML-based audio visual experience eg using Adobe FLEX AIR tools (ODaF)


Fedora based viewing systems
FEDORA based viewing systems ‘Lossless’ Compession?


ad