streaming architectures for very large collections n.
Skip this Video
Download Presentation
Streaming Architectures for Very Large Collections

Loading in 2 Seconds...

play fullscreen
1 / 11

Streaming Architectures for Very Large Collections - PowerPoint PPT Presentation

  • Uploaded on

Streaming Architectures for Very Large Collections. Jim DeRoest Director Streaming Media Technologies, ResearchChannel Assistant Director, University Computing Services University of Washington 206 543-6343. Who We Are And Support.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Streaming Architectures for Very Large Collections' - deepak

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
streaming architectures for very large collections

Streaming Architectures for Very Large Collections

Jim DeRoest

Director Streaming Media Technologies, ResearchChannel

Assistant Director, University Computing Services

University of Washington

206 543-6343

who we are and support
Who We Are And Support
  • ResearchChannel/UWTV - Exploring digital convergence and educational outreach.
      • National Echostar and CATV to over 5 million homes
      • International VoD and live streams – I2 Abilene
      • Experiments - First cinema quality HD over IP
  • KEXP – UW and EMP collaboration exploring digital radio
      • Uncompressed 1.4Mb live stream
    • Real time web playlist generation
      • World music format – DJ choice
  • Faculty, Staff, Student, Clinical, K20
  • Other Affiliate/Collaborative Projects
managing content
Managing Content
  • Large Collection Problem
    • Disparate Asset Types
      • HD, Broadcast TV, audio libraries, pictures, rich text
    • Instantiation: capture, encoding, metadata, storage
      • Multiple sources, formats, schema
    • Interfaces
      • TV post production, radio DJ, viewer/listener
    • Discovery and Administration
      • Naming, categorization, relations, internationalization
      • Distributed management, sharing, searching
    • Access Controls
      • Authentication, authorization, rights, privacy, security
mediaweb project
MediaWeb Project
  • 18 month technology/requirements review
      • Requirements
        • Interviewed engineers, DJ’s, librarians and consumers
      • Reviewed vendor products
        • Proprietary, 70% solution, consumer specific
      • Created prototype during review process
        • Pluggable, scalable, standards based architecture
      • Buy, build, both, call it quits?
        • Decision to go with prototype



Creation, Admin

TV Broadcast


Radio DJ Services

UW MediaWeb– Digital Media Asset Management System

Asset Management Services

Remote Stream


Media Store









Delivery Services

Image, Quicktime, Windows,

Real, HD, Webcasting …

core services
Core Services
  • Media Store
    • HSM online/nearline multi-vendor

High Performance Storage System (HPSS)

    • Gigabit interconnects.
    • Distributed, redundant, recoverable
    • Block I/O stream to servers (smoke and mirrors
  • Metadata Service
    • SQL, directory, imbedded objects, access control, rights
  • Stream Manager
    • Maintains asset locality state and aging
  • Request Re-Director
    • Re-direct requests to target stream or application service
  • Video
      • Airtime/Dubtime to 50Mb MPEG2 i-frame base
        • Louth/Mav triggers/APIs
      • Metadata generation
        • Looking at Virage plugins
  • Audio
      • CD ripping raw audio base
      • Metadata map CDDB sites (Muse, AMG, …)
      • DJ metadata support

Transcode from base formats as needed

  • Schema – How rich can we afford
      • Consumer specific
      • Standards – Dublin Core, MPEG7
      • Vendor (Muse, AMG, Louth playlists, …)
      • Draft submitted for comment I2-VidMid VoD and ViDe
  • Access/Rights controls
      • Vendor specific, MPEG21
      • Auth*2 – NetID Pubcookie, Shibboleth, …
      • Imbedded – Honored by clients
      • Trust
  • Metadata Objects
      • SQL (base), LDAP, Imbedded objects/methods
  • Streaming Servers
      • Standard file system or IO API required

Stream initiated as first blocks arrive to local cache

Tape, disk, OS and network speed vs. hybrid file systems

      • Selectable stream stub support

First +30 seconds of clip hide mount time content on tape

      • Low bit rate (no MPEG1)

Quicktime, Real, Windows Media

      • High bit rate

Windows Media – KEXP 1.4Mb live stream

Optivision – RC 5.6Mb live stream to remote sites

MPEG2 – Concurrent Computer Xstreme 5.6Mb VoD

MPEG4 – Looking at IBM Videocharger – interface issues

HD – In-house (>270Mb), other vendor lower bit rates

mediaweb project status
MediaWeb Project Status
  • Prototype operational with simple search
      • Geographically separated/mirrored server environment
      • Video: RC/UWTV content (~700 hrs MPEG2, lower bit rates)
      • Audio: Sample audio
  • Refining schema
      • Object experiments
      • Imbedded controls (MPEG7)
      • Reviewing large vendor schemas
      • Deploying admin tools
  • Building feed systems (RC, +35,000 disc KEXP library)
  • Incorporating faculty/student mgt (expiration, quotas, …)
  • Evaluating other large content requests
  • Ongoing evaluation, scaling, benchmarking architecture
  • Looking for opportunities to collaborate!
  • UW CSci animation teaching tool

“How Networks Change Teaching: A Case Study”

Louis Fox, Kris Martin, Barbara Mones-Hattal

I2 VIMM Friday, October 5th 1:30EDT

  • KEXP live stream archives
  • KEXP automated playlist
  • EMP live webcasting and archival
  • UW Catalyst collection interface
  • HD IP Videoconferencing