Streaming architectures for very large collections
This presentation is the property of its rightful owner.
Sponsored Links
1 / 11

Streaming Architectures for Very Large Collections PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on
  • Presentation posted in: General

Streaming Architectures for Very Large Collections. Jim DeRoest Director Streaming Media Technologies, ResearchChannel Assistant Director, University Computing Services University of Washington [email protected] 206 543-6343. Who We Are And Support.

Download Presentation

Streaming Architectures for Very Large Collections

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Streaming architectures for very large collections

Streaming Architectures for Very Large Collections

Jim DeRoest

Director Streaming Media Technologies, ResearchChannel

Assistant Director, University Computing Services

University of Washington

[email protected]

206 543-6343


Who we are and support

Who We Are And Support

  • ResearchChannel/UWTV - Exploring digital convergence and educational outreach.

    • National Echostar and CATV to over 5 million homes

    • International VoD and live streams – I2 Abilene

    • Experiments - First cinema quality HD over IP

  • KEXP – UW and EMP collaboration exploring digital radio

    • Uncompressed 1.4Mb live stream

  • Real time web playlist generation

    • World music format – DJ choice

  • Faculty, Staff, Student, Clinical, K20

  • Other Affiliate/Collaborative Projects


  • Managing content

    Managing Content

    • Large Collection Problem

      • Disparate Asset Types

        • HD, Broadcast TV, audio libraries, pictures, rich text

      • Instantiation: capture, encoding, metadata, storage

        • Multiple sources, formats, schema

      • Interfaces

        • TV post production, radio DJ, viewer/listener

      • Discovery and Administration

        • Naming, categorization, relations, internationalization

        • Distributed management, sharing, searching

      • Access Controls

        • Authentication, authorization, rights, privacy, security


    Mediaweb project

    MediaWeb Project

    • 18 month technology/requirements review

      • Requirements

        • Interviewed engineers, DJ’s, librarians and consumers

      • Reviewed vendor products

        • Proprietary, 70% solution, consumer specific

      • Created prototype during review process

        • Pluggable, scalable, standards based architecture

      • Buy, build, both, call it quits?

        • Decision to go with prototype


    Streaming architectures for very large collections

    Transcoding

    Metadata

    Creation, Admin

    TV Broadcast

    Services

    Radio DJ Services

    UW MediaWeb– Digital Media Asset Management System

    Asset Management Services

    Remote Stream

    Manager

    Media Store

    Disk

    Cache

    Request

    Director

    Robotic

    Tape

    Asset

    Metadata

    Delivery Services

    Image, Quicktime, Windows,

    Real, HD, Webcasting …


    Core services

    Core Services

    • Media Store

      • HSM online/nearline multi-vendor

        High Performance Storage System (HPSS)

      • Gigabit interconnects.

      • Distributed, redundant, recoverable

      • Block I/O stream to servers (smoke and mirrors

    • Metadata Service

      • SQL, directory, imbedded objects, access control, rights

    • Stream Manager

      • Maintains asset locality state and aging

    • Request Re-Director

      • Re-direct requests to target stream or application service


    Feeds

    Feeds

    • Video

      • Airtime/Dubtime to 50Mb MPEG2 i-frame base

        • Louth/Mav triggers/APIs

      • Metadata generation

        • Looking at Virage plugins

  • Audio

    • CD ripping raw audio base

    • Metadata map CDDB sites (Muse, AMG, …)

    • DJ metadata support

      Transcode from base formats as needed


  • Metadata

    Metadata

    • Schema – How rich can we afford

      • Consumer specific

      • Standards – Dublin Core, MPEG7

      • Vendor (Muse, AMG, Louth playlists, …)

      • Draft submitted for comment I2-VidMid VoD and ViDe

  • Access/Rights controls

    • Vendor specific, MPEG21

    • Auth*2 – NetID Pubcookie, Shibboleth, …

    • Imbedded – Honored by clients

    • Trust

  • Metadata Objects

    • SQL (base), LDAP, Imbedded objects/methods


  • Delivery

    Delivery

    • Streaming Servers

      • Standard file system or IO API required

        Stream initiated as first blocks arrive to local cache

        Tape, disk, OS and network speed vs. hybrid file systems

      • Selectable stream stub support

        First +30 seconds of clip hide mount time content on tape

      • Low bit rate (no MPEG1)

        Quicktime, Real, Windows Media

      • High bit rate

        Windows Media – KEXP 1.4Mb live stream

        Optivision – RC 5.6Mb live stream to remote sites

        MPEG2 – Concurrent Computer Xstreme 5.6Mb VoD

        MPEG4 – Looking at IBM Videocharger – interface issues

        HD – In-house (>270Mb), other vendor lower bit rates


    Mediaweb project status

    MediaWeb Project Status

    • Prototype operational with simple search

      • Geographically separated/mirrored server environment

      • Video: RC/UWTV content (~700 hrs MPEG2, lower bit rates)

      • Audio: Sample audio

  • Refining schema

    • Object experiments

    • Imbedded controls (MPEG7)

    • Reviewing large vendor schemas

    • Deploying admin tools

  • Building feed systems (RC, +35,000 disc KEXP library)

  • Incorporating faculty/student mgt (expiration, quotas, …)

  • Evaluating other large content requests

  • Ongoing evaluation, scaling, benchmarking architecture

  • Looking for opportunities to collaborate!


  • Experiments

    Experiments

    • UW CSci animation teaching tool

      “How Networks Change Teaching: A Case Study”

      Louis Fox, Kris Martin, Barbara Mones-Hattal

      I2 VIMM Friday, October 5th 1:30EDT

    • KEXP live stream archives

    • KEXP automated playlist

    • EMP live webcasting and archival

    • UW Catalyst collection interface

    • HD IP Videoconferencing


  • Login