Cs 414 multimedia systems design lecture 25 synchronization issues
This presentation is the property of its rightful owner.
Sponsored Links
1 / 31

CS 414 – Multimedia Systems Design Lecture 25 – Synchronization Issues PowerPoint PPT Presentation


  • 113 Views
  • Uploaded on
  • Presentation posted in: General

CS 414 – Multimedia Systems Design Lecture 25 – Synchronization Issues. Klara Nahrstedt Spring 2012. Administrative. MP2 posted MP2 Deadline – April 7, Saturday, 5pm. Internet Multimedia Protocol Stack. Media encaps (H.264, MPEG-4). Synchronization Service. DASH. APPLICATION. RTSP.

Download Presentation

CS 414 – Multimedia Systems Design Lecture 25 – Synchronization Issues

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cs 414 multimedia systems design lecture 25 synchronization issues

CS 414 – Multimedia Systems DesignLecture 25 – Synchronization Issues

Klara Nahrstedt

Spring 2012

CS 414 - Spring 2012


Administrative

Administrative

MP2 posted

MP2 Deadline – April 7, Saturday, 5pm.

CS 414 - Spring 2012


Internet multimedia protocol stack

Internet Multimedia Protocol Stack

Media encaps

(H.264, MPEG-4)

Synchronization Service

DASH

APPLICATION

RTSP

RSVP

RTCP

Layer 5

(Session)

SIP

RTP

HTTP

Layer 4

(Transport)

TCP

DCCP

UDP

KERNEL

Layer 3

(Network)

IP Version 4, IP Version 6

AAL3/4

AAL5

MPLS

Layer 2

(Link/MAC)

Ethernet/WiFi

ATM/Fiber Optics

CS 414 - Spring 2012


Outline

Outline

  • Synchronization in Distributed Environments

  • Clock Synchronization

  • Synchronization Reference Model

    • Media, Stream, Object, Specification Levels

CS 414 - Spring 2012


Synchronization in distributed environments

Synchronization in Distributed Environments

  • Information of synchronization must be transmitted with audio and video streams, so that receiver(s) can synchronize streams

  • Sync information can be delivered before start of presentation (used by synthetic synchronization)

    • Advantage: simple implementation

    • Disadvantage: presentation delay

  • Sync information can be delivered using separate sync channel - out-band (used by live synchronization)

    • Advantage: no additional presentation delay

    • Disadvantage: additional channel needed

CS 414 - Spring 2012


Sync in distributed environments

Sync in Distributed Environments

  • Sync information can be delivered using multiplexed data streams - in-band sync

    • Advantage: related sync information is delivered together with media units

    • Disadvantage: difficult to use for multiple sources

CS 414 - Spring 2012


Location of sync operation

Location of Sync Operation

  • Sync media objects by combining objects into new media object

  • Sync operation placed at sink

    • Demand on bandwidth is larger because additional sync operations must be transported

  • Sync operation placed at source

    • Demand on bandwidth smaller because streams are multiplexed according to sync requirements

CS 414 - Spring 2012


Clock synchronization

Clock Synchronization

  • Sync accuracy depends on clocks at source and sink nodes

    • Ta = Tav – Nla – Oa

    • Tv = Tav – Nlv – Ov

  • End-to-end delay

    • Nla = EEDa = Tav-Ta-Oa

    • Nlv=EEDv = Tav-Tv-Ov

    • EEDa = (Ta1-Ta2)/2

  • NTP (Network Time Protocol )

CS 414 - Spring 2012


Av skew calculation

AV Skew Calculation

  • Select synchronization point in AV streams in terms of frame number for each stream

  • Execute sync operation at sync points

    1. Measure

    • Arrival time of sync audio packet TsyncA

    • Arrival time of sync video packet TsyncV

      2. Calculate:

    • AVSkew = TsyncA – TsyncV

      3. Alternative measure and calculate

    • TsyncA = TsendA + EEDA; TsyncV = TsendV + EEDV

    • EED = RTT/2 = (Trecv – Tsend)/2

CS 414 - Spring 2012


Network time protocol

Network Time Protocol

  • Protocol to sync clocks of computer systems over packet-switched, variable –latency data networks

    • Uses UDP port 123

    • Designed to resist effects of variable latency (jitter buffer)

    • Designed in 1985 by Dave Mills at U. Delaware

    • Can achieve accuracy of 200 µsec

    • Based on Marzullo Algorithm

CS 414 - Spring 2012


Marzullo s algorithm 1984 intersection algorithm

Marzullo’s algorithm (1984)(intersection algorithm)

  • Agreement protocol for estimating accurate time from a number of noisy time sources

  • If we have estimates

  • 10 2, 12 1, 11 1, then interval intersection is 11.5 0.5

  • If some intervals don’t intersect, consider intersection of majority of intervals

CS 414 - Spring 2012

Source: wikipedia


Clock strata

Clock strata

  • NTP uses hierarchical system of “clock strata”

  • Stratum levels define distance from reference clock and exist to prevent cycles in hierarchy

    • Stratum 0

      • devices are atomic clocks, GPS clocks, radio clocks

    • Stratum1

      • computers attached to stratum0 devices

      • Act as servers for timing requests from Stratum 2 servers via NTP

    • Stratum2

      • (similar to Stratum1, but they also have peering relation to other stratum2 servers

CS 414 - Spring 2012


Precision time protocol ptp

Precision Time Protocol (PTP)

  • High Precision time protocol (original IEEE 1588-2002) for synchronization used in measurement and control systems residing on a local area network

  • Accuracy in the sub-microseconds

  • PTPv2 – IEEE 1588-2008 improves accuracy, precision and robustness

  • PTP serves a niche not well served by NTP and GPS

    • it achieves accuracy beyond those attainable using NTP;

    • It is cheaper than GPS

    • It is accessible to receivers for which GPS signals are inaccessible.

CS 414 - Spring 2012


Ptp details

PTP Details

  • IEEE 1588 standards describe a hierarchical master-slave architecture for clock distribution

    • Ordinary clock – source for root timing reference, called ‘grandmaster’

    • Boundary clock – intermediate synchronization bridge from one network segment to another – used by synchronization master to sync clocks within a network segment

  • PTP messages use IP-multicast over UDP (IPv4 or IPv6)

  • Selection algorithm of best clock source uses properties such as quality, accuracy, variance, …

CS 414 - Spring 2012


Ptp synchronization

PTP Synchronization

  • After selection of master source of time, clocks determine the offset between themselves and their master

    • Let t represent physical time, given slave device, the offset o(t)at time t is: o(t) = s(t) – m(t),

      • m(t) is measured time at master at time t, and

      • s(t) is measured time at local clock at time t.

    • Master periodically broadcasts the current time to other clocks (up to 10 measurements per second in 1588-2008)

CS 414 - Spring 2012


Ptp synchronization1

PTP Synchronization

T1’ – T1 = o + dand T2’ – T2 = -o + d, then we find

o = (T1’ - T1 – T2’ + T2)/2

The clock now knows offset o during the transactions and can correct

itself by this agreement with the master clock.

CS 414 - Spring 2012


Other sync issues

Other Sync Issues

  • Sync must be considered during object acquisition

  • Sync must be considered during retrieval

    • Sync access to frames of stored video

  • Sync must be considered during transport

    • If possible use isochronous protocols

  • Sync must be considered at sink

    • Sync delivery to output devices

  • Sync must consider support of functions such as pause, forward, rewind with different speeds, direct access, stop or repeat

CS 414 - Spring 2012


Other sync issues1

Other Sync Issues

  • Sync must be considered during object acquisition

  • Sync must be considered during retrieval

    • Sync access to frames of stored video

  • Sync must be considered during transport

    • If possible use isochronous protocols

  • Sync must be considered at sink

    • Sync delivery to output devices

  • Sync must consider support of functions such as pause, forward, rewind with different speeds, direct access, stop or repeat

CS 414 - Spring 2012


Reference models

Reference Models

  • We need reference models to

    • Understand various requirements for multimedia sync

    • Identify and structure run-time mechanisms to support execution of sync

    • Identify interface between run-time mechanisms

    • Compare system solutions for multimedia sync

CS 414 - Spring 2012


Synchronization reference model

Synchronization Reference Model

  • Sync model we will be evaluating in detail is according to Meyer, Effelsberg, Steinmetz:

    • Sync multimedia objects are classified according to

      • Media level

      • Stream level

      • Object level

      • Specification level

CS 414 - Spring 2012


Media level 1

Media Level (1)

  • Each application operates single continuous media streams composed of sequence of LDUs

  • Each media stream responsible for intra-stream synchronization

  • Assumption at this level: device independence

  • Supported operations at this level:

    • read(devicehandle, LDU)

    • write(devicehandle, LDU)

CS 414 - Spring 2012


Media level 2 example

Media Level (2) - Example

window = open(“videodevice”);

movie = open(“file”);

while (not EOF (movie) ) {

read(movie, &LDU);

if (LDU.time == 20)

printf(“Subtitle 1”);

else if (LDU.time == 26)

printf(“Subtitle2”);

write(window, LDU); }

close(window);

close(movie);

CS 414 - Spring 2012


Stream level 1

Stream Level (1)

  • Operates on continuous media streams and groups of streams

  • Models inter-stream synchronization for need of parallel presentation

  • Offers abstractions:

    • notion of streams,

    • timing parameters concerning QoS for intra-stream and inter-stream synchronization

CS 414 - Spring 2012


Stream level 2

Stream Level (2)

  • Supports operations:

    • Start(stream), stop(stream), create-group(list-of-streams);

    • Start(group), stop(group);

    • Setcuepoint(stream/group, at, event);

  • Classifies implementation according to

    • Support for distribution (end-to-end, local)

    • Support of type of guarantees (best effort, deterministic)

    • Support of types of supported streams (analog, digital)

CS 414 - Spring 2012


Object level 1

Object Level (1)

  • Operates on all types of media and hides differences between discrete and continuous media

  • Offers abstractions:

    • Complete sync presentation

  • Computes and executes complete presentation schedules that include presentation of non-continuous media objects and calls to stream level

  • Does not handle intra-stream and inter-stream synchronization

    • (relies on media and stream levels)

CS 414 - Spring 2012


Object level 2 example

Object Level (2) - Example

  • MHEG – Multimedia Hypermedia Experts Group of ISO

    • Defines representation and encoding of multimedia and hypermedia objects (object-based declarative programming language)

    • Provides abstractions suited to real-time presentations

      • implemented via multimedia synchronization functionalities

    • Provides abstracts for real-time exchange

      • implemented with minimal buffering

    • Evaluates status of objects and performs actions (e.g., prepare, run, stop, destroy)

      • For time-dependent streams – access to stream level

      • For time-independent streams – direct access the object to present it

  • Classification of this level according to (a) distribution capabilities, (b) type of presentation schedule, (c) schedule calculation

CS 414 - Spring 2012


Specification level 1

Specification Level (1)

  • Open layer included in tools which allow to create sync specifications

  • Examples:

    • Synchronization editors, document editors, authoring systems, conversion tools

    • Examples of such tools: multimedia document formatter that produces MHEG specifications

  • Classification:

    • Interval-based spec

    • Time-axes based spec

    • Control flow-based spec

    • Event-based spec

CS 414 - Spring 2012


Specification level 2 smil

Specification Level (2) - SMIL

  • SMIL – World Wide Web Consortium’s Synchronized Multimedia Integration Language

  • SMIL 2.0 and SMIL 3.0 (developed by CWI, PI: D. Bulterman)

  • Features: Animations, Time Manipulations, Timing and Synchronization, Content Control, Meta-information, Media objects, Linking, Layout, Transitions

  • http://www.w3.org/TR/SMIL3/

CS 414 - Spring 2012


Cs 414 multimedia systems design lecture 25 synchronization issues

SMIL

“Standards, SMIL 2.0, Examples and Comparisons”, D. Bulterman, Siemens

Corporate research, Editor: P. Liu, 2002

CS 414 - Spring 2012


Summary

Summary

CS 414 - Spring 2012


Conclusion

Conclusion

Carefully analyze what kind of synchronization is needed in your multimedia system and application

Determine at which level you need synchronization

Determine what the synchronization requirements should be based on prior experiments

CS 414 - Spring 2012


  • Login