1 / 37

MPEG-4 streams

MPEG-4 streams. Comp- ress. D E L I V E R Y I N T E R F A C E (DAI). D E L I V E R Y I N T E R F A C E (DAI). C O M P O S I T O R. decomp- ress. Comp- ress. decomp- ress. Comp- ress. decomp- ress. Comp- ress. decomp- ress. decomp-

cutler
Download Presentation

MPEG-4 streams

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MPEG-4 streams Comp- ress D E L I V E R Y I N T E R F A C E (DAI) D E L I V E R Y I N T E R F A C E (DAI) C O M P O S I T O R decomp- ress Comp- ress decomp- ress Comp- ress decomp- ress Comp- ress decomp- ress decomp- ress Scene Des. Comp- ress Obejct-des

  2. Synchronization of elementary streams with the scene

  3. Synchronization of multiple elementary streams • Based on two well known concepts • _______ references • Convey the speed of the encoder clock • _________ stamps • Convey the time at which an event should happen • Time stamps and clock references are • defined in the _____________ model • conveyed on the ______ layer

  4. System Decoder Model

  5. Object-based compression and delivery D E L I V E R Y I N T E R F A C E (DAI) D E L I V E R Y I N T E R F A C E (DAI) C O M P O S I T O R Comp- ress decomp- ress Comp- ress decomp- ress Comp- ress decomp- ress Comp- ress decomp- ress decomp- ress Comp- ress Scene Des.

  6. System Decoder Model D E L I V E R Y I N T E R F A C E (DAI) C O M P O S I T O R Decoder Buffer DB1 Composition Memory CM1 decoder Decoder Buffer DB1 Composition Memory CM1 decoder Decoder Buffer DB1 Composition Memory CM1 decoder Decoder Buffer DBn Composition Memory CMn decoder

  7. System Decoder Model • Ideal model of the decoder behavior • Instantaneous ______________ • Incorporates the__________ model • Decoding & composition time • Manage decoder buffer resources • Useful for the _______ • Ignores delivery ______ • Designed for a rate-controlled “______” scenario • Applicable also to flow-controlled “pull” scenario • Defines composition memory (CM) behavior • A random access memory to the current composition unit • CM resource management not implemented

  8. So, how does synchronization work?

  9. Synchronization of elementary streamswith time events in the scene description • How are time events handled in the scene description? • How is this related to time in the elementary streams? • Which time base is valid for the scene description?

  10. Time line (“object time base”) for the scene • Scene description stream with time stamped BIFS access units • Object descriptor stream with pointers to all other streams • Video stream with (decoding & composition) time stamps • Audio stream with (decoding & composition) time stamps • Alternate time line for audio and video

  11. Where are time stamps carried?

  12. The synchronization layer • Synchronization layer (short: sync layer or SL) • SL packet = one packet of data • consists of header and __________ • Indicates boundaries of __________ units • AccessUnitStartFlag, AccessUnitEndFlag, AULength • Provides consistency checking for ____ packets • Carries object _______ reference (OCR) stamps • Carries decoding and composition time stamps (DTS, CTS)

  13. Elementary Stream Interface Elementary stream Access Unit AU Access Unit AU SL-Packetized stream Compression Layer Sync Layer

  14. Elementary Stream Interface Elementary stream Access Unit AU Access Unit AU Elementary stream Access Unit Access Unit Elementary stream AU AU AU AU AU AU

  15. Elementary Stream Interface • Access unit _________ • Random access points • Desired __________ time • The _________ time This guy knows about E N C O D E R Elementary stream Access Unit Access Unit

  16. Elementary Stream Interface E N C O D E R Interface may be hidden in the Encoder Access Unit Access Unit Access Unit

  17. How is the sync layer designed?

  18. The sync layer design Access units are conveyed in SL packets SL Packet SL Packet SL Packet

  19. The sync layer design Access units may use more than one SL packet SL Packet SL P SL P SL P SL Packet SL Packet

  20. The sync layer design SL packets have a header to encode the information conveyed through the ESI SL Packet SL P SL P SL P SL Packet SL Packet SL packets that don’t start an AU have a smaller header

  21. How is the sync layer designed? • As flexible as possible to be suitable for • a wide range of ______________ • a wide range of different media __________s • Time stamps have • variable length • variable _______________ • Same for clock reference (OCR) values • OCR may come via another stream • Alternative to time stamps exists for lower bitrate • Indication of ______ time and • ________ of units (accessUnitDuration, compositionUnitDuration)

  22. How is the sync layer flexibility signaled? • SLConfigDescriptor within ES_Descriptor • Determines all variable parameters • Allows to select from predefined _________s •  Must be known to parse an SL-packetized stream

  23. SLConfigDescriptor syntax example class SLConfigDescriptor { uint (8) predefined; if (predefined==0) { bit(1) useAccessUnitStartFlag; bit(1) useAccessUnitEndFlag; bit(1) useRandomAccessPointFlag; bit(1) usePaddingFlag; bit(1) useTimeStampsFlag; uint(32) timeStampResolution; SDL- Syntax Description Language uint(32) OCRResolution; uint(6) timeStampLength; uint(6) OCRLength; if (!useTimeStamps) { ................

  24. How to ship the SL packets?

  25. Wrap SL packets in a suitable layer! e.g. FlexMux SL Packet SL Packet SL Packet e.g. RTP/UDP/IP SL Packet SL Packet SL Packet e.g. MPEG-2 TS SL Packet SL Packet SL Packet

  26. MPEG-4 stream delivery (a sneak preview to DMIF ;-)

  27. MPEG-4 delivery framework

  28. The delivery layer • The “delivery layer” is an abstraction to refer generically to • any ________ multiplex (a. k. a. file format) • any transmission __________ • and the associated control functionality • The DMIF (Delivery Multimedia Integration Framework) part of MPEG-4 describes the _________ to any multiplex (DAI - DMIF Application Interface) • Adaptations of MPEG-4 to, e.g., RTP, MPEG-2, H.223 are being defined by the responsible group (with MPEG help)

  29. Adaptation of the sync layer syntax • “Wrapping” of SL packets was the original design goal • MPEG-2 TS adaptation (Amendment 7 to MPEG-2 Systems)and FlexMux encapsulation really work like that • RTP encapsulation still in development • “Payload formats” specify the adaptation • They may use “Header compression”, i.e., • Syntax elements of SL are replaced by their RTP counterparts • e.g., composition time stamp --> RTP time stamp • The MPEG-4 file format stores elementary streams • conversion to SL-packetized streams during playout (if needed)

  30. Multiplex of elementary streams

  31. Multiplex of elementary streams • Not a core MPEG task • Just respond to specific needs for MPEG-4 content ______________ • Low delay • Low overhead • Low complexity • This prompted the design of the “___________” tool • One ___________ file format desirable • This lead to the design of the MPEG-4 file format

  32. The MPEG-4 content access procedure

  33. MPEG-4 content access procedure • Locate an MPEG-4 content item (e.g. by URL) and connect to it • How? • Via the _______________________ • Reminder: This abstraction is being mapped to actual signaling protocols (RTSP, MPEG-2 PSI, etc.) • Retrieve the _____________________ • This Object Descriptor points to _____ + _____________ • Open these streams via DAI • The Scene Description points to other streams through __________ Descriptors • Open the required streams via DAI • Start playing!

  34. A second attempt – more graphics

More Related