Introduction to Multimedia Networking - PowerPoint PPT Presentation

crescent
introduction to multimedia networking n.
Skip this Video
Loading SlideShow in 5 Seconds..
Introduction to Multimedia Networking PowerPoint Presentation
Download Presentation
Introduction to Multimedia Networking

play fullscreen
1 / 79
Download Presentation
Introduction to Multimedia Networking
216 Views
Download Presentation

Introduction to Multimedia Networking

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Introduction to Multimedia Networking • Classify multimedia applications • Identify the network services the apps need • Making the best of best effort service • Streaming Stored Multimedia vs. Interactive Applications • Adaptive Playback and Smart Error Recovery Algorithms • Some Common Protocols: • RTSP, RTP/RTCP, SIP Required Readings: Relevant chapters/sections from your csci5211/csci4221 Textbook CSci5221: Multimedia

  2. QoS network provides application with level of performance needed for application to function. Multimedia and Quality of Service Multimedia applications: network audio and video (“continuous media”) CSci5221: Multimedia

  3. Digital Audio • Sampling the analog signal • Sample at some fixed rate • Each sample is an arbitrary real number • Quantizing each sample • Round each sample to one of a finite number of values • Represent each sample in a fixed number of bits 4 bit representation(values 0-15) CSci5221: Multimedia

  4. Audio Examples • Speech • Sampling rate: 8000 samples/second • Sample size: 8 bits per sample • Rate: 64 kbps • Compact Disc (CD) • Sampling rate: 44,100 samples/second • Sample size: 16 bits per sample • Rate: 705.6 kbps for mono, 1.411 Mbps for stereo CSci5221: Multimedia

  5. Why Audio Compression • Audio data requires too much bandwidth • Speech: 64 kbps is too high for a dial-up modem user • Stereo music: 1.411 Mbps exceeds most access rates • Compression to reduce the size • Remove redundancy • Remove details that human tend not to perceive • Example audio formats • Speech: GSM (13 kbps), G.729 (8 kbps), and G.723.3 (6.4 and 5.3 kbps) • Stereo music: MPEG 1 layer 3 (MP3) at 96 kbps, 128 kbps, and 160 kbps CSci5221: Multimedia

  6. Analog signal sampled at constant rate telephone: 8,000 samples/sec CD music: 44,100 samples/sec Each sample quantized, i.e., rounded e.g., 28=256 possible quantized values Each quantized value represented by bits 8 bits for 256 values Example: 8,000 samples/sec, 256 quantized values --> 64,000 bps Receiver converts it back to analog signal: some quality reduction Example rates CD: 1.411 Mbps MP3: 96, 128, 160 kbps Internet telephony: 5.3 - 13 kbps A few words about audio compression CSci5221: Multimedia

  7. Digital Video • Sampling the analog signal • Sample at some fixed rate (e.g., 24 or 30 times per sec) • Each sample is an image • Quantizing each sample • Representing an image as an array of picture elements • Each pixel is a mixture of colors (red, green, and blue) • E.g., 24 bits, with 8 bits per color CSci5221: Multimedia

  8. The 2272 x 1704hand The 320 x 240hand CSci5221: Multimedia

  9. Video is sequence of images displayed at constant rate e.g. 24 images/sec Digital image is array of pixels Each pixel represented by bits Redundancy spatial temporal Examples: MPEG 1 (CD-ROM) 1.5 Mbps MPEG2 (DVD) 3-6 Mbps MPEG4 (often used in Internet, < 1 Mbps) Research: Layered (scalable) video adapt layers to available bandwidth A few words about video compression CSci5221: Multimedia

  10. Video Compression: Within an Image • Image compression • Exploit spatial redundancy (e.g., regions of same color) • Exploit aspects humans tend not to notice • Common image compression formats • Joint Pictures Expert Group (JPEG) • Graphical Interchange Format (GIF) Uncompressed: 167 KB Good quality: 46 KB Poor quality: 9 KB CSci5221: Multimedia

  11. Video Compression: Across Images • Compression across images • Exploit temporal redundancy across images • Common video compression formats • MPEG 1: CD-ROM quality video (1.5 Mbps) • MPEG 2: high-quality DVD video (3-6 Mbps) • Proprietary protocols like QuickTime and RealNetworks CSci5221: Multimedia

  12. Fundamental characteristics: Typically delaysensitive end-to-end delay delay jitter But loss tolerant: infrequent losses cause minor glitches Antithesis of data, which are loss intolerant but delay tolerant. Classes of MM applications: 1) Streaming stored audio and video 2) Streaming live audio and video 3) Real-time interactive audio and video MM Networking Applications • Jitter is the variability • of packet delays within • the same packet stream CSci5221: Multimedia

  13. Application Classes • Streaming • Clients request audio/video files from servers and pipeline reception over the network and display • Interactive: user can control operation (similar to VCR: pause, resume, fast forward, rewind, etc.) • Delay: from client request until display start can be 1 to 10 seconds CSci5221: Multimedia

  14. Application Classes (more) • Unidirectional Real-Time: • similar to existing TV and radio stations, but delivery on the network • Non-interactive, just listen/view • Interactive Real-Time : • Phone conversation or video conference • More stringent delay requirement than Streaming and Unidirectional because of real-time nature • Video: < 150 msec acceptable • Audio: < 150 msec good, <400 msec acceptable CSci5221: Multimedia

  15. Streaming Stored Multimedia Streaming: • media stored at source • transmitted to client • streaming: client playout begins before all data has arrived • timing constraint for still-to-be transmitted data: in time for playout CSci5221: Multimedia

  16. 2. video sent 3. video received, played out at client 1. video recorded network delay streaming: at this time, client playing out early part of video, while server still sending later part of video Streaming Stored Multimedia: What is it? Cumulative data time CSci5221: Multimedia

  17. Streaming Stored Multimedia: Interactivity • VCR-like functionality: client can pause, rewind, FF, push slider bar • 10 sec initial delay OK • 1-2 sec until command effect OK • RTSP often used (more later) • timing constraint for still-to-be transmitted data: in time for playout CSci5221: Multimedia

  18. Streaming Live Multimedia Examples: • Internet radio talk show • Live sporting event Streaming • playback buffer • playback can lag tens of seconds after transmission • still have timing constraint Interactivity • fast forward impossible • rewind, pause possible! CSci5221: Multimedia

  19. Interactive, Real-Time Multimedia • applications: IP telephony, video conference, distributed interactive worlds • end-end delay requirements: • audio: < 150 msec good, < 400 msec OK • includes application-level (packetization) and network delays • higher delays noticeable, impair interactivity • session initialization • how does callee advertise its IP address, port number, encoding algorithms? CSci5221: Multimedia

  20. ? ? ? ? ? ? ? But you said multimedia apps requires QoS and level of performance to be effective! ? ? ? ? Today’s Internet multimedia applications use application-level techniques to mitigate (as best possible) effects of delay, loss Multimedia Over Today’s Internet TCP/UDP/IP: “best-effort service” • no guarantees on delay, loss CSci5221: Multimedia

  21. Challenges • TCP/UDP/IP suite provides best-effort, no guarantees on expectation or variance of packet delay • Streaming applications delay of 5 to 10 seconds is typical and has been acceptable, but performance deteriorate if links are congested (transoceanic) • Real-Time Interactive requirements on delay and its jitter have been satisfied by over-provisioning (providing plenty of bandwidth), what will happen when the load increases?... CSci5221: Multimedia

  22. Challenges (more) • Most router implementations use only First-Come-First-Serve (FCFS) packet processing and transmission scheduling • To mitigate impact of “best-effort” protocols, we can: • Use UDP to avoid TCP and its slow-start phase… • Buffer content at client and control playback to remedy jitter • Adapt compression level to available bandwidth CSci5221: Multimedia

  23. Integrated services philosophy: Fundamental changes in Internet so that apps can reserve end-to-end bandwidth Requires new, complex software in hosts & routers Laissez-faire no major changes more bandwidth when needed content distribution, application-layer multicast application layer Differentiated services philosophy: Fewer changes to Internet infrastructure, yet provide 1st and 2nd class service. How should the Internet evolve to better support multimedia? Will discuss QoS later! CSci5221: Multimedia

  24. Solution Approaches in IP Networks • Just add more bandwidth and enhance caching capabilities (over-provisioning)! • Need major change of the protocols : • Incorporate resource reservation (bandwidth, processing, buffering), and new scheduling policies • Set up service level agreements with applications, monitor and enforce the agreements, charge accordingly • Need moderate changes (“Differentiated Services”): • Use two traffic classes for all packets and differentiate service accordingly • Charge based on class of packets • Network capacity is provided to ensure first class packets incur no significant delay at routers CSci5221: Multimedia

  25. Application-level streaming techniques for making the best out of best effort service: client side buffering use of UDP versus TCP multiple encodings of multimedia jitter removal decompression error concealment graphical user interface w/ controls for interactivity Streaming Stored Multimedia Media Player CSci5221: Multimedia

  26. Internet Multimedia: Simplest Approach • audio or video stored in file • files transferred as HTTP object • received in entirety at client • then passed to player audio, video not streamed: • no, “pipelining,” long delays until playout! CSci5221: Multimedia

  27. Internet multimedia: Streaming Approach • browser GETs metafile • browser launches player, passing metafile • player contacts server • server streams audio/video to player CSci5221: Multimedia

  28. Streaming from a streaming server • This architecture allows for non-HTTP protocol between server and media player • Can also use UDP instead of TCP. CSci5221: Multimedia

  29. HTTP Does not target multimedia content No commands for fast forward, etc. RTSP: RFC 2326 Client-server application layer protocol. For user to control display: rewind, fast forward, pause, resume, repositioning, etc… What it doesn’t do: does not define how audio/video is encapsulated for streaming over network does not restrict how streamed media is transported; it can be transported over UDP or TCP does not specify how the media player buffers audio/video User Control of Streaming Media: RTSP CSci5221: Multimedia

  30. FTP uses an “out-of-band” control channel: A file is transferred over one TCP connection. Control information (directory changes, file deletion, file renaming, etc.) is sent over a separate TCP connection. The “out-of-band” and “in-band” channels use different port numbers. RTSP messages are also sent out-of-band: RTSP control messages use different port numbers than the media stream: out-of-band. Port 554 The media stream is considered “in-band”. RTSP: out of band control CSci5221: Multimedia

  31. RTSP Example Scenario: • metafile communicated to web browser • browser launches player • player sets up an RTSP control connection, data connection to streaming server CSci5221: Multimedia

  32. Metafile Example <title>Twister</title> <session> <group language=en lipsync> <switch> <track type=audio e="PCMU/8000/1" src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi"> </switch> <track type="video/jpeg" src="rtsp://video.example.com/twister/video"> </group> </session> CSci5221: Multimedia

  33. RTSP Operation CSci5221: Multimedia

  34. RTSP Exchange Example C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY S: RTSP/1.0 200 1 OK Session 4231 C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0- C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37 C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 S: 200 3 OK CSci5221: Multimedia

  35. client video reception constant bit rate video playout at client variable network delay buffered video client playout delay Streaming Multimedia: Client Buffering constant bit rate video transmission • Client-side buffering, playout delay compensate for network-added delay, delay jitter Cumulative data time CSci5221: Multimedia

  36. Streaming Multimedia: Client Buffering “constant” drain rate, d variable fill rate, x(t) • Client-side buffering, playout delay compensate for network-added delay, delay jitter buffered video CSci5221: Multimedia

  37. Streaming Multimedia: UDP or TCP? UDP • server sends at rate appropriate for client (oblivious to network congestion !) • often send rate = encoding rate = constant rate • then, fill rate = constant rate - packet loss • short playout delay (2-5 seconds) to compensate for network delay jitter • error recover: time permitting TCP • send at maximum possible rate under TCP • fill rate fluctuates due to TCP congestion control • larger playout delay: smooth TCP delivery rate • HTTP/TCP passes more easily through firewalls CSci5221: Multimedia

  38. Streaming Multimedia: client rate(s) 1.5 Mbps encoding 28.8 Kbps encoding Q:how to handle different client receive rate capabilities? • 28.8 Kbps dialup • 100Mbps Ethernet A:server stores, transmits multiple copies of video, encoded at different rates CSci5221: Multimedia

  39. PC-2-PC phone instant messaging services are providing this, e.g. Yahoo messenger, googletalk Skype PC-2-phone Dialpad Net2phone Skype videoconference with Webcams Going to now look at a PC-2-PC Internet phone example in detail Real-time interactive applications CSci5221: Multimedia

  40. Voice Over IP (VoIP) • Delivering phone calls over IP • Computer to computer • Analog phone to/from computer • Analog phone to analog phone • Motivations for VoIP • Cost reduction • Simplicity • Advanced applications • Web-enabled call centers • Collaborative white boarding • Do Not Disturb, Locate Me, etc. • Voicemail sent as e-mail CSci5221: Multimedia

  41. 7040 External line 212-8538080 7041 Corporate/Campus Telephone switch Another switch Private Branch Exchange 7042 7043 Internet Corporate/Campus LAN Traditional Telecom Infrastructure CSci5221: Multimedia

  42. Corporate/Campus Another campus 7040 8151 External line 8152 7041 PBX PBX 8153 VoIP Gateway 8154 7042 VoIP Gateway 7043 Internet LAN LAN IP Phone Client VoIP Gateways CSci5221: Multimedia

  43. VoIP With an Analog Phone • Adapter • Converts between analog and digital • Sends and receives data packets • Communicates with the phone in standard way CSci5221: Multimedia

  44. Skype • Niklas Zennström and Janus Friis in 2003 • Developed by KaZaA • Instant Messenger (IM) with voice support • Based on peer-to-peer (P2P) networking technology CSci5221: Multimedia

  45. Skype Network Architecture • Login server is the only central server (consisting of multiple machines) • Both ordinary host and super nodes are Skype clients • Any node with a public IP address and having sufficient resources can become a super node • Skype maintains their own super nodes CSci5221: Multimedia

  46. Challenges of Firewalls and NATs • Firewalls • Often block UDP traffic • Usually allow hosts to initiate connections on port 80 (HTTP) and 443 (HTTPS) • NAT • Cannot easily initiate traffic to a host behind a NAT • … since there is no unique address for the host • Skype must deal with these problems • Discovery: client exchanges messages with super node • Traversal: sending data through an intermediate peer CSci5221: Multimedia

  47. Interactive Multimedia: Internet Phone Introduce Internet Phone by way of an example • speaker’s audio: alternating talk spurts, silent periods. • 64 kbps during talk spurt • pkts generated only during talk spurts • 20 msec chunks at 8 Kbytes/sec: 160 bytes data • application-layer header added to each chunk. • Chunk+header encapsulated into UDP segment. • application sends UDP segment into socket every 20 msec during talkspurt. CSci5221: Multimedia

  48. Internet Phone: Packet Loss and Delay • network loss: IP datagram lost due to network congestion (router buffer overflow) • delay loss: IP datagram arrives too late for playout at receiver • delays: processing, queueing in network; end-system (sender, receiver) delays • typical maximum tolerable delay: 400 ms • loss tolerance: depending on voice encoding, losses concealed, packet loss rates between 1% and 10% can be tolerated. CSci5221: Multimedia

  49. client reception constant bit rate playout at client variable network delay (jitter) buffered data client playout delay Delay Jitter constant bit rate transmission • Consider the end-to-end delays of two consecutive packets: difference can be more or less than 20 msec Cumulative data time CSci5221: Multimedia

  50. Internet Phone: Fixed Playout Delay • Receiver attempts to playout each chunk exactly q msecs after chunk was generated. • chunk has time stamp t: play out chunk at t+q . • chunk arrives after t+q: data arrives too late for playout, data “lost” • Tradeoff for q: • large q: less packet loss • small q: better interactive experience CSci5221: Multimedia