1 / 34

ARMOR - A System for Adjusting Repair and Media Scaling for Video Streaming

ARMOR - A System for Adjusting Repair and Media Scaling for Video Streaming. Huahui Wu, Mark Claypool and Robert Kinicki Elsevier Journal of Visual Communication and Image Representation (JVCIR) Volume 19, Number 8, Pages 489-499 December 2008. Introduction - Motivation. Video Frames.

hiero
Download Presentation

ARMOR - A System for Adjusting Repair and Media Scaling for Video Streaming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ARMOR - A System for Adjusting Repair and Media Scaling for Video Streaming Huahui Wu, Mark Claypool and Robert Kinicki Elsevier Journal of Visual Communication and Image Representation (JVCIR) Volume 19, Number 8, Pages 489-499 December 2008

  2. Introduction - Motivation Video Frames Repair by Forward Error Correction (FEC)

  3. Adjusting Repair and Media Scaling Given Network and Application Environment For each valid FEC and scaling combination, measure the video quality Find the optimal point Optimal Points Video Quality More Repair and More Scaling Operations Research Concept

  4. Outline • Introduction • Background • Models • Algorithms • User Study • Implementation • Conclusions

  5. Video Compression Standard • MPEG • Popular compression standard • Intra-compression and inter-compression • Three types of frames: I, P and B • Group Of Pictures (GOP) • ARMOR models MPEG dependencies

  6. Forward Error Correction (FEC) • Media-Independent FEC • Reed-Solomon codes [Reed+ 60] • ARMOR models benefits of FEC for frame transmission

  7. Media Scaling (1 of 2) • Sacrifice data to fit the capacity • Temporal Scaling (TS) • Pre-Encoding Temporal Scaling • Post-encoding Temporal Scaling (below )

  8. Media Scaling (2 of 2) • Quality Scaling • MPEG uses quantization in coding to save bits • Quantization Value (1~31) • For example: original data = 23, 13, 7, 3 • ARMOR models both Temporal Scaling and Quality Scaling

  9. Video Quality Measurements • Subjective Measurement • User study, expensive, not practical • Objective Measurements • Playable Frame Rate (R) • Good for Temporal Scaling, not for Quality Scaling • Peak Signal Noise Ratio (PSNR) • Good for Quality Scaling, not for Temporal Scaling • Video Quality Metric (VQM) • By Institute for Telecommunication science • Extracts 7 perception-based features • Only one for frame losses • Report a distortion value from 0 (no distortion) to 1 (many) • ARMOR uses both R and VQM • Includes a user study

  10. Outline • Introduction • Background • Models • Streaming Bitrate Model (cost) • Video Quality Model (benefit) • Algorithms • User Study • Implementation • Conclusions

  11. Parameters and Variables Video Frames Repair by Forward Error Correction (FEC)

  12. Streaming Bitrate Model • Total streaming bitrate (B), including video packets and FEC packets: where G is the constant GOP rate NPD and NBD are the numbers of transmitting P and B frames depending on Temporal Scaling level lTS

  13. Video Quality Model - Overview • Two distortion factors • Frame Loss • Caused by Temporal Scaling and network packet loss • Appears jerky in the video playout • Measured by Playable Frame Rate • Quantization Distortion • Caused by a high quantization value with Quality Scaling • Appears visually as coarse granularity in every frame • Measured by VQM • Overall Quality • Distorted Playable Frame Rate [20]

  14. Playable Frame Rate (R) • Frame Successful Transmission Probability • Where Frame Size • Frame Dependencies • Total Playable Frame Rate

  15. Distorted Playable Frame Rate (RD ) • Quality scaling distortion varies exponentially with the quantization level • Distorted Playable Frame Rate [4]

  16. ARMOR Algorithm • For each Repair and Scaling combination • Estimate video frame sizes (SI, SP, SB) • Compute streaming bitrate B and make sure it’s under capacity constraint T • Use frame sizes and FEC amount to get successfully frame transmission rate (qI, qP, qB) • Compute playable frame rate (R) • Estimate quality scaling distortion (D) • Compute distorted playable frame rate (RD) • Exhaustively search all FEC and Scaling combinations and find optimal quality

  17. Outline • Introduction • Background • Models • Algorithms • User Study  • Implementation • Conclusions

  18. User Study Goals • Accuracy of RD • Correlation with user perceptual quality • Versus PSNR and VQM? • Temporal Scaling versus Quality Scaling • What are the differences? • Adjusted Repair (FEC) versus No Repair • Is Adjusted Repair an effective method for increasing perceptual quality?

  19. Video Clips • Compare degraded clips to the original • Original: 30 fps, no quality scaling • Degraded: Combinations of 4 independent factors (2 options each) • Video and Network environment • Video content: low motion (News)http://www.youtube.com/watch?v=uN3yUm0WZwYor high motion (Coastguard) http://www.youtube.com/watch?v=JQVclZWH5UM • Packet loss rate: low loss (1%)or high loss (4%) • ARMOR Layer • Repair: adjusted repair or no repair • Scaling: Quality Scaling or Temporal Scaling • 24 = 16 combinations for evaluation

  20. User Study Application 54321 • Two-week volunteer study • 74 users, most CS undergraduate students (male, young) • Most LCD, high-rez monitor Double Stimulus Impairment Scale (DSIS) [ITU-R BT.500-11]

  21. Results – Video Quality Metrics (1 of 3) (Same as original clip) User Score vs. PSNR (Much worse than original clip)

  22. Results – Video Quality Metrics (2 of 3) User Score Vs. VQM Score (1 – VQM distortion)

  23. Results – Video Quality Metrics (3 of 3) User Score vs. Distorted Playable Frame Rate (RD)

  24. RD 30.0 22.5 15.0 7.5 0.0 Results – Scaling Methods Temporal Scaling versus Quality Scaling ARMOR Prediction (Coastguard) User Score

  25. RD 30.0 22.5 15.0 7.5 0.0 Results – Repair Methods Adjusted Repair versus No Repair User Score ARMOR Prediction (Coastguard)

  26. Outline • Introduction • Background • Models • Algorithms • User Study • Implementation  • Conclusions

  27. Implementation Goals • Provide architecture for ARMOR system • Validate ARMOR model • Determine if can make improvements to video quality in real-time

  28. Architecture 3 3 2 2 1 2 3 4 1 8 7 6 5

  29. Experiment Settings • Video clip Paris • http://www.youtube.com/watch?v=XU74KL_72RA • medium motion and details • two people sitting, talking, with high-motion gestures • 1200 CIF (352x288) images • average I / P / B frame sizes: 24.24 KB / 5.20 KB / 1.18 KB

  30. RD RD ARMOR Analytical Results ARMOR Measurement Results Results

  31. Conclusions • Distorted playable frame rate high correlation with user perceptual quality • Higher than PSNR or VQM • Adjusting repair improves video streaming quality significantly • Better than fixed repair and no repair • Quality Scaling is more effective than Temporal Scaling • But when bandwidth low and network loss high, Quality Scaling should be used with Temporal Scaling • Proof of concept  ARMOR can be implemented in real-time to effectively improve streaming quality

  32. Future Work?

  33. Future Work • Implementation of quality scaling • Implementation of streaming media protocols • Bandwidth estimation techniques for initial streaming rate • Alternative repair techniques • Evaluate with time-varying bandwidth and packet loss • Classification of video motion and scene complexity to predict exponential coefficients • User studies to determine if RD works for different scaling combinations

  34. ARMOR - A System for Adjusting Repair and Media Scaling for Video Streaming Huahui Wu, Mark Claypool and Robert Kinicki Elsevier Journal of Visual Communication and Image Representation (JVCIR) Volume 19, Number 8, Pages 489-499 December 2008

More Related