1 / 28

Error Concealment

Error Concealment. CMPT 820 October 19 2010. Presented by: Cameron Harvey. Internet Basics. The internet is not reliable as a c ommunication channel Packets can become corrupted Packets can get lost Packets can have high latency Packets can arrive out of order. Encoder/decoder.

aislin
Download Presentation

Error Concealment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Error Concealment CMPT 820 October 192010 Presented by: Cameron Harvey

  2. Internet Basics The internet is not reliable as a communication channel Packets can become corrupted Packets can get lost Packets can have high latency Packets can arrive out of order

  3. Encoder/decoder MULTIMEDIA OVER IP AND WIRELESS NETWORKS, pg 16 FIGURE 2.4

  4. Motion Compensated Prediction • Standard compression techniques use MCP to estimate a macroblock based on previous or future frames • Packet loss will affect not just the frame being decoded, but many other frames as well • Errors are propagated temporally until a new I-frame is encountered

  5. Slices • Slices • A slice is composed of a series of macroblocks • Slices are non-overlapping • Slices are structure in such a way that motion vectors are all contained within the slice

  6. Slice Coding Schemes • Constant MBs/slice • Maximum bytes/slice

  7. Macroblock Allocation Maps • FMO – Flexible Macro-block Ordering • New to the H.264/AVC standard is the ability to define slice groups • Groups can be further divided into separate slices

  8. Flexible Macroblock Ordering • FMO provides a useful tool for error concealment • We can use FMO to arrange for slice groups in such a way that macroblocksare not adjacent to macroblocks from the same slice group • Slices are packetized separately and transmitted

  9. Some types of FMO patterns Y Dhondt, P Lambert, Flexible Macroblock Ordering – an error resilience tool in H.264/AVC, Gent, Belgium, Dec 2004

  10. FMO – Type 1 • Slices are interleaved in such a way that a missing slice will be surrounded by many samples which are decoded correctly MULTIMEDIA OVER IP AND WIRELESS NETWORKS, pg 31 FIGURE 2.12

  11. Example • Suppose the packet containing macroblocks of slice group #0 is lost

  12. Example (2) • The missing macroblock can be interpolated using the data from the surrounding macroblocks • This is type of concealment is called Spatial Concealment T L R B

  13. Spatial Error Concealment • The key assumption behind Spatial Error Concealment is that image content is continuous and has smooth texture

  14. Spatial Error Concealment • The pixel, Xij, is calculated as the weighted average of four pixel values on the edge of the macroblock MULTIMEDIA OVER IP AND WIRELESS NETWORKS, pg 33 FIGURE 2.13

  15. Spatial Error Concealment (2) • The calculation for pixel (i,j) is • The values of determine the impact the border pixels will have and are generally related to the distance from the pixel under consideration

  16. Spatial Error Concealment (3) • This technique will result in blurred areas as the reconstructed macroblock is basically an average of the surrounding macroblocks • Refinements of this technique can be used to recover some texturing information

  17. Spatial Error Concealment (4) Picture 11 from a GOP – 5% packet loss Error concealment without FMO Error concealment using FMO Y Dhondt, P Lambert, Flexible Macroblock Ordering – an error resilience tool in H.264/AVC, Figure 2, Gent, Belgium, Dec 2004

  18. Temporal Error Concealment • Temporal concealment operates with the assumption that a video sequence is continuous • In its simplest form, one can simply copy a missing macroblock from the corresponding macroblock in the previous frame. This is called Previous Frame Concealment (PFC) • Useful if the scene is static • Problematic if the scene changes rapidly as this creates many artifacts

  19. Temporal Error Concealment (2) • We can improve on PFC with an assumption of uniform motion. With this assumption, one can examine the motion vectors of the adjacent macroblocks to infer the motion of the lost macroblock • Many possibilities exist. A boundary matching technique can be used to select from the many possibilities

  20. Boundary Matching Technique • The sum of the squares of the differences between border pixels of Top, Right, and Bottom is determined • The motion vector which minimizes this sum is a good candidate MULTIMEDIA OVER IP AND WIRELESS NETWORKS, pg 35 FIGURE 2.15

  21. Other Temporal Concealment Refinements • Overlapped Block Motion • The average of three 16X16 pixel regions is used • Use Median Motion Vector of T, R, and B • Sum of Absolute Difference (instead of square) • Estimate motion with the Top macroblock • Use PFC if top motion vector is unavailabel • Multi-hypothesis error concealment uses a weighted average of multiple frames

  22. Example • Click to see an example of Spatial-temporal error concealment • http://w3.antd.nist.gov/public_safety/concealment/Project_conceal_files/ecdemo.shtml

  23. Temporal Concealment over a Scene Change • Packet loss over a scene change violates the assumption of temporal continuity • Scene change detection can be used at the cost of additional complexity • Macroblock information over a scene change exhibit distinct characteristics

  24. Abrupt Scene Changes S. Pei and Y Chou, Novel Error Concealment Method With Adaptive Prediction to the Abrupt and Gradual Scene Changes, Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3, Figure 2(a) Apr 2002

  25. Abrupt Scene Changes (2) • Use Spatial Concealment on the P-frame (Why?) • Use Temporal Error Concealment on the B frames • Use the P or I frame from the same scene as reference • For IBBP, all four frames must be processed before concealment can occur

  26. Gradual Scene Changes S. Pei and Y Chou, Novel Error Concealment Method With Adaptive Prediction to the Abrupt and Gradual Scene Changes, Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3, Figure 2(b) Apr 2002

  27. Gradual Scene Changes (2) • In the first sub-GOP (IBBP or PBBP), temporal concealment can be used for the final P-frame (Why?) • B frames are calculated as a weighted average of the first P or I frame and the final P-frame • Across P frame in different sub-GOP, interpolation is used aided by the large number of intracodedmacroblocks

  28. Questions He must be very ignorant for he answers every question he is asked. Voltaire

More Related