Digital video compression fundamentals and standards
This presentation is the property of its rightful owner.
Sponsored Links
1 / 53

Digital Video Compression Fundamentals and Standards PowerPoint PPT Presentation


  • 59 Views
  • Uploaded on
  • Presentation posted in: General

Digital Video Compression Fundamentals and Standards. Web Technology. Outline. Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions. Outline. Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions.

Download Presentation

Digital Video Compression Fundamentals and Standards

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Digital video compression fundamentals and standards

Digital Video Compression Fundamentals and Standards

Web Technology


Outline

Outline

  • Introduction

  • Video Compression Standards

  • Simulation Reference Software

  • Future Work and Conclusions

Digital Video Compression Fundamentals and Standards


Outline1

Outline

  • Introduction

  • Video Compression Standards

  • Simulation Reference Software

  • Future Work and Conclusions

Digital Video Compression Fundamentals and Standards


Introduction 1 2

Introduction (1/2)

  • Why video compression technique is important ?

  • One movie video without compression

    • 720 x 480 pixels per frame

    • 30 frames per second

    • Total 90 minutes

    • Full color

The total quantity of data = 167.96 G Bytes !!

Digital Video Compression Fundamentals and Standards


Introduction 2 2

Introduction (2/2)

  • What is the difference between video compression and image compression?

    • Temporal Redundancy

  • Coding method to remove redundancy

    • Intraframe Coding

      • Remove spatial redundancy

    • Interframe Coding

      • Remove temporal redundancy

Digital Video Compression Fundamentals and Standards


The most intuitive method to remove spatiotemporal redundancy

The most intuitive method to remove Spatiotemporal redundancy

  • 3-Dimensional DCT

    • Remove spatiotemporal correlation

    • Good for low motion video

    • Bad for high motion video

Digital Video Compression Fundamentals and Standards


The most popular method to remove temporal redundancy

The most popular method to remove temporal redundancy

  • The Block-Matching Algorithm

Digital Video Compression Fundamentals and Standards


Matching function

Matching Function

  • The dissimilarity between two blocks and

  • The matching criteria

    • Mean square error (MSE)

      • High precision is needed

    • Mean absolute difference (MAD)

      • Low precision is enough

Digital Video Compression Fundamentals and Standards


The exhaustive block matching algorithm

The Exhaustive Block-Matching Algorithm

Reference Frame

Current Frame

Motion

Vector

|A|=12

12 22 33 44 55 66 77 88

12 22 33 44 55 66 77 88

12 22 33 44 55 66 77 88

12 22 33 44 55 66 77 88

11 23 34 44 55 66 77 88

11 23 34 44 55 66 77 88

11 23 34 44 55 66 77 88

11 23 34 44 55 66 77 88

11 22 33 44 55 66 77 88

11 22 33 44 55 66 77 88

11 22 33 44 55 66 77 88

11 22 33 44 55 66 77 88

11 22 33 44 55 66 77 88

11 22 33 44 55 66 77 88

11 22 33 44 55 66 77 88

11 22 33 44 55 66 77 88

1 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0

0 1 1 0 0 0 0 0

0 1 1 0 0 0 0 0

0 1 1 0 0 0 0 0

0 1 1 0 0 0 0 0

Search Range

Digital Video Compression Fundamentals and Standards


Fast block matching algorithms

Fast Block-Matching Algorithms

  • EBMA needs Intensive computation

  • Fast Algorithm is needed

    • Find the possible local optimal

Reference Frame

Current Frame

Digital Video Compression Fundamentals and Standards


Fast block matching algorithms1

Fast Block-Matching Algorithms

  • The characteristics of fast algorithm

    • Not accurate as EBMA

    • Save large computation

  • Two famous fast algorithm

    • 2-D logarithm Search Method

    • Three Steps Search Method

Digital Video Compression Fundamentals and Standards


2 d logarithm search method

2-D logarithm Search Method

1

2

1

1

1

1

2

4

4

4

1

2

2

4

3

3

4

4

3

Digital Video Compression Fundamentals and Standards


Three step search method

Three Step Search Method

2

2

2

3

3

3

1

1

2

1

1

3

2

2

3

3

3

3

3

2

2

2

1

1

1

1

1

1

Digital Video Compression Fundamentals and Standards


Multiresolution motion estimation 1 3

Multiresolution Motion Estimation (1/3)

  • The number of levels is L

  • l-th level images of the target frames

    where is set of pixels at level L

  • At the l-th level, the MV is

  • At the l-th level, the estimated MV is

  • Determine update such that error is minimized

  • The new motion vector is

Digital Video Compression Fundamentals and Standards


Multiresolution motion estimation 2 3

Multiresolution Motion Estimation (2/3)

  • Variable block size method

Digital Video Compression Fundamentals and Standards


Multiresolution motion estimation 3 3

Multiresolution Motion Estimation (3/3)

Digital Video Compression Fundamentals and Standards


Outline2

Outline

  • Introduction

  • Video Compression Standards

  • Simulation Reference Software

  • Future Work and Conclusions

Digital Video Compression Fundamentals and Standards


The development of video compression standards

The Development of Video Compression Standards

Digital Video Compression Fundamentals and Standards


The mpeg 1 standard

The MPEG-1 Standard

  • Group of Pictures

  • Motion Estimation

  • Motion Compensation

  • Differential Coding

  • DCT

  • Quantization

  • Entropy Coding

Digital Video Compression Fundamentals and Standards


Group of pictures 1 2

Group of Pictures (1/2)

  • I-frame (Intracoded Frame)

    • Coded in one frame such as DCT.

    • This type of frame do not need previous frame

  • P-frame (Predictive Frame)

    • One directional motion prediction from a previous frame

      • The reference can be either I-frame or P-frame

    • Generally referred to as inter-frame

  • B-frame (Bi-directional predictive frame)

    • Bi-directional motion prediction from a previous or future frame

      • The reference can be either I-frame or P-frame

    • Generally referred to as inter-frame

Digital Video Compression Fundamentals and Standards


Group of pictures 2 2

Forward Motion Compensation

N=9

M=3

GOP

I

B

B

P

B

B

P

B

B

I

Bidirectional Motion Compensation

Group of Pictures (2/2)

  • The distance between two nearest P-frame or P-frame and I-frame

    • denoted by M

  • The distance between two nearest I-frames

    • denoted by N

Digital Video Compression Fundamentals and Standards


The mpeg 1 encoder 1 4

The MPEG-1 Encoder (1/4)

Intra-frame

Residue

DCT

DCT

DCT

Q

Q

Q

Entropy

Coding

Entropy

Coding

Entropy

Coding

Inter-frame

Q-1

Q-1

Motion

Compensation

Motion

Compensation

Motion Vector

IDCT

IDCT

Motion

Estimation

Motion

Estimation

Frame

Memory

Frame

Memory

Digital Video Compression Fundamentals and Standards


The mpeg 1 encoder 2 4

The MPEG-1 Encoder (2/4)

  • Differential Coding

    • is the input image

    • is the predictive image

  • DCT

  • Digital Video Compression Fundamentals and Standards


    The mpeg 1 encoder 3 4

    The MPEG-1 Encoder (3/4)

    • Quantization

      • Intra quantization matrix

      • Inter quantization matrix

    Digital Video Compression Fundamentals and Standards


    The mpeg 1 encoder 4 4

    The MPEG-1 Encoder (4/4)

    • Motion Compensation

      • Exploit motion vector and the previous reconstructed frame to generate the predictive frame

        • is the compensated image

        • is the previous image

        • is the motion vector

    Reference frame

    Target frame

    Digital Video Compression Fundamentals and Standards


    The mpeg 2 standard

    The MPEG-2 Standard

    • Field/Frame DCT Coding

    • Field/Frame Prediction Mode Selection

    • Alternative Scan Order

    • Various Picture Sampling Formats

    • User Defined Quantization Matrix

    Digital Video Compression Fundamentals and Standards


    Progressive scan and interlaced scan

    Progressive Scan and Interlaced Scan

    Progressive Scan

    Interlaced Scan

    Digital Video Compression Fundamentals and Standards


    Field frame dct coding

    Field DCT Coding

    Luminance MB

    Frame DCT Coding

    Field/Frame DCT Coding

    • The field type DCT

      • Fast motion video

    • The frame type DCT

      • Slow motion video

    Digital Video Compression Fundamentals and Standards


    Alternative scan order

    Zig-zag scan

    Alternate scan

    1 1 1 1 1 1 1 1

    0 0 0 0 0 0 0 0

    1 1 1 1 1 1 1 1

    0 0 0 0 0 0 0 0

    1 1 1 1 1 1 1 1

    0 0 0 0 0 0 0 0

    1 1 1 1 1 1 1 1

    0 0 0 0 0 0 0 0

    4 0 0 0 0 0 0 0

    0.72 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0

    0.85 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0

    1.27 0 0 0 0 0 0 0

    0 0 0 0 0 0 0 0

    3.62 0 0 0 0 0 0 0

    2D DCT

    Alternative Scan Order

    • Zigzag scan order

      • Frame DCT

    • Alternative scan order

      • Field DCT

    Digital Video Compression Fundamentals and Standards


    The mpeg 2 encoder 2 2

    The MPEG-2 Encoder (2/2)

    • Quantization

      • User can change the quantization if necessary

      • Intra quantization matrix

      • Inter quantization matrix

    • Various picture sampling formats

      • 4:4:4

      • 4:2:2

      • 4:2:0

    Digital Video Compression Fundamentals and Standards


    The mpeg 2 encoder 1 2

    The MPEG-2 Encoder (1/2)

    SNR Enhanced Layer

    Q

    Entropy

    Coding

    • Base Layer

      • Basic quality requirement

      • For SDTV

    • Enhanced Layer

      • High quality service

      • For HDTV

    +

    Bits Enhance

    Q-1

    +

    Bits Base

    Entropy

    Coding

    DCT

    Q

    Q-1

    +

    Q-1

    Base Layer

    +

    Motion

    Compensation

    IDCT

    +

    Motion

    Estimation

    +

    Frame

    Memory

    Digital Video Compression Fundamentals and Standards


    H 264 avc

    H.264/AVC

    • Variable Block Size

    • Multiple Reference Frames

    • Integer Transform

    • Intra Prediction

    • In-loop Deblocking Filtering

    • 1/4-pel Resolution Motion Estimation

    • CAVLC

    Digital Video Compression Fundamentals and Standards


    Variable block size

    16 x 16

    16 x 8

    8 x 16

    8 x 8

    0

    0

    0

    1

    0

    1

    1

    2

    3

    8 x 8

    8 x 4

    4 x 8

    4 x 4

    0

    0

    0

    1

    0

    1

    1

    2

    3

    Variable Block Size

    • The fixed block size may not be suitable for all motion objects

      • Improve the flexibility of comparison

      • Reduce the error of comparison

    • 7 types of blocks for selection

    Digital Video Compression Fundamentals and Standards


    Multiple reference frames

    Multiple Reference Frames

    • The neighboring frames are not the most similar in some cases

    • The B-frame can be reference frame

      • B-frame is close to the target frame in many situations

    Digital Video Compression Fundamentals and Standards


    Integer transform for reducing the spatial redundancy 1 2

    Integer Transform for Reducing The Spatial Redundancy (1/2)

    • The transform matrix C

    • 4×4 Block Size

    • Separable Integer Transform

    • The transform coefficients are CXCT

    Digital Video Compression Fundamentals and Standards


    Integer transform for reducing the spatial redundancy 2 2

    Integer Transform for Reducing The Spatial Redundancy (2/2)

    C

    X

    CT

    E

    Digital Video Compression Fundamentals and Standards


    Intra prediction

    Intra

    Prediction

    Transform/

    Quantization

    Entropy

    Coding

    Vertical

    Horizontal

    DC

    Vertical left

    Diagonal down-left

    Diagonal right

    +

    +

    +

    +

    Horizontal down

    +

    Horizontal up

    +

    Mean

    +

    +

    Intra Prediction

    • Predict the similarity between the neighboring pixels in one frame in advance, and exploit differential coding transform coding to remove the redundancy.

    Vertical right

    Digital Video Compression Fundamentals and Standards


    Remove perceptual redundancy

    q0

    q1

    q2

    q3

    p3

    p2

    p1

    p0

    Remove Perceptual Redundancy

    • In-loop deblocking filtering

      • Remove blocking artifact

        • Result from block based motion compensation

        • Result from block based transform coding

    QP

    Digital Video Compression Fundamentals and Standards


    1 4 pel resolution motion estimation

    Integer search positions

    Best integer match

    Half-pel search positions

    Best half-pel match

    Quarter-pel search positions

    Best quarter-pel match

    1/4-pel Resolution Motion Estimation

    Digital Video Compression Fundamentals and Standards


    The h 264 avc encoder

    The H.264/AVC Encoder

    Control Data

    Coder

    Controller

    Residue

    Transform/

    Quantization

    Entropy

    Coding

    Inverse

    Transform/

    De-Quantization

    Motion Vector

    Motion

    Compensation

    Intra-frame

    Prediction

    De-blocking

    Filter

    Motion

    Estimation

    Digital Video Compression Fundamentals and Standards


    Outline3

    Outline

    • Introduction

    • Video Compression Standards

    • Simulation Reference Software

    • Future Work and Conclusions

    Digital Video Compression Fundamentals and Standards


    H 264 reference software

    H.264 Reference Software

    http://iphome.hhi.de/suehring/tml/

    • JM

      • Current software version: JM 15

      • Benchmark

    Digital Video Compression Fundamentals and Standards


    Reference software demo

    Reference Software Demo

    Digital Video Compression Fundamentals and Standards


    Outline4

    Outline

    • Introduction

    • Video Compression Standards

    • Simulation Reference Software

    • Future Work and Conclusions

    Digital Video Compression Fundamentals and Standards


    Future work

    Future Work

    • Fast Mode Decision Algorithm

    • Interpolation Filter Design

    • Deblocking Filter Design

    • DCT-Based Motion Estimation

    • Implementation Based on TI DSP

    Digital Video Compression Fundamentals and Standards


    Fast mode decision algorithm

    Fast Mode Decision Algorithm

    • The computational cost of H.264 is large

      • Variable block-size ME

      • Multiple reference frames

    • Fast mode decision is needed for reducing the computation time

    Digital Video Compression Fundamentals and Standards


    Interpolation filter design

    Interpolation Filter Design

    • In order to estimate and compensate the fractional-pel displacements

    • Adaptive Interpolation filter for replacing the fixed coefficient filter

    Digital Video Compression Fundamentals and Standards


    Deblocking filter design

    Deblocking Filter Design

    • Block based ME and Transform result in the annoying blocking artifact

    • Reduce the blocking artifact can increase the quality of compressed video

    Digital Video Compression Fundamentals and Standards


    Dct based motion estimation

    DCT Based Motion Estimation

    • Robust even in noisy environment

    • Complexity comparison

      • DCT Based ME

        • O(M2)

      • Block Based ME

        • O(N2×M2)

    • Concept

      • Pseudo Phase

      • Similar to

    Digital Video Compression Fundamentals and Standards


    Implementation based on ti dsp

    Implementation Based on TI DSP

    • TMS320C6416

    • TI DM642

    Digital Video Compression Fundamentals and Standards


    Conclusions

    Conclusions

    Thank You

    Q & A

    Digital Video Compression Fundamentals and Standards


    References 1 2

    References (1/2)

    • [1] Yun Q.Shi and Huifang Sun, “Image and Video Compression for Multimedia Engineering: Fundamentals, Algorithms, and Standards”, CRC press, 2000.

    • [2] Yao Wand, Jorn Ostermann and Ya-Qin Zhang, “Video Processing and Communications”, Prentice Hall, 2007.

    • [3] Richardson, Lain E. G., “Video Codec Design: Developing Image and Video Compression Systems”, John Wiley & Sons Inc, 2002.

    • [4] Barry G, Haskell, Atul Puri and Arun N. Netravali, “Digital Video : An Introduction to MPEG-2”, Boston : Kluwer Academic, 1999.

    • [5] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H.264/AVC video coding standard”, IEEE Trans. on Circuits and systems for video Technology, vol. 13, no. 7, pp. 560-576, July 2003.

    • [6] G. Sullivan and T. Wiegand, “Video Compression - From Concepts to the H.264/AVC Standard”, Proceedings of the IEEE, Special Issue on Advances in Video Coding and Delivery, December 2004.

    • [7] 酒井善則、吉田俊之 共著,白執善 編譯,“影像壓縮技術”,全華,2004.

    Digital Video Compression Fundamentals and Standards


    References 2 2

    References (2/2)

    • [8] Thomas Wedi, “Adaptive Interpolation Filters and High-Resolution Displacements For Video Coding”, IEEE Trans. on Circuits and Systems For Video Technology, vol.

    • [9] Dong-Hwan Kim, Hwa-Yong Oh, O˘guzhan Urhan, Sarp Ertürk and Tae-Gyu Chang, “Optimal Post-Process/In-Loop Filtering for Improved Video Compression Performance”, IEEE Trans. on Consumer Electronics, vol. 53, no. 4, Nov. 2007.

    • [10] Shu-Fa Lin, Meng-Ting Lu, and Homer Chen, ” Fast Multi-Frame Motion Estimation for H.264 andIts Applications to Complexity-Aware Streaming”, IEEE International Symposium on Circuits and Systems, 2005.

    • [11] Kai-Ting Cheng and Hsueh-Ming Hang, “Acceleration and Implementation of H.264 Encoder and Scalable Extension of H.264 Decoder on TI DSP Platform”, master thesis, June 2007

    Digital Video Compression Fundamentals and Standards


  • Login