1 / 21

FrameFree Video Compression

FrameFree Video Compression. Critical Points versus Block Matching. FrameFree Technologies http://framefree.com Igor Borovikov. Overview. Motion compensation for video compression, Critical Points Filter: brief introduction,

adia
Download Presentation

FrameFree Video Compression

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FrameFree Video Compression Critical Points versus Block Matching FrameFree Technologies http://framefree.com Igor Borovikov

  2. Overview • Motion compensation for video compression, • Critical Points Filter: brief introduction, • FrameFree Video: a bird eye view and side by side comparison of schematics with MPEG, • Occlusions and CPF, • Side by side comparison CPF3 and H.264, • FFV features overview, • Future work.

  3. Motion Compensation • Block Matching Algorithm – dominant motion predictor. • Advantages: • Conceptually simple yet effective algorithm, • Low CPU requirement on decoder side, • Motion vectors data is well suited for entropy compression.

  4. BMC Downsides Inherently discrete nature of blocks leads to: • Resolution dependence (partially addressed by adaptive block size), • Block artifacts (addressed by deblocking filters), • Optimal motion vectors (exhaustive search) are usually prohibitively expensive, • No simple way to reconstruct good quality non-integer frames, • BMC has no direct relation to Human Visual System.

  5. Alternatives • Optical flow and related methods.A large group of pixel matching algorithms, yet not directly suitable for motion compensation in video compression. • Critical Points Filter.The exceptional robustness of CPF pixel matching and adaptive mesh brings CPF to practical applications.

  6. Critical Points Filter (CPF) • Y. Shinagawa, and T. L. Kunii, Unconstrained Automatic Image Matching Using Multi-resolutional Critical-Point Filters, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.20, no.9, pp994-1001, Sep 1998 • K. Habuka and Y. Shinagawa, Image Interpolation Using Enhanced Multi-resolutional Critical-Point Filters, International Journal of Computer Vision, vol.58, no.1, pp19–35, 2004

  7. CPF mimics HVS HVS can reconstruct transition between these two images <====> So can CPF.

  8. Motion Compensation with CPF “I Frame” “B-Frame” “P-Frame” Original frame compressed with, say, JPEG. CPF- Interpolated between I and P frames: information from both sides of interval is used. Forward motion compensated and corrected frame.

  9. Motion Compensation with CPF

  10. BMC vs. CPF Source image Motion compensation with CPF (uncorrected) Block motion compensation, Search radius 16 (uncorrected) Destination image

  11. CPF vs. BMC BMC Metaphor: - “moving around tiles”; CPF Metaphor: - “deforming flexible film fixed on a rectangular frame, no tearing or folding allowed”.

  12. Motion Compensation with CPF • Bijectivity – no “holes” left after compensation, • Adaptive mesh - resolution independence, • Continuous time - no discretization whatsoever, • “Key features” – critical points – are matched with high precision, thus: • Remaining artifacts are not too conspicuous for HVS, • Residual errorof motion compensation is relatively well suited for combination of custom JPEG-like and entropy coding.

  13. MPEG vs. FFV encoding comparison

  14. MPEG vs. FFV encoding comparison

  15. “Practical Scheme” TRANS: Image Translator －: Differential Encoder & Noise Filter INTERP: Interpolation Unit ＋: Differential Decoder

  16. Occlusions and CPF “Boy and Toys” (NHK test sequence) with ball bouncing off the floor: CPF3 Original CPF “CPF3” is the latest modification of FFV Practical Scheme that addresses occlusion problem.

  17. Compression ratio and PSNR FFV is approaching H.264: many sequences are in between Med and Low settings in terms of PSNR and compression ratio. The table represents work in progress.

  18. Side by Side Comparison Original Original CPF* CPF3 H.264Low H.264Med *) The “worst” frames (occlusion artifacts are apparent) were picked for original CPF to show CPF3 improvements.

  19. FFV/CPF3 Features Overview • Continuous time – arbitrary playback speed, • Mesh based rendering – eliminates resolution dependence.Combination of 1 and 2 “liberates video from frames”, hence “FrameFree video” ; • Playback fits well modern graphics hardware – it is nothing else but rendering of textured meshes with simple pixel shader enhancements (ps_1_1), • CPF3 allows fallback Practical Scheme playback in case of dropped packets or CPU-bounded client.

  20. Future work • Simplify encoding and decoding by better occlusion handling, • Coming soon: • DirectShow filters and QT components, • CODEC, • Integration with popular players on both Mac and PC, • Parallel and GPU-based encoding, • Hardware support, • Mobile platforms support, • Lightweight decoder/player. • Some applications: • Sports video: arbitrary speed playback and web streaming, • Security video archiving, • Integration with FrameFree Studio, • Much more.

  21. Thank you! • Q&A • Business contact: Tom Randolph tom@framefree.us • Engineering contact: Igor Borovikov igor@framefree.us • General information: www.framefree.com

More Related