1 / 30

The MPEG-7 Visual Standard for Content Description-An Overview Thomas Sikora, Senior Member, IEEE

The MPEG-7 Visual Standard for Content Description-An Overview Thomas Sikora, Senior Member, IEEE. A presentation by Modupe Omueti For CMPT 820:Multimedia Systems Spring 2005. Contents. Introduction Scope Methodology Visual Descriptors Conclusion. Introduction.

kaiser
Download Presentation

The MPEG-7 Visual Standard for Content Description-An Overview Thomas Sikora, Senior Member, IEEE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The MPEG-7 Visual Standard for Content Description-An OverviewThomas Sikora, Senior Member, IEEE A presentation by Modupe Omueti For CMPT 820:Multimedia Systems Spring 2005

  2. Contents • Introduction • Scope • Methodology • Visual Descriptors • Conclusion

  3. Introduction • Moving Pictures Expert Group • MPEG-1 for interactive video (1992) • MPEG-2 for digital television (1994) • MPEG-4 for multimedia with emphasis on visual objects (1998 v1, 1999 v2) • MPEG-7 for multimedia content description (2001)

  4. Trends • Initially few sources of audio, image and video • Increase in volume of digitized audio, images and video • Still images digital video

  5. MPEG-7 • Formally named Multimedia Content Description Interface • Supports some degree of interpretation of the information’s meaning • Interpretation can be passed on to or accessed by a device or computer code • Not aimed at one application in particular

  6. Scope • Goals • Standardized descriptions • Meaningful descriptions • Elements • Description tools: visual decriptors and description schemes • Description Definition Language • System tools • Figure 1: Scope of MPEG-7 • Figure 2: MPEG-7 main elements

  7. Figure 1: Scope of MPEG-7 Normative part of MPEG-7 standard

  8. Figure 2: MPEG-7 main elements

  9. Applications • Digital libraries (image catalogue, film) • Broadcast media selection (TV channels) • Investigation services (human characteristics recognition, forensics) • Multimedia editing (personalized electronic news service) • Figure 3: Abstract Representation

  10. Figure 3: Abstract representation of possible applications using MPEG-7

  11. Methodology Standard Development Proposal Evaluation Specification for Technology Requirements Technology Request Experimentation Model Definition Core Experiments

  12. Visual Descriptors • General visual descriptors • Color, texture, shape, and motion features • Domain specific • Identification of human faces and face recognition

  13. Visual Color Descriptors • Color Spaces (HSV, HMMD) • Supports above for normative purposes • Also supports RGB, YCbCr color spaces • Scalable color descriptorFigure 4 • Global color Distribution of Images in color histograms • HSV space, uniformly quantized into 255 bins • Haar Transform used to encode histogram • Histogram bin non-uniformly quantized • color coefficients or histogram bin values for matching

  14. Visual Color Descriptors • Dominant color descriptor • Global + local spatial color distribution • Colors clustered into a small no of representative colors representative color, %age, spatial coherency, variance • Color layout descriptor • Spatial distribution of color in an arbitrarily shaped region • Color structure descriptor • HMMD, local color feature, sliding window • Histogram on color appearance count • Group of Frames/Group of Pictures • SCD for a collection of similar images (frames) or video frames • Average, median, intersection histograms of GoF or GoP

  15. Figure 4: Three color images and their MPEG-7 histogram color distribution, depicted using a simplified color histogram. Based on the color distribution, the two left images would be recognized as more similar compared to the one on the right.

  16. Visual Texture Descriptors • Texture Features • Visual patterns (homogenous or non-homogenous) • Multiple colors in images • Multiple intensities in images • Surface structural information Figure 5

  17. Figure 5: Examples of grayscale images with different textures. Using the MPEG-7 Visual texture descriptors, the two images on the bottom would be rated of similar texture, while less similar in texture compared to the two images on the top.

  18. Visual Texture Descriptors • Homogenous texture descriptorFigure 6 • Scale and orientation sensitive filters • Mean and SD of frequency coefficients (RT-FT) • Scale and rotation-invariant description and matching • 2D Gabor functions for filtering feature channels • Non homogenous texture descriptor (Edge Histogram) • Spatial distribution of edges • Division of image into 16 non overlapping blocks of equal size • Five edge categories: vertical, horizontal, 45 , 135 , and non directional edge. • Rotation-sensitive and rotation-invariant • Non uniform quantization using 3 bits, descriptor size of 240 bits (16x5x3)

  19. Figure 6: Frequency layout for MPEG-7 Homogenous Texture Descriptor frequency extraction. Energy and energy deviation values are extracted from this frequency division into 30 channels.

  20. Visual Shape Descriptors • Provides a powerful visual clue • Invariant to scaling, rotation, and translation • 2-D or 3-D in nature For 2-D there are two categories • Contour based which uses only boundary information of objects • Region-based which the entire shape region

  21. Visual Shape Descriptors • 3-D Shape Descriptor—Shape Spectrum • Based on a shape spectrum concept • Histogram of a shape index • Measures local convexity of each local 3-D surface • Histograms with 100 bins are used—each quantized by 12 bits. • Region Based Shape Descriptor (Art) Figure 7 • Uses all pixels constituting a shape within a frame • Region-based moments invariant to transformations • Coefficients of ART basis functions quantized

  22. Figure 7: Examples of various shapes that can be indexed using MPEG-7 Region-Based Shape Descriptor. Images contained in either of the sets (a)–(d) would be rated similar and dissimilar to the ones in the remaining sets. For example, images in set (a) would be identified being similar and dissimilar to the ones in set (b), (c), or (d).

  23. Types of Visual Shape Descriptors • Contour based shape descriptor Figure 9 • Curvature scale-space (CCS) • Eccentricity and circularity values • Robust to non-rigid motion partial occlusion of the shape and perspective transformations • 2-D/3-D shape descriptor • Representation of 3-D objects using multiple 2-D snapshots

  24. Figure 8: Examples of shapes that can be indexed using MPEG-7 Contour-Based Shape Descriptor.

  25. Motion Descriptors for Video • Motion Activity Descriptors • Activity level and pace of motion in a scene • Motion activity intensity descriptor • SD of motion vector magnitude • SDs quantized into five activity levels • Optional Features • motion direction • spatial distribution of motion activity • Temporal distribution of motion activity • Camera Motion Descriptor Figure 9 • Global motion parameters in time • zoom activity • translatory motion • Motion similarity matching in particular time periods

  26. Figure 9: Camera model for MPEG-7 Camera Motion Descriptor. Perspective projection to image plane p and camera motion parameters. The (virtual) camera is located in O.

  27. Motion Descriptors for Video • Warping Parameters • Parametric motion descriptor • Object description using 2-D parametric models • translations, rotations, scaling and combination of them • planar perspective models • quadratic models • Arbitrary objects, defined as regions (group of pixels) in the image over a specified time interval • Global sprite or mosaic • Motion Trajectory • Description for independently moving objects • Object displacement over time

  28. Conclusion • Identify, filter and browse images using visual content • Specification to allow interoperability and flexibility • Other MPEG-7 standards • Storage, access and transmission of descriptors and descriptors schemes in system specification

  29. Thank you

More Related