Multimedia databases
1 / 49

Multimedia Databases - PowerPoint PPT Presentation

  • Updated On :

Multimedia Databases. Prepared by Pradeep Konduri Instructor: Dr. Yingshu Li Georgia State University. Plan of Attack. Introduction Architecture Image Content Analysis Modeling Constructs Logical Implementation Real-World Applications Conclusion. Types of multimedia data.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Multimedia Databases' - ciel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Multimedia databases l.jpg

Multimedia Databases

Prepared by Pradeep Konduri

Instructor: Dr. Yingshu LiGeorgia State University

Plan of attack l.jpg
Plan of Attack

  • Introduction

  • Architecture

  • Image Content Analysis

  • Modeling Constructs

  • Logical Implementation

  • Real-World Applications

  • Conclusion

Types of multimedia data l.jpg
Types of multimedia data

  • Text: using a standard language (SGML, HTML)

  • Graphics: encoded in CGM, postscript

  • Images: bitmap, JPEG, MPEG

  • Video: sequenced image data at specified rates

  • Audio: aural recordings in a string of bits in digitized form

Nature of multimedia applications l.jpg
Nature of Multimedia Applications

  • Repositories: central location for data maintained by DBMS, organized in storage levels

  • Presentations: delivery of audio and video data, temporarily stored.

  • Collaborative: complex design, analyzing data

Management issues l.jpg
Management Issues

  • Modeling: complex objects, wide range of types

  • Design: still in research

  • Storage: representation, compression, buffering during I/O, mapping

  • Queries: techniques need to be modified

  • Performance: physical limitations, parallel processing

Research problems l.jpg
Research Problems

  • Information Retrieval in Queries: Modeling the content of documents

  • Multimedia/Hypermedia Data Modeling and Retrieval: Hyperlinks, Used in WWW

  • Text Retrieval: Use of a thesaurus

Multimedia database applications l.jpg
Multimedia Database Applications

  • Documentation and keeping Records

  • Knowledge distribution

  • Education and Training

  • Marketing, Advertisement, Entertainment, Travel

  • Real-time Control, Monitoring

A generic architecture of mmdbms l.jpg
A Generic Architecture of MMDBMS

  • Media organization: organize the features for retrieval(i.e., indexing the features with effective structures)

  • Media query processing: accommodated with indexing structure, efficient search algorithm with similarity function should be designed

Multimedia database architecture l.jpg




Multimedia Database Architecture

MM Data











Query Interface

MM Data

MM Data


MM Data



Multimedia DBMS

Multimedia Data

Preprocessing System

Database Processing

Document database architecture l.jpg




Document Database Architecture






DTD files

DTD Parser




Query Interface
















C++ Types


Multimedia DBMS



C++ Objects


Processing System

Database Processing

Image database architecture l.jpg

Image Analysis

and Pattern Recognition




Image Database Architecture





Image Content



Query Interface






Multimedia DBMS


Processing System

Database Processing

Image content analysis l.jpg
Image Content Analysis

  • Image content analysis can be categorized in 2 groups:

    • Low-level features: vectors in a multi-dimensional space

      • Color

      • Texture

      • Shape

    • Mid- to high-level features: Try to infer semantics

    • Semantic Gap

Image content analysis color l.jpg
Image Content Analysis: Color

  • Color space:

    • Multidimensional space

    • A dimension is a color component

    • Examples of color space: RGB, HSV

    • RGB space: A color is a linear combination of 3 primary colors (Red, Green and Blue)

  • Color Quantization

    • Used to reduce the color resolution of an image

  • Three widely used color features

    • Global color histogram

    • Local color histogram

    • Dominant color

Color histograms l.jpg
Color Histograms

  • Color histograms indicate color distribution without spatial information

    • Color histogram distance metrics

Image content analysis texture l.jpg
Image Content Analysis: Texture

  • Refers to visual patterns with properties of homogeneity that do not result from the presence of only a single color

  • Examples of texture: Tree barks, clouds, water, bricks and fabrics

  • Texture features: Contrast, uniformity, coarseness, roughness, frequency, density and directionality

  • Two types of texture descriptors

    • Statistical model-based

      • Explores the gray level spatial dependence of texture and extracts meaningful statistics as texture representation

    • Transform-based

      • DCT transform, Fourier-Mellin transform, Polar Fourier transform, Gabor and wavelet transform

Image content analysis shape l.jpg
Image Content Analysis: Shape

  • Object segmentation

    • Approaches:

      • Global threshold-based approach

      • Region growing,

      • Split and merge approach,

      • Edge detection app

    • Still a difficult problem in computer vision. Generally speaking it is difficult to achieve perfect segmentation

Salient objects vs salient points l.jpg

Segmented images with region boundaries


salient points

Original images

Salient Objects vs. Salient Points

Generic low-level description of images into salient objects and salient points

Modeling images principles l.jpg
Modeling Images – Principles

  • Support for multiple representations of an image

  • Support for user-defined categorization of images

  • Well-defined set of operations on images

  • An image can have (semantic, functional, spatial) relationships with other images (or documents) which should be represented in the DBMS

  • An image is composed of salient objects (meaningful image components)

Salient object modeling l.jpg
Salient Object Modeling

  • Multiple representations of a salient object (grid, vector) are allowed

  • A salient object O is of a particular type which belongs to a user defined salient object types hierarchy

  • An image component may have some (semantic, functional, spatial) relationships with other salient objects

Semantic gap l.jpg
Semantic Gap”

semantics-intensive multimedia systems & applications


multimedia data models

Semantic Gap



raw data,primitive properties (size, format, etc)

semantic meaning of the data

Semantic modeling of multimedia why hard l.jpg
Semantic modeling of multimedia -- Why hard?

  • Context-dependency

    • Semantics is not a static and intrinsic property

    • The semantics of an object often depends on:

      • the application/user who manipulate the object

      • the role that the object plays

      • other objects in the same “context”


Van Gogh’s paintings


Why hard cont l.jpg
Why hard? (cont.)

  • Modality-independency

    • Media objects of different modalities may suggest the similar/related semantic meanings.

    • Example:



Harry Potter has never been the star of a Quidditch team, scoring points while riding a broom far above the ground. He knows no spells, has never helped to hatch a dragon, and has never worn a cloak of invisibility.




Mediaview a semantic bridge l.jpg
MediaView– A “Semantic Bridge”

  • An object-oriented view mechanism that bridges the semantic gap between multimedia systems and databases

  • Core concept –media view (MV)

    • a customized context for semantic interpretation of media objects (text docs, images, video, etc)

    • collectively constitute the conceptual infrastructure of a multimedia system & application

Architecture l.jpg



Basic concepts l.jpg
Basic Concepts

An example…

Basic concepts26 l.jpg
Basic Concepts

Semantics-based data reorganization via media views

View operators l.jpg
View Operators

  • A set of operators that take media views and view instances as operands.

  • Focus on the operators that are indispensable in supporting queries and navigation over multimedia objects.

View operators28 l.jpg
View Operators



syntax<boolean>:= v-overlap (<media view1, mediaview2 >)

semanticstrue, if and only if ( o  O)(oextent(<media view1>) andoextent(<media view2>))


syntax{<object>}:= cross (<media view1, media view2 >)

semantics{<object>} := {o  O | o extent(<media view1>) andoextent(<media view2>)}


syntax{<object>}:= sum (<media view1, meida-view2 >)

semantics{<object>} := {o  O | o  extent(<media view1>) oroextent(<media view2>)}


syntax{<object>}:= subtract (<media view1, media view2>)

semantics{<object>}:= {o  O | o extent(<media view1>) andoextent(<media view2>)}

View operators29 l.jpg
View Operators



syntax<base class> := class(<view instance>)

semantics<view instance> is a instance of <base class>


syntax{<object>} := components (<view instance>)

semantics {<object>} := { oO | o is a component (direct or indirect) of <view instance>}


syntax<boolean> := i-overlap (<view instnace1>, <view instance2>)

semanticstrue, if and only if ( o  O) (o  components (<view instance1>) and o  components(<view instance2>))

View algebra l.jpg
View Algebra

  • Functions

    -- derivation of new MVs from existing MVs

    Heuristic Enumeration

    • Blind enumeration

    • Content-based enumeration

    • Semantics-based enumeration

View algebra31 l.jpg
View Algebra

  • Algebra Operators

    • select from src-MV where <predicate>

    • project <property-list> from src-MV

    • intersect (src-MV1, src-MV2)

    • union (src-MV1, src-MV2)

    • difference (src-MV1, src-MV2)

Logical implementation l.jpg
Logical Implementation

  • MediaView Construction

  • MediaView Customization

  • MediaView Evolution

Mediaviews construction l.jpg
MediaViews Construction

  • Work with CBIR systems to acquire the knowledge from queries

    • Learn from previously performed queries

    • A multi-system approach to support multi-modality of media objects

  • Organize the semantics by following WordNet

Why wordnet l.jpg
Why WordNet?

  • Different queries may greatly vary with the liberty of choosing query keywords

  • We need an approach to organize those knowledge into a logic structure

    • A simple “context”: a concept in WordNet

    • Common media views: corresponds to simple contexts

    • We provide all common media views, based on which users can build complex ones.

Navigating the multimedia database l.jpg
Navigating the Multimedia Database

  • Navigating via semantic relationships of WordNet

    Semantic Relationship Examples

    Synonymy (similar) pipe, tube

    Antonymy (opposite) fast, slow

    Hyponymy (subordinate) tree, plant

    Meronymy (part) chimney, house

    Troponomy (manner) march, walk

    Entailment drive, ride

Mediaview customization l.jpg
MediaView Customization

  • Two level MediaView Framework

Mediaview customization41 l.jpg
MediaView Customization

  • Dynamically construct complex-context-based media views based on simple ones

    • An example complex context: “the Grand Hall in City University”

  • Several user-level operators are devised to support more complex/advanced contexts, besides the basic operators

User level operators l.jpg
User-level Operators

  • INHERIT_MV(N: mv-name, NS: set-of-mv-refs, VP: set-of-property-ref, MP: set-of-property-ref): mv-ref

  • UNION_MV(N: mv-name, NS: set-of-mv-refs): mv-ref

  • INTERSECTION_MV(N: mv-name, NS: set-of-mv-refs): mv-ref

  • DIFFERENCE_MV(N1: mv-ref, N2: mv-ref): mv-ref

Build a mediaview in run time l.jpg
Build a MediaView in Run-time

  • Example: find out info about "Van Gogh"

    • Who is "Van Gogh"?

    • What is his work?

    • Know more about his whole life.

    • Know more about his country.

    • See his famous painting "sunflower"

Build a mediaview in run time44 l.jpg
Build a MediaView in Run-time

  • Who is “Van Gogh”?

    • INHERIT_MV(“V. Gogh“, {<painter>},name=”Van Gogh” ,);

  • What is his work?

    • INTERSECTION_MV(“work”, {<painting>, vg});

  • Know more about his whole life.

    • INTERSECTION_MV(“life”, {<biography>, vg});

  • Know more about his country.

    • INTERSECTION_MV(“country”, {<country>, vg});

  • See his famous painting “sunflower”

    • Set sunflower = INTERSECTION_MV(“sunflower”, {<sunflower>, <painting>});Set vg_sunflower = INTERSECTION_MV(“vg_sunflower”, {vg_work, sunflower});

Authoring scenario l.jpg
Authoring Scenario

  • Creates a new media view named after the subject

    • All multimedia materials used in the document would be put into this MediaView for further reference.

  • To collect the most relevant materials for authoring, the user performs the MediaView building process.

    • Import suitable media objects by browsing media views

  • Reference the manner and style of authoring, to find other media views with similar topics.

    • Drag & Drop

    • “learning-from-references”

Summary l.jpg

  • Types of multimedia data: Text, Audio, Video, Images.

  • Management issues: Design, Storage, Modeling, Queries

  • Image Content Analysis: Color, Texture, Shape

  • MediaView – a semantic multimedia database modeling mechanism

    • to bridge the semantic gap between conventional database and semantics-intensive multimedia applications

  • A set of user-level operators to accommodate the specialization/generalization relationships among the media views

Summary contd l.jpg
Summary (contd..)

  • MediaView promises more effective access to the content of media databases

    • Users could get the right stuff and tailor it to the context of their application easily.

  • Providing the most relevant content from pre-learnt semantic links between media and context

    • high performance database browsing and multimedia authoring tools can enable more comprehensive applications to the user.

    • Users could customize specific media view according to their tasks, by using user-level operators

Further issues l.jpg
Further Issues

  • The development and transition of MediaView to a fully-fledged multimedia database system supporting “declarative” queries

  • Intensive and extensive performance studies

  • Advanced semantic relations (eg. temporal and spatial ones) can also be incorporated in combining individual media views