1 / 56

TERM PAPER PRESENTATION

TERM PAPER PRESENTATION. QUERY PROCESSING IN MULTIMEDIA DATABASES. TURKER YILMAZ. STUDIED ARTICLES. 1. Conceptual Modeling and Querying in Multimedia Databases. CHITTA BARAL, GRACIELA GONZALEZ, TRAN SON, 2. An Approach to a Content-Based Retrieval

margot
Download Presentation

TERM PAPER PRESENTATION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. TERM PAPER PRESENTATION QUERY PROCESSING IN MULTIMEDIA DATABASES TURKER YILMAZ

  2. STUDIED ARTICLES 1. Conceptual Modeling and Querying in Multimedia Databases.CHITTA BARAL, GRACIELA GONZALEZ, TRAN SON, 2. An Approach to a Content-Based Retrieval of Multimedia Data.GUISEPPE AMATO, GIOVANNI MAINETTO, PASQUALE SAVINO, 3. Integrated Spatial and Feature Image QueryJOHN R.SMITH, SHIH-FU CHANG, 4. An Image Database System with Support for Traditional Alphanumeric Queries and Content-Based Queries by Example.DANIEL L.SWEETS, YOGESH PATHAK, JOHN J.WENG,

  3. PRESENTATION PLAN • 1. Introduction. • 2. Presentation of articles in subject order, • which support or completes each other. • 3. Conclusion

  4. INTRODUCTION • Multimedia Data is any unstructured piece of • information stored in the Multimedia Database. • A Multimedia Database differs from a conventional • database. • Large image databases are commonly employed in • applications like criminal records, customs, plan root • databases and, voters’ registration databases.

  5. RESEARCHES • Most researches in Multimedia Database design • have focused on a particular kind of MM data. • Another direction of research on Multimedia • Databases is focused on data structures and • algorithms for storing and processing Multimedia • content.

  6. FOCUS • An object oriented approach to multimedia • database design. • Spatial approaches to image retrieval methods. • Content based queries, including automated feature • extraction methods supported with the alphanumeric • query modules. • Solutions to modeling problems and query display • methods.

  7. A GENERAL APPROACH • MULTIMEDIA MODEL CLASSIFICATIONS: • Multimedia Description Model provides the linguistic • mechanism for identifying the huge amount of • conceptual entities stored in raw objects. • Multimedia Presentation Model(MPM) describes the • temporal and spatial relationships among differently • structured multimedia data. • Multimedia Interpretation Model • The feature level, manages recognizable measurable • aspects of description level objects. • The concept level, describes the semantic content • of the description level objects.

  8. A GENERAL APPROACH (cont’d) • A Canonical media object (CO) is a higher level • view of a raw object and corresponds to the entire • raw objects where a media object represents a • relevant portion of a canonical media object. • Examples of MO’s are regions of images, sequences • of regions of video frames, video shots and, words • or paragraphs in text documents. • Operations defined on MO’s are the usual editing • primitives like creation, modification, access etc.

  9. ANALYSIS AND RETRIEVAL

  10. MULTIMEDIA DATABASE GENERATION • DATABASE POPULATION • ACCES STRUCTURE GENERATION • QUERY FORMULATION AND EXECUTION

  11. DATABASE POPULATION • Data is stored completely. • Features are extracted . • Recognition of concepts are associated with, • relevant objects.

  12. ACCES STRUCTURE GENERATION Using feature & concept values the system creates appropriate access structures that will speed up the subsequent process.

  13. QUERY FORMULATION AND EXECUTION • The user formulates the query by interacting • with the graphical interface provided by the • Query formulation tool or writes appropriate • query command into the shell. • Concepts can be recognized either; • During the database population • At retrieval time.

  14. QUERYING THE MULTIMEDIA DATABASE • Browsing: Users have foggy ideas of what they’re • looking for. • Content Based Retrieval: Where a request is • specified and retrieval of objects satisfying the • queries is expected. • Content Based Retrieval in Multimedia environments • generally takes the form of similarity queries, • which are needed when; • an exact comparison is not possible, • retrieved objects need to be ranked.

  15. QUERY RESTRICTIONS • Feature and Concepts: The user may express • restrictions on the values of the object’s features • and on the values of concepts. • Object Structure: Allowing the user to make • restrictions on the structure of the Multimedia • objects to be retrieved. • Spatio Temporal Relationships: Formulating • restrictions on the spatial and temporal • relationships of the objects to be retrieved. • Uncertainty: Users may not be certain of the • attribute of an object.

  16. THE MULTIMEDIA QUERY LANGUAGE • If the user specifies a certain concept in the query, • the answer set may also contain objects that do not • contain that concept but other related concepts, • which defined through a relationship between • concepts. • Weights are included in order to provide a ranked • based retrieval. • Selectors are needed to cope with features, • recognition degrees and structure.

  17. THE MULTIMEDIA QUERY LANGUAGE (cont’d) Example: After this schema, we can identify four classes containing canonical objects: MPEG, MJPEG, Frames, JPEG and GIF.

  18. THE MULTIMEDIA QUERY LANGUAGE (cont’d) By looking at this conceptual level schema we can say that skyscrapers, churches and bell towers are subclasses of BUILDING class.

  19. THE MULTIMEDIA QUERY LANGUAGE (cont’d) Example: Let us suppose that the user needs to “retrieve all images of all skyscrapers that are higher than 200m” This can be done with; SELECT I FROM I in images WHERE I match any (SELECT SS FROM SS in SKYSCRAPERS WHERE SS.height>200)

  20. SPATIAL APPROACHES TO FEATURE IMAGE QUERY Image queries can be performed by regions and their spatial and feature attributes.

  21. SaFe In spatial image query (SaFe) the images are matched based upon the relative locations of symbols. For example a relative SQ may ask for images in which symbol A is to the left of symbol B.

  22. HOW DOES IT WORK? • Regions and their feature and spatial attributes are • extracted from the images. • The overall match score between images is computed.

  23. IN SaFe SYSTEM • Each object is assigned a minimum bounding • rectangle. • Distances between objects are computed. • The user assigns the relative weighting “x” to each • object. • The overall single region query distance between • region q and t is given by;

  24. STARTEGIES FOR SPATIAL IMAGE QUERIES • Two strategies for image queries: • Parallel attribute query strategy • Pipeline attribute query strategy

  25. FEATURE QUERY • In order to provide color image retrieval, • query-by-color method is used. • In order to support query-by-color method, • an automated color region extraction system • is proposed which is named • “single-color quadratic back projection system” • (SCQBP).

  26. FEATURE QUERY (cont’d) • The system first generates a color histogram h • for each image. • For each image m such that h[m] r in a binary set • c is generated. • Then, each binary set is back projected onto • the image using;

  27. FEATURE QUERY (cont’d)

  28. RELATED IMAGE RETRIEVAL TECHNIQUES • Synthetic color region image retrieval • Color photographic image retrieval • are proposed and implemented. • This implementation can be found in the WEB at • URL: http://disney.ctr.columbia.edu/safe

  29. CONTENT BASED QUERIES and ALPHANUMERIC QUERY SUPPORT • The proposed system offers support for both • alphanumeric query, • based on alphanumeric data attached to the image file • and, • content based query utilizing image examples • which is accessible from within a user friendly GUI.

  30. SHOSLIF-O The proposed system implements image retrieval method using; Self Organizing Hierarchical Optimal Subspace and LearnIng Framework for Object Recognition.

  31. SHOSLIF-O (cont’d) The system incorporates 3 major modules:

  32. SHOSLIF-O (cont’d) The SHOSLIF-O module analyzes all images in the database and builds a hierarchical structure for efficient search providing the query-by-image content capability of the system. Alphanumeric database fields can be defined by the user in the definition phase and a flat file imported by the user can act as a database provided it matches the field count given in the definition phase. In the query phase, the user can enter a text query and the alphanumeric database modules search the database and come out with the image files that satisfy the given conditions.

  33. SHOSLIF-O (cont’d)

  34. PROCESS

  35. PROCESS (cont’d) • MEFs (Most Expressive Features) • MDFs (Most Discriminating Features)

  36. PROCESS (cont’d) When building the tree, the system can proceed in a supervised or unsupervised learning mode.

  37. KEYWORD-BASED QUERY SUPPORT • In large image databases, alphanumeric data • associated with an image is entered in an • alphanumeric database. • This system uses a relational database structure • for storage and retrieval of images and associated • data.

  38. GENERAL FEATURES • The matched items of appearance-based retrieval • have pointers to the associated text. • One can also start searching with a key field and • retrieve images. • One can use alphanumeric search to find all the • matched persons and their face images. Then the user • can use those images to find people who look similar • to those matched.

  39. GENERAL FEATURES

  40. PROPOSALS FOR EASING THE QUERY PROBLEMS AND RESULT DISPLAY An example: A movie database can be created using the following attributes: MOVIE(Title, Year, Producer, Director, Length, Movie_type, Prod_studio)

  41. Cont’d • Here another data type called “CORE” is proposed • in order to refer to the digitized item directly • without causing any confusion between special • attribute names. • New definition is: • MOVIE (Title, Year, Producer, Director, Length, • Movie_type, Prod_studio, CORE) • No need to be familiar with the schema.

  42. CER DIAGRAM FOR WWW

  43. CER FOR WWW (cont’d) • After creating CORE ER Diagram (CER), • the table definitions are: • HTMLDoc(h_url, title, type, length, lastmodify, • CORE) • Links (l_url, label) • Include (h_url, l_url) • After defining the following methods: • -contains (HTMDoc.Title, string) • -reach_by (HTMLDoc.url, url_to, by_n, by_type) • -mentions (HTMLDoc,string) • -linktype (HTMLDoc,url)

  44. CER FOR WWW (cont’d) “Starting from the Computer Science home page, find all documents that are linked through paths of lengths two or less containing only local links. Keep only the documents containing the string ‘database’ in their title.” SELECT Links.l_url FROM HTMLDoc,Links,Include WHERE substring(“database”, HTMLDoc.title) AND HTMLDoc.h_url= Include.h_url AND Links.l_url=Include.l_url AND reach_by(“http://cs.bilkent.edu.tr”, Links.l_url,2,local)

  45. CER FOR WWW (cont’d) • Adding two additional methods which are • displayDoc(HTMLDoc) • displayObj (WebObject, properties.position, • properties.size, properties.props) • Additional queries can be performed such as; • “List all documents that have video clip or picture • labeled ‘Atatürk’” • SELECT HTMLDoc.h_url • FROM HTMLDoc, WebObject, Include • WHERE HTMLDoc.h_url = Include.h_url • AND WebObject.w_url = Include.w_url • AND (WebObject.objectType= “IMAGE” • OR WebObject.objectType = “VIDEO”) • AND WebObject.label = “Atatürk”

  46. CER FOR WWW (cont’d)

  47. A RESULT DISPLAY PROPOSAL: SQL+D • “DISPLAY” word is proposed to be reserved. • Example: Consider a database for a video rental • store containing movie titles and other general • information of the movies, plus a movie clip and • a picture of the promotional poster. Also available • is a list of the actors in a movie, and other • information about the actors, including their picture. • The Schema looks as follows: • MOVIE (Available, title, director, producer, date, • classification, rating, CORE, poster) • MOVIE_ACTORS(title, name, role) • ACTORS (name, dob, biography, picture)

  48. A RESULT DISPLAY PROPOSAL: SQL+D (cont’d) “List all actors in ‘Gone with the wind’ with their pictures and biographies.” SELECT MOVIE_ACTORS.name, ACTORS.biography, ACTORS.picture FROM MOVIE_ACTORS,ACTORS WHERE MOVIE_ACTORS.title=”Gone with the wind” AND ACTORS.name=MOVI_ACTORS.name DISPLAY PANEL main, PANEL info ON main (east), WITH MOVIE_ACTORS.name AS list ON main (west), ACTORS.picture AS image ON info (north), ACTORS.biography AS text ON info (south)

  49. A RESULT DISPLAY PROPOSAL: SQL+D (cont’d)

  50. A RESULT DISPLAY PROPOSAL: SQL+D (cont’d)

More Related