OVID: Design and Implementation of Video-Object Database System

1. OVID: Design and Implementation of Video-Object Database System E. Oomoto and K. Tanaka IEEE Transactions on Knolwedge and Data Engineering, 5(4), August 1993. Modified from a presentation by Fahd Al-Qaddaa

2. Outline • Video-Object Data Model • Introduction • Motivating Problems • Basic Ideas • Video Object Data Model • OVID: A Video-Object Database System

3. Introduction • OO concept is considered suitable for describing the structure and semantic of multimedia data • Major features provided by OODB systems: • Representation and management of complex type • Handling object identity • Encapsulation of data and procedures • Inheritance of attribute structure and methods based on class hierarchy

4. Introduction • Is the modeling power offered by OODB features enough for MM data, especially video? • New data modeling for video database management is needed because • Video itself is raw data and independent from the content and the database structure • Meaningful scenes in video are identified and described incrementally • Meaningful scenes may be overlapped or included by others and may share some descriptional data.

5. Introduction • The authors introduced a new data model called Video Object Data Model • They introduce video objects • The model is schemaless • The inheritance is based on interval inclusion relationships • A collection of composition operations for video objects are also provided

6. Motivating Problems • Difficulties in defining attributes: • To describe a video in conventional OODBMS’s • We should decide what basic objects are • Each frame (still image) could be a single object • However, semantically a scene is a sequence of frames • Objects could be defined as semantically meaningful sequences of frame (Can be done using OODBMS’s) • Each object is defined by the tuple: (starting frame#, ending frame#, and other attributes to describe the scene)

7. Motivating Problems • The attributes are difficult to be defined in advance: • Different catalogers/taggers may give the same scene different descriptions • The whole content of a video is difficult to describe as a whole. • It should be done incrementally • A scene may contain another scene as its subinterval, so descriptional data may be shared among scenes based on time interval inclusion relationship.

8. Motivating Problems • Difficulties in querying • In a conventional DBMS query, • a database schema (tables, or class hierarchy) must be known • Users must know the attribute structures of class hierarchy in order to retrieve desired objects • When each object has a different attribute structure, the following problem appears: • Users should inspect each object to know its attributes • When there is no schema, users must describe all definitions of attributes and their values

9. Motivating Problems • Treatment of composed objects • Video data stored may be used for several editorial works • Video data are often cut and/or concatenated for multimedia presentation • In order to do this, the system should allow: • Retrieving Video data and composing the result into a new video objects • Different editorial operations

10. Basic Ideas • Consider any portion of a video frame sequence as an independent entity • A video objects corresponds to a certain set of video frame sequence • An object has its own attributes and their values to represent the content (meaning) of the corresponding video scene

11. Basic Ideas • The features of the video object model: • Schemaless description of database • It does not assume a specific database schema such as class hierarchy, so users can specify any attribute structure for video object • For example, broadcast stations have many varieties of video libraries such as news, movies, documentaries, and so on • It is very difficult to offer common attribute structure for all of them • Users may have different viewpoint to describe and/or retrieve video scenes

12. Basic Ideas • Schemaless description … • It is difficult to decide rigidly the attribute structures • They should be extensible freely and incrementally • As new object are defined, new attributes may be needed • Interval Inclusion inheritance • Descriptional data can be inherited to other video objects • For example • Consider a night scene A and another object B is defined over some portion of A. B is also a night scene. • If A has an attribute situation with value night, then B must have the same attribute and value by interval inclusion inheritance

13. Basic Ideas • Composition of video objects based on is-a hierarchy • For example, in broadcast stations: • video scene are edited from source video • They are chopped in order to remove redundant scenes • They are concatenated with each other to make TV programs • In order to do this, new operations is defined • Interval projection • Merge • Overlap • These operations produce new video objects and based on is-a hierarchy inherit attributes and their values to the new one.

14. Video Object Data Model • Video Object • A video-object is a descriptional data of meaningful scene (motion picture) • It consists of: • Its object identifier • An interval • A collection of attribute/value pairs • Each video object corresponds to a video-frame sequence

15. Video Object Data Model • Video Object … • An interval is represented by a pair of • Starting frame number • Ending frame number • Assume the following mutually disjoint sets • D: a set of atomic values (numbers, strings, symbols Τ and  ) • ID: a set of object identifiers • I: a set of intervals • A: a set of attribute names

16. Video Object Data Model • Video Object … • Definition. A video-object is a triple (oid,I,v), where: • oid is an object identifier and oid ID • I is a finite subset of I • v is an n-tupe [a1 : v1, ….., an : vn], where each ai(1  i  n) is an attribute name in A and vi is a value defined in the following manner: • Each element x  D is a value • Each interval i  I is a vlaue • For v1, … , vn(0  n), {v1, … , vn} is a value called a set value • Each video-object is also a value

17. Video Object Data Model • Video Object …

18. Video Object Data Model • Generalization Hierarchy for Values and Objects • A generalization (is-a) hierarchy for atomic values is assumed to be G = (A, , Τ, ), where: • A is a set of atomic values •  is an is-a relationship among values • Τ denotes unknown •  denotes undefined • The binary relation  denotes more informative

19. Video Object Data Model • Generalization Hierarchy … • This is-a relationship can be extended to general values and also to video-objects • Suppose we have two video-objects o1 = (oid1, I1, v1) and o2 = (oid2, I2, v2) • If v1 and v2 are set-type values, then v1 is-a v2 if for each element y in v2, there exists an element x in v1 such that x is-a y • If v1 and v2 are tuple-type values, then v1 is-a v2 if v1.a is-a v2.a for each attribute a in v2 • For video object o1 and o2if v1 is-a v2 then o1 is-a o2

20. Video Object Data Model • Generalization Hierarchy … • Definition. Least Upper Bound of Values • For two values v1 and v2, v = v1 v2 is the least upper bound of v1 and v2, if the following conditions hold: • v1 is-a v, • v2 is-a v , and • There does not exists v’ such that vv’, v’ is-av, v1 is-a v’, and v2 is-a v’ • v1 v2 represents the maximum information that is common to both of values v1and v2

21. Video Object Data Model • Generalization Hierarchy … • Definition. Greatest Lower Bound of Values • For two values v1 and v2, v = v1 v2 is the greatest lower bound of v1 and v2, if the following conditions hold: • v is-a v1, • v is-a v2, • There does not exists v’ such that vv’, v is-av’, v’ is-a v1, and v’ is-a v2 • v1 v2 represents the minimum information that contains both of values v1and v2

22. Video Object Data Model • Inheritance Based on Interval Inclusion Relationship • For a given two intervals I1 and I2, the inclusion relationship between them is defined as: • For each i I1, there exists i’ I2, such that i i’, then I1 is said to be included by the set interval I2, denoted by I1 I2 • Consider the two objects o1 = (oid1, I1, v1) and o2 = (oid2, I2, v2), such that I1 I2 holds, then some attribute/value pairs of v2 are inherited by the object o1

23. Video Object Data Model • Inheritance … • Definition. Evaluation of Interval Inclusion Inheritance by a Single Object • Suppose we have video-objects o1 = (oid1, I1, v1) and o2 = (oid2, I2, v2), such that I1 I2 holds, and a set A of inheritable attributes • The result of applying the interval inclusion inheritance is called evaluation

24. Video Object Data Model • Inheritance … • The evaluation of o1 by o2 and A, denoted by eval(o1, o2, A) is a video object o’ = (oid1, I1, v’1) such that • attr(v’) = attr(v1)  (attr(v2)  A) • v’1.a = v1.a , if a is in attr(v1), and • v’1.a = v2.a , if a is in (attr(v2)  A), but not in attr(v1) • Note that identifier of o1 and o’ are the same, which means the evaluation is not physically stored in the database and is calculated dynamically

25. Video Object Data Model • Inheritance … • Definition. Evaluation of Interval Inclusion Inheritance by a Multiple Objects • Suppose we have video-objects o1 = (oid1, I1, v1). Let O = {o21,…, o2m} be the set of all video-objects in the database, such that I1 I2 holds for each o2i = (oid2i, I2i, v2i) • The evaluation of o1by O – {o1}and a set A of inheritable attributes, denoted by eval(o1, O, A), is a video-object o’1 = (oid1, I1, v’1) such that: • attr(v’1) = attr(v1)  (attr(v21)  A) …  (attr(v2m)  A) • v’1.a = v1.a , if a is in attr(v1), • For each a in A, let Va = {v’2i.a | a  attr(v2i), a attr(v1), and 1 i  m} • For each nonempty set Va , v’1.a = glb(Va ), where glb(Va ) denotes the greatest lower bound of all elements in Va

26. Video Object Data Model • Composition Operations for Video-Objects • Interval Projection Operation • Definition. Interval Projection Operation • Let o= (oid, I, v), and I’ I holds, and let A be a set of inheritable attributes. The interval projection on o onto I’ is a video-object o’= (oid’, I’, v’) such that oid’ is a new identifier, and v’ satisfies the following: • attr(v’) = attr(v)  A, and • v’.a = v.a for each attribute a in attr(v’) • This operation is useful when defining a new video-object for a certain portion of an already existing one

27. Video Object Data Model • Composition Operations … • Merge and Overlap of Interval Sets • Definition. Merge and Overlap. For two intervals I1 and I2 the merge and overlap if I1 and I2, denoted by I1I2 and I1I2, respectively, are defined as: • I1I2 is the minimal set of intervals such that: • For each interval i in I1 there exists an interval i’ in I1I2 such that i’ i • For each interval i in I2 there exists an interval i’ in I1I2 such that i’  i • For every interval i1 and i2 in I1I2 such that i1 i2, i1 i2 and i2 i1

28. Video Object Data Model • Composition Operations … • I1I2 is the maximal subset of {i1i2| i1 I1 and i2 I2} such that i’1i’2and i’2i’1for arbitrary intervals i’1andi’2 in I1I2 • The merge operation creates a new video-objects o form existing objects o1 and o2 such that some descriptional data common to both is inherited by o • The overlap extracts the scene described by both of two existing video objects as new video-objet.

29. Video Object Data Model • Composition Operations … • Merge of Video-objects • The merge of two objects o1 = (oid1, I1, v1) and o2 = (oid2, I2, v2) denoted by o1o2, is the video object o = (oid, I1I2, v) such that v = [a1 : v1, … ai : vi ..., an : vn] where each ai (1  in) is in attr(v1)  attr(v2), and for each ai: • If both o1.ai and o2.ai are values, then vi = o1.ai  o2.ai • If both o1.ai and o2.ai are video-objects, then vi = o1.aio2.ai

30. Video Object Data Model • Composition Operations … • Overlap of Video-objects • For two video objects o1 = (oid1, I1, v1) and o2 = (oid2, I2, v2) such that I1I2 is nonempty, the overlap of o1 and o2, denoted by o1o2, in a new video object o = (oid, I1I2, v) such that oid is a new identifier and v = [a1 : v1, … ai : vi ..., an : vn] where • Each ai is in attr(v1)  attr(v2), • If both v1.ai and v2.ai are values, then v.ai = v1.ai  v2.ai • If both v1.ai and v2.ai are video-objects, then vi = v1.aiv2.ai

31. OVID • OVID is a video-object database system which developed based on the video object data model • The features of OVID: • Video Object as Central Units: • The video-objects are the central units of OVID system. • Each video-object consists of: • The oid of the video-object • a set of pairs of starting video frame# and an ending frame# of the video scene • a set of attribute/value pairs which describe the content of the corresponding video frame sequence • Basic methods such as Play, Inspect, and Disaggregate operations.

32. OVID • Features of OVID … • Dynamic and Incremental Object Identification and Definition: • Users can identify a meaningful scene at any time and define this scene with its descriptional data as a video-object • Users can add necessary attributes as well as use attributes of predefined video-objects

33. OVID • Features of OVID … • Video Object Composition: • OVID allows definition of a video-object which corresponds to more than one consecutive sequence of video frames. • This operation corresponds to the merge operation described. • It also supports creation of a video-object by applying the overlap operation to predefined video-objects.

34. OVID • Features of OVID … • Browsing by Video Object Disaggregation: • In order to browse a large video-object, OVID provides a decomposition of a video-object into smaller video-objects. • This decomposition corresponds to the interval projection operation described. • In OVID, this operation is called Disaggregation. • Disaggregation can be repeatedly applied. • This operation is useful to find a necessary scene contained by a large video-object in a navigational manner.

35. OVID • Features of OVID … • Generalization Hierarchy for Atomic Values: • OVID supports the usage of a generalization is-a hierarchy, which consists of atomic values that are used as attribute values of video-objects, in both cases: creation and retrieval of video-objects.

36. OVID • Features of OVID … • Video Objects as Attribute Values: • OVID allows a video-object to be an attribute value of another video-object. • This is very useful when it is difficult to describe a meaningful scene by text.

37. OVID • Features of OVID … • Ad hoc Query Facility for Video Objects: • OVID has an ad hoc query facility, called VideoSQL. • It facilitates retrieval of a collection of video-objects that satisfy a given condition.

38. OVID • Component of OVD • The OVID system consists of the following Components • VideoChart: A bar-chart type, visual interface for manipulating video-object. • VideoSQL: An ad hoc query facility to retrieve video-objects. • Video Object Definition Tool: A facility for object definition.

39. OVID • VideoChart • VideoChart has the following facilities: • Browsing video-objects in a bar-chart form • Playing video-object as live video • Inspecting and updating video-objects • Composing (merge and overlap) video-objects • Decomposing (disaggregating) of video-objects • Moving to the video object definition tool and VideoSQL.

40. OVID • Browsing of a Video Database by VideoChar: • VideoChart can view the content of a video database in a visual form. • A collection of video-objects appearing in a specified range are displayed in bar-chart form. • Each line denotes a single video-object. • The figure shows the currently displayed range is from frame# 26 830 to frame# 31 140.

41. OVID • Features of OVID …

42. OVID • Several operations can be applied on the selected video-object. • Copy: copies the selected object into a buffer for later use in VideoSQL • Play: replays the selected video-object on the monitor screen as a live video • Inspect: first it evaluates the specified video-object based on the interval inclusion inheritance, and displays the attributes and their values including the inherited ones.

43. OVID • Merge and Overlap Operations in VideoChart: • After the user selects two arbitrary video-objects, he can apply the operations merge and overlap to those video-objects. • Disaggregate of a Video-Object • When the operation disaggregate is applied on a video-object, it is automatically divided into ten sub-video-objects. • The figure shows that Video Disaggregation System as just launched:

44. OVID • Features of OVID …

45. OVID • VideoSQL • VideoSQL is a query language of OVID for retrieving video-objects. • Users can formulate VideoSQL query in the fill-in-the-blank manner • The result of VideoSQL is a collection of video-objects that satisfy the specified condition. • The figure shows an example query formulated by VideoSQL.

46. OVID • Features of OVID …

47. OVID • A VideoSQL query consists of the following clauses: • SELECT clause: This is different from ordinary SQL. It specifies only the category of the resulting video-object as: • Continuous: objects consisting of a single continuous video frame sequence. • Incontinuous: objects consisting of more than one continuous video frame sequence. • AnyObject: all types of objects whether they are continuous or not. • FROM clause: This clause is used to specify the name of the video database

48. OVID • WHERE clause: This clause is used to specify the condition, consisting of attribute/value pairs and comparison operators. • The user can select necessary attribute names by means of pop-up menu. • The user can specify the following condition: • [attribute] is [value | video object] • [attribute] contains [value | video object], this is concerned with set-type attributes. • definedOver [video sequence | video frame], this returns video-objects that are defined over the specified video frame sequence of frame.

49. OVID • Video Object Definition Tool • The object definition tool is used to define and update video-objects. • The object definition is accomplished by filling-in-the-blanks manner • Attribute names are selected by a pull-down menu • Atomic values can be specified by selecting an appropriate value from the scroll windows of the generalization browser

50. OVID