1 / 33

STRG-Index: Spatio-Temporal Region Graph Indexing for Large Video Databases

Outline. IntroductionGraph-based ApproachesDistance MeasureClustering Object GraphSTRG-Index StructureExperimentsConclusions and Future works. Introduction. IntroductionContent-based video retrieval systemsIssues in video retrieval systemsGraph-based ApproachesDistance MeasureClustering O

keran
Download Presentation

STRG-Index: Spatio-Temporal Region Graph Indexing for Large Video Databases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. STRG-Index: Spatio-Temporal Region Graph Indexing for Large Video Databases Jeongkyu Lee University of Bridgeport

    2. Outline Introduction Graph-based Approaches Distance Measure Clustering Object Graph STRG-Index Structure Experiments Conclusions and Future works

    3. Introduction

    4. Content-based video retrieval system Visual feature based retrieval systems : color, shapes, and textures of key frames Keyword based retrieval systems : manual annotation of segments Object based retrieval systems : spatial and temporal features of extracted objects

    5. Issues in video retrieval systems How to parse a video efficiently : low level feature or high level feature How to compute (dis)similarity : considering time (e.g. Lp-norms, DTW, LCS, and ED) How to index and retrieve the units : spatial and temporal relationships (3DR-tree, RT-tree, and M-tree)

    6. Our Solution is

    7. Graph-based approach

    8. Motivations for graph-based approach Graph is a powerful tool for pattern representation and classification Key issues Modeling unstructured data using a graph : spatial and temporal relationships Reducing high computational complexity : graph decomposition, indexing Graph matching to compute (dis)simialrity : graph edit distance

    9. Region Adjacency Graph Region segmentation using EDISON (Edge Detection and Image Segmentation System) Region Adjacency Graph (RAG)

    10. RAG cont. Example

    11. STRG cont.

    12. STRG cont. STRG Temporally connected RAGs Represent temporal as well as spatial relationships among segmented regions

    13. Spatio-Temporal Region Graph Neighborhood Graph Finding corresponding neighborhood graph over RAGs : using most common subgraph and maximal clique detection SimGraph (G, G’) = |Gc| / min (G, G’)

    14. Object Graph Object Region Graph (ORG) Extract temporal subgraphs, which represent a trjectory of tracked regions Temporal relationships Object Graph (OG) Merge ORGs into OG A linear graph connected by only temporal edges

    15. Background Graph Backgound Graph (BG) Foreground/background distinction OG elimination Nodes overlapping Reducing index size

    16. Distance measure

    17. Extended Graph Edit Distance Good distance metric for OGs requires Lower computation, handling time-varying, and considering attribute value Extended Graph Edit Distance (EGED) Let and be s-th and t-th OGs

    19. Metric vs. non-metric spaces EGED is not in metric space, since it does not satisfy the triangle inequality. Example: {0}, {1, 1}, and {2, 2, 3}. EGED cannot be used for indexing key values. If gi is a fixed constant, then EGED is a metric. It is proved by using induction. We refer EGED in metric as EGEDM. EGEDM can be used for indexing key values.

    20. Clustering Object Graph

    21. Purpose of clustering Graph-based video indexing needs clusters of OGs for more effective indexing. Each cluster has a similar pattern of OGs. Each cluster forms a cluster node of indexing tree as a branch node. For this clustering, we use EM and EGED.

    22. EM clustering for OG Gaussian mixture density with EGED Log-likelihood function Algorithm E-step: Compute the conditional expectation of the complete log-likelihood M-step: Update the parameter estimates

    23. STRG-Index structure

    24. STRG-Index tree structure [Top-level] Root node: a BG for each record [Mid-level] Cluster node: a centroid OG of cluster for each record [Low-level] Leaf node: OGs belonging to a cluster indexed by EGEDM(OGmem, OGclus) (a

    25. STRG-Index cont.

    26. STRG-Index construction Root node: extracted BGs are stored. All OGs sharing one background are in a same cluster node. This can reduce the index size significantly. Cluster node: synthesized centroid OGs are stored. Each record is a representative OG of each cluster. This centroid OG can be updated when member are changed. Leaf node: actual OGs in a cluster are stored. Records are indexed by EGEDM. Key value is EGEDM (OGmem, OGclus)

    27. Search algorithm using STRG-Index k-NN Search Algorithm

    28. Experimental results

    29. Results of clustering EM-EGED performance

    30. Indexing power STRG-Index Performance

    31. Real video streams Real video data set

    32. Conclusions

    33. Conclusions We propose a new data structure, STRG for video based on graphs. It represents spatial and temporal relationships of video objects We propose a new distance function, EGED in both non-metric and metric spaces for matching and indexing, respectively. We propose a new indexing method, STRG-Index, which is faster and more accurate indexing.

More Related