1 / 33

STRG-Index: Spatio-Temporal Region Graph Indexing for Large Video Databases

STRG-Index: Spatio-Temporal Region Graph Indexing for Large Video Databases. Jeongkyu Lee University of Bridgeport. Outline. Introduction Graph-based Approaches Distance Measure Clustering Object Graph STRG-Index Structure Experiments Conclusions and Future works. Introduction.

forest
Download Presentation

STRG-Index: Spatio-Temporal Region Graph Indexing for Large Video Databases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. STRG-Index: Spatio-Temporal Region Graph Indexing for Large Video Databases Jeongkyu Lee University of Bridgeport

  2. Outline • Introduction • Graph-based Approaches • Distance Measure • Clustering Object Graph • STRG-Index Structure • Experiments • Conclusions and Future works

  3. Introduction • Introduction • Content-based video retrieval systems • Issues in video retrieval systems • Graph-based Approaches • Distance Measure • Clustering Object Graph • STRG-Index Structure • Experiments • Conclusions and Future works

  4. Content-based video retrieval system • Visual feature based retrieval systems : color, shapes, and textures of key frames • Keyword based retrieval systems : manual annotation of segments • Object based retrieval systems : spatial and temporal features of extracted objects

  5. Issues in video retrieval systems • How to parse a video efficiently : low level feature or high level feature • How to compute (dis)similarity : considering time (e.g. Lp-norms, DTW, LCS, and ED) • How to index and retrieve the units : spatial and temporal relationships (3DR-tree, RT-tree, and M-tree)

  6. Our Solution is Graph-based Approaches

  7. Graph-based approach • Introduction • Graph-based Approaches • Motivations for graph-based approach • Region Adjacency Graph • Spatio-Temporal Region Graph • Object Graph • Background Graph • Distance Measure • Clustering Object Graph • STRG-Index Structure • Experiments • Conclusions and Future works

  8. Motivations for graph-based approach • Graph is a powerful tool for pattern representation and classification • Key issues • Modeling unstructured data using a graph : spatial and temporal relationships • Reducing high computational complexity : graph decomposition, indexing • Graph matching to compute (dis)simialrity : graph edit distance

  9. Region Adjacency Graph • Region segmentation using EDISON (Edge Detection and Image Segmentation System) • Region Adjacency Graph (RAG)

  10. RAG cont. • Example Frame #14 Region segmentation RAG

  11. STRG cont. Frame #14 Frame #15 Frame #16 Region segmentation STRG Magnifying part of STRG

  12. STRG cont. • STRG • Temporally connected RAGs • Represent temporal as well as spatial relationships among segmented regions

  13. Spatio-Temporal Region Graph • Neighborhood Graph • Finding corresponding neighborhood graph over RAGs : using most common subgraph and maximal clique detection SimGraph (G, G’) = |Gc| / min (G, G’)

  14. Object Graph • Object Region Graph (ORG) • Extract temporal subgraphs, which represent a trjectory of tracked regions • Temporal relationships • Object Graph (OG) • Merge ORGs into OG • A linear graph connected by only temporal edges Sample Object ORGs Merged OG

  15. Background Graph • Backgound Graph (BG) • Foreground/background distinction • OG elimination • Nodes overlapping • Reducing index size BG Part of STRG

  16. Distance measure • Introduction • Graph-based Approaches • Distance Measure • Extended Graph Edit Distance • Metric vs. non-metric spaces • Clustering Object Graph • STRG-Index Structure • Experiments • Conclusions and Future works

  17. Extended Graph Edit Distance • Good distance metric for OGs requires • Lower computation, handling time-varying, and considering attribute value • Extended Graph Edit Distance (EGED) Let and be s-th and t-th OGs

  18. Metric vs. non-metric spaces • EGED is not in metric space, since it does not satisfy the triangle inequality. • Example: {0}, {1, 1}, and {2, 2, 3}. • EGED cannot be used for indexing key values. • If gi is a fixed constant, then EGED is a metric. • It is proved by using induction. • We refer EGED in metric as EGEDM. • EGEDM can be used for indexing key values.

  19. Clustering Object Graph • Introduction • Graph-based Approaches • Distance Measure • Clustering Object Graph • Purpose of clustering in video processing • EM clustering for OG • STRG-Index Structure • Experiments • Conclusions and Future works

  20. Purpose of clustering • Graph-based video indexing needs clusters of OGs for more effective indexing. • Each cluster has a similar pattern of OGs. • Each cluster forms a cluster node of indexing tree as a branch node. • For this clustering, we use EM and EGED.

  21. EM clustering for OG • Gaussian mixture density with EGED • Log-likelihood function • Algorithm • E-step: Compute the conditional expectation of the complete log-likelihood • M-step: Update the parameter estimates

  22. STRG-Index structure • Introduction • Graph-based Approaches • Distance Measure • Clustering Object Graph • STRG-Index Structure • STRG-Index tree structure • STRG-Index construction • Search algorithm using STRG-Index • Experiments • Conclusions and Future works

  23. STRG-Index tree structure • [Top-level] Root node: a BG for each record • [Mid-level] Cluster node: a centroid OG of cluster for each record • [Low-level] Leaf node: OGs belonging to a cluster indexed by EGEDM(OGmem, OGclus) (a

  24. STRG-Index cont.

  25. STRG-Index construction • Root node: extracted BGs are stored. • All OGs sharing one background are in a same cluster node. • This can reduce the index size significantly. • Cluster node: synthesized centroid OGs are stored. • Each record is a representative OG of each cluster. • This centroid OG can be updated when member are changed. • Leaf node: actual OGs in a cluster are stored. • Records are indexed by EGEDM. • Key value is EGEDM (OGmem, OGclus)

  26. Search algorithm using STRG-Index • k-NN Search Algorithm Algorithm: k-NN Search Algorithm Input: a query example q, a STRG-Index TR, and k Output:k most similar OGs 1: extract OGq and BGgfromq; 2: find the similar BG for BGq in root node of TR using SimGraph; 3: find the similar OGclus for OGqin cluster node of TR using EGED; 4: Keyq = EGEDM(OGq, OGclus); 5: find k-nearest neighbor OGs to OGq using Key in leaf node of TR and Keyq 6: returnOGs;

  27. Experimental results • Introduction • Graph-based Approaches • Distance Measure • Clustering Object Graph • STRG-Index Structure • Experiments • Results of clustering • Indexing power • Real video streams • Conclusions and Future works

  28. Results of clustering • EM-EGED performance Clustering Error Rate Distortion

  29. Indexing power • STRG-Index Performance Building Index Time K-NN query performance Accuracy

  30. Real video streams • Real video data set

  31. Conclusions • Introduction • Graph-based Approaches • Distance Measure • Clustering Object Graph • STRG-Index Structure • Experiments • Conclusions

  32. Conclusions • We propose a new data structure, STRG for video based on graphs. It represents spatial and temporal relationships of video objects • We propose a new distance function, EGED in both non-metric and metric spaces for matching and indexing, respectively. • We propose a new indexing method, STRG-Index, which is faster and more accurate indexing.

More Related