1 / 16

Spatio-temporal Querying Recurrent Multimedia Databases using a Semantic Sequence State Graph

Spatio-temporal Querying Recurrent Multimedia Databases using a Semantic Sequence State Graph. By : M. M. Nair M. Sigdel R. S. Aygun. What does Spatio-Temporal mean ? What is recurrent Multimedia Database ? What is a Semantic Sequence State Graph ?

norris
Download Presentation

Spatio-temporal Querying Recurrent Multimedia Databases using a Semantic Sequence State Graph

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spatio-temporal Querying Recurrent Multimedia Databases using a Semantic Sequence State Graph By : M. M. Nair M. Sigdel R. S. Aygun

  2. What does Spatio-Temporal mean ? • What is recurrent Multimedia Database ? • What is a Semantic Sequence State Graph ? • Why do we need a different method of Indexing and Querying ? 1. Split Spatio-Temporal Querying ( Split STQ) 2. Coupled Spatio-Temporal Querying (Coupled STQ)

  3. Problems with Traditional methods, 1. Tree based approach, - moderately effective for Split STQ - Not when sequence data is to be retrieved - Always assumes data’s hierarchical format - No semantic Querying 2. Sequence matching approach, - moderately effective with Coupled STQ - time-consuming when data is not continue video pattern - Still used for detecting video duplication at times - No semantic Querying

  4. Semantic Modeling and Retrieval System (SMART) - Represents semantic information as grammar-based strings. X - Player 1 Y - Player 2 B - Ball f/b - forehand/backhand shot X7Y10B4 bY X7Y10B8

  5. Semantic Sequence State Graph (S3G) - Objects, events and locations mapped into “States” and “Transitions”. - A “State” is object-location pair with pointer to video clips. - A “Transition” is semantic sequence of events which results in location change for any object. CLIP 2 fX S1{ X=7Y=10B=7 } S2{ X=7Y=4B=4 } CLIP 1

  6. CLIP 5 Semantic Sequence State Graph (S3G) (Cont..) - List of possible states as outcome - Cyclic nature - List of Clips CLIP 9 S1 { X=5Y=6B=5 } bX bY fX fX S4 { X=5Y=6B=6 } S2 { X=5Y=9B=3 } CLIP 1 S3 { X=7Y=4B=4 } CLIP 7 CLIP 3 CLIP n

  7. Building S3G - S3G were build by decoding SMART string inputs. - Strings were given input with help of SQL and algorithm was developed to decode them and search the states and clips through transition paths. X7Y10B4fY4B4 fX S1 { X=7Y=10B=7 } S2 { X=7Y=4B=4 }

  8. CLIP 1 CLIP 3 Inserting State in S3G - Decode the string and find the existing state. X7B10fY10B7 CLIP 5 S2 { X=7Y=4B=4 } fX CLIP 9 S1 { X=7Y=10B=7 } CLIP 6 bX CLIP 3 CLIP 2 fY S3 { X=7Y=6B=4 } S4 { X=7Y=10B=10 } CLIP 4 CLIP 2 CLIP 6

  9. Querying S3G - Assumptions about querying S3G 1. S3G is mapped and stored in database as relational model. 2. States which are being queried have at least 1 clip common among them. 3. Only 1 S3G for whole Tennis match video. - Define initial state and query S3G. if desired state found, corresponding clips will be retrieved. - Define initial state and try to find desired state through path of transition with different operators. For ex, NEXT and EVENTUALLY.

  10. Querying S3G (Cont..) - NEXT operator helps retrieve the next possible states for the given state, but only those who has common clips with current given state. S1 {X=7Y=4B=4} fX CLIP 1 CLIP 9 S5 {X=7Y=10B=7} CLIP 2 CLIP 6 bX CLIP 5 CLIP 3 CLIP 2 fY S4 {X=7Y=6B=4} S2 {X=7Y=10B=10} CLIP 4 CLIP 2 CLIP 6

  11. Querying S3G (Cont..) - EVENTUALLY operator determines if the desired state eventually occurs for the given current state on the transition path. bX S5 {X=7Y=10B=7} S1 {X=7Y=6B=4} CLIP 3 CLIP 3 fX fY S4 {X=7Y=10B=4} S2 {X=7Y=10B=10} CLIP 3 CLIP 3

  12. Querying S3G (Cont..) - Any combination of NEXT and EVENTUALLY operator can be used to get the desired result. - User Interface allows queries build on previous states. So, complex and interactive query building is possible. - S3G contains only those stages which actually occurs in video, not all possible stages. So, size remains manageable and querying is done faster.

  13. Comparing S3G and Traditional methods, Assume one time line. No Semantic Events. Considers only single moving object location. No Such special operators. Queries take time in Seconds. • Considers time factor. • Considers Semantic Events as transitions. • Considers multiple object locations. • Facilitates queries with NEXT and EVENTUALLY. • Queries take time in milliseconds.

  14. Extendibility & Usage - Can be extended for any videos with 3 major qualities, 1. video content should be classifiable in spatial and temporal domains. 2. States should be repeated to take benefits over traditional methods. 3. Semantic events ( or transitions) should be present, which trigger state change in objects. - For ex, Sports videos, Surveillance videos, News-anchor, distance learning education videos and many more.

  15. Conclusion - S3G utilizes semantic events as transitions, while spatial data is maintained as states. - Semantic events helps in ordering temporal (time-based) data. - New operators like NEXT and EVENTUALLY helps making effective and faster queries. - S3G User Interface allows building interactive queries. - Can be applied to any domain where objects, events and locations are finite and repetition is frequent.

  16. Thank you

More Related