1 / 36

Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic

Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic. Del Bimbo, E. Vicario and D. Zingoni IEEE Transactions on Knowledge and Data Engineering, 7(4), August 1995. Outline. Overview Spatial logic Region-based formulation Object-based formulation

kylene
Download Presentation

Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic Del Bimbo, E. Vicario and D. Zingoni IEEE Transactions on Knowledge and Data Engineering, 7(4), August 1995.

  2. Outline • Overview • Spatial logic • Region-based formulation • Object-based formulation • Temporal logic • Sequence Description using STL • A System for Image Sequence Retrieval

  3. Overview:Spatio-temporal logic (STL) • A symbolic representation of spatio-temporal relationships between objects within image sequences. • Completely based on logic. • Two main components: • Spatial logic for individual scenes • Temporal logic for scene sequences

  4. Spatial logic • Expresses geometric ordering and relationships between objects • In fact, between the projections of the objects in N-dimensional space. • Based on knowing the points (or regions) that are covered by the object.

  5. Spatial logic (cont.) • A scene is a represented as a triple (V, Obj, F): • V: N-dimensional space • Mostly N=2 or 3 • Obj: a set of objects in V • F: a mapping from Obj to the powerset of V, i.e. the set of all subsets of V. • Associates each object with a set of points over which it stands, i.e. F(p) := { r  V | p stands over r)

  6. Spatial logic (cont.) • V can be partitioned into a grid of rectangular regions. • A scene will be then represented as a discrete scene. • And F will be then a mapping between objects and the regions they cover.

  7. Example e2 I24 I23 p2 I22 I21 p1 e1 I11 I12 I13 I14

  8. Spatial logic (cont.) • Two possible formulations: • Region-based formulation • Object-based formulation

  9. Region-based formulation • Express the positioning of a set of objects with respect to a single region in the scene. • A spatial assertion has the form: where • In addition, there are some shorthand operations: • spatial-eventually (can be +ve or –ve) • spatial-always (can be +ve or –ve)

  10. Semantics • (SE, J , en ) |= p- iff the orthogonal projections on axis en of region g(J) and object p have a nonempty intersection • (SE, J , en ) |= - iff the spatial assertion (SE, J , en ) |=  does not hold • (SE, J , en ) |= 12- iff both (SE, J , en ) |= 1 and (SE, J , en ) |= 2

  11. Semantics (Cont.) • (SE, J , en ) |= 1untS+2- iff there esists a region g(J’) which is reached from region g(J) moving along the positive direction of axis en such that assertion 2 holds in region g(J’) and 1 holds in all the regions from g(J) to g(J’) • (SE, J , en ) |= 1untS-2- iff there esists a region g(J’) which is reached from region g(J) moving along the negetive direction of axis en such that assertion 2 holds in region g(J’) and 1 holds in all the regions from g(J) to g(J’)

  12. Shorthands • (SE, J , en ) |= ◊S+/--  will eventually hold in some of the regions encountered moving from region g(J) along axis en • (SE, J , en ) |= S+/--  holds in all the regions encountered moving from region g(J) along axis en

  13. Example em g(J) g(K) p en

  14. Object-based formulation • Express the relationships between objects. • A spatial assertion has the form: ( read:  holds in q) where q is an object, expresses that (SE, J , en ) |= holds in any region g(J) containing q • Five different situations occur as follows:

  15. q p p p p (SE, q , x ) |= p - expresses that the projection of q on the x axis is entirely contained in the projection of p

  16. (SE, q , x ) |= ◊S+ p - expresses that every point of the projection of q on the x axis has at least one point of the projection of p to its right side q p p p p p

  17. (SE, q , x ) |= q untS+ p - expresses that the projection of q extends until one point is found that belongs to the projection of p q q p p p p

  18. (SE, q , x ) |=  p - expresses that the projection of q on the x axis does not intersect with the projection of p q p p

  19. - expresses that some of the regions of q are aligned with some of the regions of p ((SE, q , x ) |=  p ) q q p p p p p p p p p p p

  20. Temporal logic • In general, used to express the ordering of states or actions in time. • A temporal (or state assertion) has the form: where

  21. Semantics • (, k) |= iff the spatial assertion  holds in the kth scene of sequence  • (, k) |= iff the temporal assertion (, k) |=  does not hold • (, k) |= 12iff both (, k) |= 1 and (, k) |= 2 hold • (, k) |= 1untt2iff 2 holds in a scene with index k’ > k and 1 holds in all the scenes form k to k’

  22. Shorthands Two shorthands can be derived • (, k) |= ◊tmeans that  will hold in some scene subsequent to the kth one◊t := true untt • (, k) |= tmeans that  holds in all the scenesubsequent to the kth onet := ( true untt(  ) )

  23. Describing Scene Sequences S1 S2 S3 S4 S5 1 1 1 2 • each node is labeled with the spatial assertion, satisfied in its scene • S1, S2 , S3, S4 and S5 are different states on a time line • (, k) |= ( 1 untt2 ) holds for …………… • (, k) |= ◊t2 holds for ……………… • (, k) |= t ( 1 2 ) holds for ……………..

  24. Sequence Description Using STL • 3D scene-based descriptions are used to avoid ambiguity • Two different descriptions are possible 1) Observer-centered description 2) Object-centered description

  25. Two Different Descriptions • Observer-centered description -- images of the same scene taken from different viewpoints -- valid only when the camera is fixed • Object-centered description -- how one object sees the rest of the scene-- does not depend on the camera position

  26. Example

  27. Example • the spatial positions of the house, h as perceived by the car, c along the course of the scene • considering object-centered descriptions and referring to the reference system Ec associated with the car c • 1 := (SEc , c, x) |= h • 2 :=  ((SEc , c, x) |= h ) • 3 := (SEc , c, y) |= h • 4 := (SEc , c, z) |= h • 5 :=  ((SEc , c, z) |= h ) The description of the scene using the above assertions can be as follows:

  28. A System for Image Sequence Retrieval A. Iconic Querying 1) Visual Querying 2) Automatic Parsing B. Retrieval from Database1) Sequence Representation (created manually) 2) Sequence Retrieval

  29. Retrieval from Database • To allow for different level of details in queries, three levels of operations are used. • Level 1: Least details • 3 possibilities (before, overlapping, after) • Level 2: Modest details • 6 possibilities • Level 3: Very fine details • 13 possibilities (STL full representational power)

  30. Level-1 Operators

  31. Level-2 Operators

  32. User Interface of the Retrieval System

  33. Example 1 (Query Specification)

  34. Result of Retrieval Only two frames of the Sequence are shown

  35. Example 2 (Query Specification) Definition of motion of one car during the play back of the other

  36. Result of Retrieval Two frames of the sequence

More Related