Spatiotemporal and Thematic Semantic Analytics

1. Matthew Perry PhD. Student Kno.e.sis Center, Wright State University Spatiotemporal and Thematic Semantic Analytics

2. Three Dimensions of Information Temporal Dimension: When Thematic Dimension: What North Korea detonates nuclear device on October 9, 2006 near Kilchu, North Korea Spatial Dimension: Where

3. Using named relationships to connect thematic entities with spatial locations in a variety of meaningful ways (different contexts) E2:Soldier E4:Address lives_at located_at located_at lives_at E6:Address Georeferenced Coordinate Space (Spatial Regions) E1:Soldier E1:Soldier occurred_at E7:Battle assigned_to participates_in E8:Military_Unit E8:Military_Unit participates_in assigned_to E5:Battle occurred_at Residency Battle Participation E3:Soldier Named Places Spatial Occurrents Dynamic Entities

4. Temporal Properties of Paths Soldier_2 assigned_to:[3, 12] Soldier_1 assigned_to:[6, 13] Platoon_45 assigned_to:[14, 20] assigned_to:[19, 25] Soldier_4 Soldier_3 Which soldiers were members of Platoon_45 during the interval [5, 15] ? Which soldiers were members of Platoon_45 at the same time ?

5. Example: Bioterrorism assigned_to E10:Doctor E9:Base E6:Attack used_in Spotted Close in Time [0, 2] stationed_at E8:Soldier carried_out E1:Soldier [0, 2] E4:Chemical member_of E5:Terrorist E7:Platoon [0, 10] member_of causes E3:Disease exhibits [8, 10] spotted_at [3, 5] E2:Symptom sign_of E11:Location E13:Soldier participated_in [4, 6] After the Battle E14:Battle E12:Platoon member_of Near in Space participated_in

6. New types of applications exploiting named relationships between entities (semantic graphs) Data Mining – Link Mining, Graph Mining Semantic Web – Semantic Analytics Analysis of relationships in Large RDF graphs Detecting Conflict of Interest, Collaboration, Insider Threat Problem Background

7. fname lname Semantically Connected “Matt” “Perry” Semantic Connectivity • Two entities e1 and en are semantically connected if there exists a sequence e1, P1, e2, P2, e3, … en-1, Pn-1, en in an RDF graph where ei, 1  i  n, are entities and Pj, 1  j < n, are properties name “Wright State University” &r1 &r6 worksFor associatedWith &r5 name “Knoesis Center”

8. Semantically Similar “Fred” “Smith” Semantic Similarity • Two entities e1 and f1 are semantically similar if there exist two semantic paths e1, P1, e2, P2, e3, … en-1, Pn-1, en and f1, Q1, f2, Q2, f3, …, fn-1, Qn-1, fn semantically connecting e1 with en and f1 with fn, respectively, and that for every pair of properties Pi and Qi, 1  i < n, either of the following conditions holds: Pi = Qi or Pi Qi or Qi Pi ( means rdf:subPropertyOf ) • We say that the two paths originating at e1 and f1, respectively, are semantically similar and thus so are the entities e1 and f1 Passenger Ticket Corporate Account “Bill” fname purchased &r2 paidby &r3 &r1 “Jones” lname fname &r7 purchased &r8 paidby &r9 lname

9. Spatial and Temporal Semantic Analytics Soldier_6 Soldier_7 Member_of Spatial Relationship Temporal Relationship Axis Soldier_1 Position_3 Employed_unit Soldier_2 Trained_at: [2, 6] Held_position: [4, 12] Member_of Trained_at: [8, 10] Position_1 9th SS Panzer Soldier_3 101st Airborne Specializes_in Held_position: [2, 10] Explosives Allies Employed_unit Camp Claiborne Held_position: [3, 7] Employed_unit Position_2 Soldier_4 Soldier_5 Trained_at: [1, 5] 82nd Airborne Member_of Thematic Relationship

10. Examples: Which Military Units have spatial extents which are within 20 miles of (48.45° N, 44.30° E) in the context of Battle participation? Which infantry unit’s operational area overlaps the operational area of the 3rd Armored Division? Spatial Relationships Between Entities Quantitative Relationships Qualitative Relationships

11. Temporal Relationships Between Entities • Examples: • Which Speeches by President Roosevelt were given within one day of a major battle? • Who were members of the 101st Airborne during November 1944? Quantitative Relationships Qualitative Relationships

12. Define a Domain-independent Ontology which integrates Spatial and Thematic Knowledge Allows exploiting the flexibility and extensibility of Semantic Web data models Can deal with incompleteness of information on the web Incorporate temporal metadata into this model Identify and formalize basic spatial and temporal relationship-based query operators which complement current thematic operators of SemDis Current Work Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACM-GIS '06), Arlington, VA, November 10 - 11, 2006

13. Upper-level Ontology modeling Theme and Space Continuant Occurrent Dynamic_Entity Named_Place Spatial_Occurrent located_at occurred_at Spatial_Region rdfs:subClassOf property Final Classification of Domain Classes depends upon the intended application Occurrent: Events – happen and then don’t exist Continuant: Concrete and Abstract Entities – persist over time occurred_at: Links Spatial_Occurents to their geographic locations located_at: Links Named_Places to their geographic locations Named_Place: Those entities with static spatial behavior (e.g. building) Dynamic_Entity: Those entities with dynamic spatial behavior (e.g. person) Spatial_Region: Records exact spatial location (geometry objects, coordinate system info) Spatial_Occurrent: Events with concrete spatial locations (e.g. a speech)

14. Upper-level Ontology Continuant Occurrent Named_Place Dynamic_Entity located_at occurred_at Spatial_Occurrent Spatial_Region City Person trains_at Speech gives Politician participates_in Military_Unit Military_Event Soldier assigned_to on_crew_of Bombing used_in Battle Vehicle Domain Ontology rdfs:subClassOf used for integration rdfs:subClassOf relationship type

15. Thematic Context • Specifies a type of connection between resources in the thematic dimension of our ontology Schema on_crew_of used_in Person Military_Vehicle Bombing on_crew_of used_in ‘John Smith’ ‘B24#123’ ‘Bombing#456’ Path Template Person.on_crew_of.Military_Vehicle.used_in.Bombing ‘John Smith’.on_crew_of.Military_Vehicle.used_in.Bombing

16. Thematic Query (ρ-theme) ρ-theme (G, tc)  {pt} Example: find all Bombing events connected to ‘John Smith’ through a vehicle participation context ρ-theme (G, ‘John Smith’.on_crew_of.Military_Vehicle. used_in.Bombing) Result ‘John Smith’.on_crew_of.‘B-24#123’.used_in.‘Bombing#456’ ‘John Smith’.on_crew_of.‘B-24#123’.used_in.‘Bombing#789’ G = temporal RDF Graph, tc = thematic context, pt = thematic context instance

17. Thematic Contexts Linking Non-Spatial Entities to Spatial Entities E2:Soldier E4:Address lives_at located_at located_at lives_at E6:Address Georeferenced Coordinate Space (Spatial Regions) E1:Soldier E1:Soldier occurred_at E7:Battle assigned_to participates_in E8:Military_Unit E8:Military_Unit participates_in assigned_to E5:Battle occurred_at Residency Battle Participation E3:Soldier Named Places Spatial Occurrents Dynamic Entities

18. Use Temporal RDF Graphs defined by Gutiérrez, et al1 Models Absolute Time Considers time as a discrete, linearly-ordered domain Associate time intervals with statements which represent the valid-time of the statement Essentially a quad instead of a triple Incorporation of Temporal Information into RDF 1. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman: Temporal RDF. ESWC 2005: 93-107

19. Example Temporal Graph: Platoon Membership assigned_to [5, 15] E4:Soldier assigned_to [1, 10] E1:Soldier E2:Platoon assigned_to [11, 20] E3:Platoon E5:Soldier assigned_to [5, 15]

20. Provide a means to query about spatial, thematic, and temporal properties/relationships of all entities Path Query in the thematic dimension Thematic Context Associate spatial region with a path Associate temporal interval with a path Query operators based on properties of and relationships between associated spatial regions and temporal intervals Querying in the STT dimensions

21. Deriving Spatial and Temporal Properties of Entities ρ-spatial_extent (G, {pt})  {pt, sr} Retrieves the Spatial_Region connected (through occurred_at or located_at) to the terminating Spatial Entity of the context instances Example: Where were the battles in which the ‘101st Airborne Division’ fought? ρ-spatial_extent (G,ρ-theme (G, ‘101st Airborne Division’. .participates_in.Battle)) Result ‘101st Airborne Division’.participates_in. ‘Operation Market Garden’, ‘Geom#123’ G = temporal RDF Graph, pt = thematic context instance, sr = spatial region

22. Temporal Properties of Context Instances Intersection [6, 12] assigned_to:[3, 12] assigned_to:[6, 20] Soldier#123 Platoon#456 Soldier#789 Range [3, 20]

23. Deriving Spatial and Temporal Properties of Entities ρ-temporal_intersect ({pt})  {pt, [t1, t2]} Retrieves the interval during which the entire path is valid Example: Which Soldiers were members of the ‘1st Armored Division’ at the same time? ρ-temporal_intersect (ρ-theme (G, Soldier.assigned_to.‘1st Armored Division’.assigned_to.Soldier)) Result ‘Fred Smith’.assigned_to.’1st Armored Division’.assigned_to. ‘Bill Jones’, [1941:04:15, 1943:02:30] pt = thematic context instance, [t1, t2] = temporal interval

24. Querying Relationships Between Entities (Thematic Context Instance tp, Temporal Interval [ti, tj], Spatial Region sr) 3 Spatial Relationship Operators ρ-spatial_locate ρ-spatial_eval ρ-spatial_find 3 Temporal Relationship Operators ρ-temporal_restrict ρ-temporal_eval ρ-temporal_find Identify 6 major Spatiotemporal Relationship Queries which can be answered by combining previously defined operators

25. S1 ρ-spatial_extent (G, ρ-theme (G, ‘101st Airborne Division’. participates_in.Battle)) S2 ρ-spatial_extent (G, ρ-theme (G, ‘1st Armored Division’. participates_in.Battle)) ANS  ρ-temporal_intersect (ρ-spatial_eval (S1, S2, distance (S1, S2) 10 miles) Spatiotemporal Relationship Queries Example: When did the 101st Airborne Division come within 10 miles of the 1st Armored Division in the context of Battle participation

26. Spatiotemporal Semantic Associations • Define setting as a region of space in combination with an interval of time • How is entity X related to Spatial setting S? ( ρ (entity, setting)) Group 1 Account_1234 125 Broad Street Fred Jim Attack Site How is Group 1 connected to the setting of the expected attack?

27. Spatiotemporal Semantic Associations How are entity X and entity Y related w.r.t Spatial setting S? ρ (entity, entity, setting) Group 2 Group 1 Account_1234 Fred Jim 125 Broad Street How are Group 1 and Group 2 connected with respect to the location of the crime?

28. Idea of Virtual Links between entities based on Spatiotemporal information Possible definition of rules to define a virtual link type Collaboration: entity X and Y are in close ST proximity more often than a given threshold Knows: entity X and Y are in close ST proximity regularly Spatiotemporal Semantic Associations

29. How do temporal relationships affect association semantics 2 works_for relationships (overlapping times, disjoint times, etc) Complex queries based on all 3 dimensions Which location is the most likely storage facility for exfiltrated weapon material Thematic (correct capabilities, linked to correct people) Spatial (where was the material last seen) Temporal (how long can the material stay out of storage) Other Aspects