IAAI - Session 23F Robert S. Engelmore Award* Lecture Standing in for Deborah L. McGuinness*

1. An Open-World Iterative Methodology for the Development and Evaluation of Semantically-Enabled Applications IAAI - Session 23F Robert S. Engelmore Award* Lecture Standing in for Deborah L. McGuinness* July 17, 2013 Peter Fox (RPI) pfox@cs.rpi.edu, @taswegian Tetherless World Constellation

2. Outline • Origins of this effort and the working premise • Semantics in 2004 • Ontologies and the software and production! • Semantics between 2004 and 2009 • The design and development methodology • The expressivity and implementability balance and one more … • 2009-2013 … • And, a bit about what we are up to and where it is going… Tetherless World Constellation

3. Origins (1) … • In 2000-2001 the need for capturing and preserving knowledge in Earth sciences data settings became very clear but the barriers were high • In 2004 we started a virtual observatory project based on semantic technologies Tetherless World Constellation

4. Content: Coupling Energetics and Dynamics of Atmospheric Regions Community data archive for observations and models of Earth's upper atmosphere and geophysical indices and parameters needed to interpret them. Includes browsing capabilities by periods, instruments, models, …

5. Content: Mauna Loa Solar Observatory Near real-time data from Hawaii from a variety of solar instruments. Source for space weather, solar variability, and basic solar physics

6. Working premise • Scientists – actually ANYONE - should be able to access a global, distributed knowledge base of scientific data and information that: • appears to be integrated • appears to be locally available • is accessible in a suitable vocabulary • Then BAM - Data intensive – volume, complexity, mode, scale, heterogeneity, … in an OPEN WORLD 6

7. Origins (2) … • Use case driven – in solar and solar-terrestrial physics with an emphasis on instrument-based measurements and real data pipelines; we needed implementations • We knew we also needed integration and provenance (but that came later) • We aimed to push semantics into our systems to build new ‘prototypes’ but we ‘failed’ ;-) Tetherless World Constellation

8. Early days of VOs ? VO2 VO3 VO1 DBn DB2 DB3 … … … … DB1

9. The Astronomy approach; data-types as a service Limited interoperability • VOTable • Simple Image Access Protocol • Simple Spectrum Access Protocol • Simple Time Access Protocol VO App2 VO App3 VO App1 VO layer Lightweight semantics Limited meaning, hard coded Limited extensibility DBn DB2 DB3 … … … … DB1

10. In 2004 Tetherless World Constellation

11. Design and Development • Only develop ontologies that were required to answer specific use cases • Use whatever ontologies were available** • Would rules be needed? • We ignored query* Tetherless World Constellation

12. Science and technical use cases Find data which represents the state of the neutral atmosphere anywhere above 100km and toward the arctic circle (above 45N) at any time of high geomagnetic activity. Provide semantically-enabled, smart data query services via the Web for the Virtual Ionosphere-Thermosphere-Mesosphere Observatory that retrieve data, filtered by constraints on Instrument, Date-Time, and Parameter in any order and with constraints included in any combination.

13. Use Case example • Plot the neutral temperature from the Millstone-Hill Fabry Perot, operating in the non-vertical mode during January 2000 as a time series. • Plotthe neutral temperaturefrom the Millstone-HillFabry Perot, operatingin thenon-vertical modeduringJanuary 2000as atime series. • Objects: • Neutral temperature is a (temperature is a) parameter • Millstone Hill is a (ground-based observatory is a) observatory • Fabry-Perot is a interferometer is a optical instrument is a instrument • Non-vertical mode is a instrument operating mode • January 2000 is a date-time range • Time is a independent variable/ coordinate • Time series is a data plot is a data product

14. Knowledge representation • Statements as triples: {subject-predicate-object} interferometer is-a optical instrument Fabry-Perotis-a interferometer Optical instrumenthas focal length Optical instrument is-ainstrument Instrumenthas instrument operating mode Instrument has measured parameter Instrument operating modehas measured parameter NeutralTemperatureis-atemperature Temperature is-aparameter • A query*: select all optical instruments which have operating mode vertical • An inference: infer operating modes for a Fabry-Perot Interferometer which measures neutral temperature • ISWC paper award 2006, IAAI best paper (2007), Fox et al. 2009 in Computers and Geosciences.

15. Added value Education, clearinghouses, other services, disciplines, etc. Semantic interoperability Added value Added value Semantic query, hypothesis and inference Semantic mediation layer – mid-upper-level Added value VO API Web Serv. VO Portal Query, access and use of data Mediation Layer • Ontology - capturing concepts of Parameters, Instruments, Date/Time, Data Product (and associated classes, properties) and Service Classes • Maps queries to underlying data • Generates access requests for metadata, data • Allows queries, reasoning, analysis, new hypothesis generation, testing, explanation, etc. Semantic mediation layer - VSTO - low level Metadata, schema, data DBn DB2 DB3 … … … … DB1 Fox - APAC 2007, Driving e-research: Grids and Semantics

16. Semantic filtering by domain or instrument hierarchy Partial exposure of Instrument class hierarchy - users seem to LIKE THIS Fox - APAC 2007, Driving e-research: Grids and Semantics

18. Inferred plot type and return required axes data

19. Semantic Web Services

20. Semantic Web Services OWL document returned using VSTO ontology - can be used both syntactically or semantically Fox - APAC 2007, Driving e-research: Grids and Semantics

21. http://escience.rpi.edu/schemas/vsto_all.owl

22. Developing ontologies • Use cases and small team • Identify classes and minimal properties (leverage controlled vocab.) • Review, vet, publish • Only code them (in RDF or OWL) when needed (CMAP, …) • Ontologies: small and modular

23. Implications and OWL 1.0 • Lack of numeric support meant that the the rules and procedural logic were implemented in java, i.e. in the code • On several occasions the tools (not to be named) pushed us into OWL-Full, introduced inconsistencies, etc. • Finally, they stabilized, and in 2005 (and again in 2006 and twice in 2007) we had stable releases Tetherless World Constellation

24. Expressivity VSTO 1.0 Tetherless World Constellation

25. Expressivity VSTO dev. version Tetherless World Constellation

26. Yikes Tetherless World Constellation

27. Ontologies and the software • Protégé 2.x and then 3.x built from our ontology on the web • Java class generation • Eclipse as a development environment • Leveraged a portal code base (from the Earth System Grid project) Tetherless World Constellation

28. Implementation choices • Our big challenge was time – in use cases and in the representation • Depending on the level of granularity there were > 200,000 day-time records, and > 70,000,000 sub-day time intervals – no triple store could handle this** • Reasoning in finite time does not mean 3-4 secs! • We descoped our effort to delay use cases such as: find all neutral temperature data around the summer solstice for the last decade • We chose a minimal time encoding in the ontology and delegated that to a relational DB Tetherless World Constellation

30. Web Service VSTO - semantics and ontologies in an operational environment: www.vsto.org Fox - APAC 2007, Driving e-research: Grids and Semantics

31. Semantic Web Benefits (1) • Unified/ abstracted query workflow: Parameters, Instruments, Date-Time • Decreased input requirements for query: in one case reducing the number of selections from eight to three • Generates only syntactically correct queries: which was not always insurable in previous implementations without semantics

32. Semantic Web Benefits (2) • Semantic query support: only expose coherent query (portal and services) • Semantic integration: mediated • understanding of coordinate systems, relationships, data synthesis, transformations. • returns independent variables and related parameters • A broader range of potential users (PhD scientists, students, professional research associates and those from outside the fields)

33. Evaluation • Formative and Summative Tetherless World Constellation

34. Iteration • We needed the ability to evolve the ontology and not break the framework • As we broadened re-use of these ontologies and creation of new ones Tetherless World Constellation

35. Maintenance • Support for collaborative feedback, evolution • Change management • Support for ‘comments’ and ‘annotations’, i.e. self-documentation • Package management: creation, dependency, consistency checking Tetherless World Constellation

36. Semantics between 2004 and 2009 • Ontologies were needed for data integration and provenance and mediation for data mining • Protégé 3.x and then 4.0 came out • SWOOP development was interrupted • Cmap added OWL predicate support* • SPARQL became a recommendation • Triple stores exploded in use and capability • Linked Open Data started to take off • Pellet 2.0 came out • We invaded OWLED 2006, 2007, 2009, (2010) Tetherless World Constellation

37. Semantics approach • Use cases • Stakeholders • Distributed authority • Access control • Ontologies • Maintaining Identity

38. Working with knowledge Rule execution Expressivity Implement -ability Query Inference Maintainability/ Extensibility

39. Expressivity/ Implementation Declarative Procedural Linked open data URI/http/RDF * Ontology encoded

40. Semantics between 2009 and 2013 • Semantic data frameworks (SeSF) • Substantial knowledge provenance work • Data quality, uncertainty and bias representations and applications • Multi-sensor advisor** • Applications: • Sea Ice, Carbon Observatory, Integrated Ecosystem Assessments, globalchange.gov, ocean.data.gov, energy.data.gov …. Tetherless World Constellation

41. Anomaly Example: South Pacific Anomaly Anomaly MODIS Level 3 dataday definition leads to artifact in correlation

42. …is caused by an Overpass Time Difference

43. RuleSet Development [DiffNEQCT: (?s rdf:type gio:RequestedService), (?s gio:input ?a), (?a rdf:type gio:DataSelection), (?s gio:input ?b), (?b rdf:type gio:DataSelection), (?a gio:sourceDataset ?a.ds), (?b gio:sourceDataset ?b.ds), (?a.ds gio:fromDeployment ?a.dply), (?b.ds gio:fromDeployment ?b.dply), (?a.dply rdf:type gio:SunSynchronousOrbitalDeployment), (?b.dply rdf:type gio:SunSynchronousOrbitalDeployment), (?a.dply gio:hasNominalEquatorialCrossingTime ?a.neqct), (?b.dply gio:hasNominalEquatorialCrossingTime ?b.neqct), notEqual(?a.neqct, ?b.neqct) -> (?s gio:issueAdvisory giodata:DifferentNEQCTAdvisory)] Multi-sensor Data Synergy Advisor (NASA), Leptoukh, Lynnes, Zednik, et al.

44. Advisor Knowledge Base Advisor Rules test for potential anomalies, create association between service metadata and anomaly metadata in Advisor KB

45. Semantic Advisor Architecture RPI Multi-sensor Data Synergy Advisor (NASA), Leptoukh, Lynnes, Zednik, et al.

46. Advisory Report • Tabular representation of the semantic equivalence of comparable data source and processing properties • Advise of and describe potential data anomalies/bias

47. Advisory Report (Dimension Comparison Detail)

48. Going forward… • No surprise – lots of application • Water quality, environmental conditions • Global change information system; traceable accounts • Deep Carbon Observatory • And … maturing ontologies • Discrete and Continuous Additive effects (in Event Calculus) – driven by the desire to explore physical processes in Earth’s atmosphere when a volcano erupts

49. RPI Tetherless World Constellation tw.rpi.edu • Government Data • Health care/Life Sciences • Environmental Informatics • Future Web • Web Science • Policy • Social Themes Hendler • Xinformatics • Data Science • Semantic eScience • Data Frameworks Fox Lots of RDF Web Infrastructure Scaling and Distributed Query and Reasoning AI, Rule reasoning Visualizing SW ‘data’ Social SW, SMWiki Policy Lang Ontologies/Tools, … • Semantic Foundations • Knowledge Provenance • Inference • Trust McGuinness Luciano, Erickson + ~ 40 = Post-doc, Staff, Grad, UGrad

50. Respect vocabularies!