Selected Semantic Web Trends, Progress, and Directions

1. Selected Semantic Web Trends, Progress, and Directions Deborah McGuinness Acting Director and Senior Research Scientist Knowledge Systems, AI Laboratory Stanford University http://www.ksl.stanford.edu/people/dlm CEO McGuinness Associates (soon Tetherless World Constellation Chair RPI)

2. Semantic Web Layers(and DLM) Ontology Level • Language (OWL, IKL, DAML+OIL, OIL, CLASSIC, …) • Environments (FindUR, Chimaera, Ontolingua, OntoBuilder / Server, Sandpiper Tools, Cerebra, …) • Services OWL-S, SWSL, … • Standards body leverage (W3C’s WebOnt, W3C’s Semantic Web Best Practices, EU/US Joint Com, OMG ODM, W3C’s RIF, Scientific Markup Standards, NAPLPS…) Query • OWL-QL, … Rules • RIF, SWRL , … Logic • Description Logics, FOL Proof • PML, Inference Web Services and Infrastructure Trust • IWTrust http://www.w3.org/2003/Talks/1023-iswc-tbl/slide26-0.html, http://flickr.com/photos/pshab/291147522/ Deborah L. McGuinness

3. Applications Applications • Virtual Solar Terrestrial Observatory, SESDI, SKIF, SSOA, BISTI, … • Cognitive Assistants (PAL, CALO, GILA, …) • Domain ontologies (immunology, atmosphere, volcano, …) & environments • Tools academic & industry (Sandpiper,…) • Startups: Health Information Flow, Blue Chip Expert, Katalytik, Radar Networks… Deborah L. McGuinness

4. Two Trends • Trust/Explanation • Trust Motivation and Requirements • Inference Web approach • Interlingua: Proof Markup Language • Explanation & Provenance Browsing / Abstraction / Presentation Strategies (examples from DARPA PAL, NSF TAMI, DTO NIMD, NSF VSTO) • Trust (examples from Wikipedia study for explainable knowledge aggregation with extensions to text analytics, connection to NSF TAMI) • Semantic Integration of Scientific Data • Virtual Observatories (e.g., NSF-funded Virtual Solar Terrestrial Observatory) • Semantically-Enabled Scientific Data Integration (NASA-funded) • Conclusion / Discussion Deborah L. McGuinness

5. Virtual Observatories Scientists should be able to access a global, distributed knowledge base of scientific data that: • appears to be integrated • appears to be locally available But… data is obtained by multiple instruments, using various protocols, in differing vocabularies, using (sometimes unstated) assumptions, with inconsistent (or non-existent) meta-data. It may be inconsistent, incomplete, evolving, and distributed Deborah L. McGuinness

6. Virtual Observatory Defined • Workshop: A Virtual Observatory (VO) is a suite of software applications on a set of computers that allows users to uniformly find, access, and use resources (data, software, document, and image products and services using these) from a collection of distributed product repositories and service providers. A VO is a service that unites services and/or multiple repositories. • VxOs - x is one discipline Deborah L. McGuinness

7. Virtual Observatories in Practice Make data and tools quickly and easily accessible to a wide audience. Operationally, virtual observatories need to find the right balance of data/model holdings, portals and client software that a researchers can use without effort or interference as if all the materials were available on his/her local computer using the user’s preferred language. They are likely to provide controlled vocabularies that may be used for interoperation in appropriate domains along with database interfaces for access and storage and “smart” search functions and tools for evolution and maintenance. Deborah L. McGuinness

8. Virtual Solar Terrestrial Observatory (VSTO) • a distributed, scalable education and research environment for searching, integrating, and analyzing observational, experimental, and model databases. • subject matter covers the fields of solar, solar-terrestrial and space physics • it provides virtual access to specific data, model, tool and material archives containing items from a variety of space- and ground-based instruments and experiments, as well as individual and community modeling and software efforts bridging research and educational use • 3 year NSF-funded project just beginning the second year Deborah L. McGuinness

9. Content: Coupling Energetics and Dynamics of Atmospheric Regions WEB Community data archive for observations and models of Earth's upper atmosphere and geophysical indices and parameters needed to interpret them. Includes browsing capabilities by periods, instruments, models, … Deborah L. McGuinness

10. Content: Mauna Loa Solar Observatory Near real-time data from Hawaii from a variety of solar instruments. Source for space weather, solar variability, and basic solar physics Other content used too – CISM – Center for Integrated Space Weather Modeling Deborah L. McGuinness

11. Content: Volcanoes…Mt. Spurr, AK. 8/18/1992 eruption, USGS http://www.avo.alaska.edu/image.php?id=319 Deborah L. McGuinness

12. Eruption cloud movement from Mt.Spurr, AK,1992 USGS Deborah L. McGuinness

13. Tropopause http://aerosols.larc.nasa.gov/volcano2.swf

14. Atmosphere Use Case • Determine the statistical signatures of both volcanic and solarforcings on the height of the tropopause From paleoclimate researcher – Caspar Ammann – Climate and Global Dynamics Division of NCAR - CGD/NCAR Layperson perspective: - look for indicators of acid rain in the part of the atmosphere we experience… (look at measurements of sulfur dioxide in relation to sulfuric acid after volcanic eruptions at the boundary of the troposphere and the stratosphere) Nasa funded effort with Fox - NCAR, Sinha - Va. Tech, Raskin - JPL Deborah L. McGuinness

15. Use Case detail: A volcano erupts • Preferentially it’s a tropical mountain (+/- 30 degrees of the equator) with ‘acidic’ magma; more SiO2, and it erupts with great intensity so that material and large amounts of gas are injected into the stratosphere. • The SO2gasconverts to H2SO4 (Sulfuric Acid) + H2O (75% H2SO4 + 25% H2O). The half life of SO2 is about 30 - 40 days. • The sulfuric acidcondensates to little super-cooledliquiddroplets. These are the volcanic aerosol that will linger around for a year or two. • Brewer Dobson Circulation of the stratosphere will transportaerosol to higher latitudes. The particles generate great sunsets, most commonly first seen in fall of the respective hemisphere. The sunlight gets partially reflected, some part gets scattered in the forward direction. • Result is that the direct solar beam is reduced, yet diffuse skylight increases. The scattering is responsible for the colorful sunsets as more and more of the blue wavelength are scattered away.in mid-latitudes the volcanic aerosol starts to settle, but most efficient removal from the stratosphere is through tropopause folds in the vicinity of the stormtracks. • If particles get over the pole, which happens in spring of the respective hemisphere, then they will settle down and fall onto polar icecaps. Its from these icecaps that we recover annual records of sulfateflux or deposit. • We get icecores that show continuous deposition information. Nowadays we measure sulfate or SO4(2-). Earlier measurements were indirect, putting an electric current through the ice and measuring the delay. With acids present, the electric flow would be faster. • What we are looking for are pulse likeevents with a build up over a few months (mostly in summer, when the vortex is gone), and then a decay of the peak of about 1/e in 12 months. • The distribution of these pulses was found to follow an extreme value distribution (Frechet) with a heavy tail. Deborah L. McGuinness

16. Use Case detail: … climate • So reflection reduces the total amount of energy, forward scattering just changes the beam, path length, but that's it. • The dryfogs in the sky (even after thunderstorm) still up there, thus stratosphere not troposphere. • The tropical reservoir will keep delivering aerosol for about two years after the eruption. • The particles are excellent scatterers in short wavelength. They do absorb in NIR and in IR. Because of absorption, there is a local temperature change in the lower stratosphere. • This temperature change will cause some convective motion to further spread the aerosol, and second: Its good factual stuff. Once it warms up, it will generate a temperature gradient. Horizontal temperaturegradients increase the baroclinicity and thus storms, and they speedup the local zonalwinds. This change in zonal wind in high latitudes is particularly large in winter. This increased zonal wind (Westerly) will remove all coldair that tries to buildup over winter in high arctic. • Therefore, the temperature anomaly in winter time is actually quite okay. • Impact of volcanoes is to cool the surface through scattering of radiation. • In winter time over the continents there might be some warming. In the stratosphere, the aerosol warm. • The amount of GHG emitted is comparably small to the reservoir in the air. • The hydrologic cycle responds to a volcanic eruption. Deborah L. McGuinness

17. Atmosphere (portions from SWEET) Deborah L. McGuinness

18. Atmosphere II Deborah L. McGuinness

19. Deborah L. McGuinness

20. A few observations worth noting • CMAPS have been convenient knowledge capture tools • We facilitate knowledge acquisition meetings AND provide a starting point • We are experiencing good reuse of ontologies and infrastructure • Next – Quick VSTO walk thru Deborah L. McGuinness

21. www.vsto.org Deborah L. McGuinness

22. Deborah L. McGuinness

23. Semantic filtering by domain or instrument hierarchy Partial exposure of Instrument class hierarchy - users seem to like this Deborah L. McGuinness

24. Deborah L. McGuinness

25. Inferred plot type and return required axes data Deborah L. McGuinness

26. VSTO • Conceptual model and architecture developed by combined team; KR experts, domain experts, and software engineers • Semantic framework developed and built with a small, cohesive, carefully chosen team in a relatively short time (deployments in 1st year) • Production portal released, includes security, etc. with community migration (and so far endorsement) • VSTO ontology version 1.0, (vsto.owl) • Web Services encapsulation of semantic interfaces • More Solar Terrestrial use-cases to drive the completion of the ontologies - filling out the instrument ontology • Using ontologies in other applications (volcanoes, climate, …) Deborah L. McGuinness

27. Semantic Web Methodology and Technology Development Process • Establish and improve a well-defined methodology vision for Semantic Technology-based application development Adopt Technology Approach Leverage Technology Infrastructure Expert Review & Iteration Rapid Prototype Open World: Evolve, Iterate, Redesign, Redeploy Use Tools Analysis Use Case Develop model/ ontology Small Team, mixed skills Joint with P. Fox Deborah L. McGuinness

28. Benefits • Unified query workflow • Decreased input requirements for query: in one base reducing the number of selections from eight to three • Interface generates only syntactically correct queries: which was not always true in previous implementations without semantics • Semantic query support: by using background ontologies and a reasoner, our application has the opportunity to only expose coherent queries • Semantic integration: in the past users had to remember (and maintain codes) to account for numerous different ways to combine and plot the data whereas now semantic mediation provides the level of sensible data integration required • understanding of coordinate systems, relationships, data synthesis, transformations, etc. • A broader range of potential users (PhD scientists, students, professional research associates and those from outside the fields) Deborah L. McGuinness

29. Explanation Transition Deborah L. McGuinness

30. General Motivation Provide interoperableknowledgeprovenance infrastructure that supports explanations of sources, assumptions, learned information, and answers as an enabler for trust. Interoperability– as systems use varied sources and multiple information manipulation engines, they benefit more from encodings that are shareable & interoperable Provenance– if users (humans and agents) are to use and integrate data from unknown, unreliable, or evolving sources, they need provenance metadata for evaluation Explanation/Justification– if information has been manipulated (i.e., by sound deduction or by heuristic processes), information manipulation trace information should be available Trust – if some sources are more trustworthy than others, representations should be available to encode, propagate, combine, and (appropriately) display trust values Deborah L. McGuinness

31. Requirements gathered from… DARPA Agent Markup Language (DAML) Enable the next generation of the web DARPA Personal Assistant that Learns (PAL) Enable computer systems that can reason, learn, be told what to do, explain actions, reflect on their experience, & respond robustly to surprise DARPA Integrated Learning (IL) Enable learning general plans or processes from human users by being shown one example by opportunistically assembling knowledge from many different sources, including generating it by reasoning, in order to learn. DARPA Rapid Knowledge Formation (RKF) Allow distributed teams of subject matter experts to quickly and easily build, maintain, and use knowledge bases without need for specialized training DTO Novel Intelligence for Massive Data (NIMD) Avoid strategic surprise by helping analysts be more effective (focus attention on critical information and help analyze/prune/refine/explain/reuse/…) DTO IKRIS – Interoperable knowledge representation for intelligence apps NSF & NASA Scientific Data Integration (NSF Virtual Observatories (VSTO), NSF GEON, NASA SESDI, NASA SKIF, …) NSF Cybertrust Transparent Accountable Data Mining (TAMI) Govt Classified applications that must defend their conclusions Deborah L. McGuinness

32. Files/WWW Toolkit Trust computation IWTrust Semantic Discovery Service (DAML/SNRC) OWL-S/BPEL Proof Markup Language (PML) End-user friendly visualization IW Explainer/ Abstractor CWM (NSF TAMI) N3 Expert friendly Visualization Trust KIF JTP (DAML/NIMD) IWBrowser search engine based publishing Justification SPARK (DARPA CALO) SPARK-L IWSearch Provenance provenance registration UIMA (DTO NIMD Exp Aggregation) Text Analytics IWBase Inference Web Infrastructure primary collaborators Pinheiro da Silva, Ding, Chang, Fikes, Glass, Zeng • Framework for explaining question answering tasks by • abstracting, storing, exchanging, • combining, annotating, filtering, segmenting, • comparing, and rendering proofs and proof fragments • provided by question answerers. Deborah L. McGuinness

33. How PML Works isQueryFor IWBase Query foo:query1 <formal internal structured query> Question foo:question1 <Input language question> hasAnswer Justification Trace hasLanguage NodeSet foo:ns1 (hasConclusion …) Language hasInferencEngine fromQuery isConsequentOf InferenceEngine InferenceStep hasRule InferenceRule hasAntecendent Source NodeSet foo:ns2 (hasConclusion …) … hasVariableMapping Mapping isConsequentOf fromAnswer hasSourceUsage hasSource SourceUsage InferenceStep usageTime … Deborah L. McGuinness

34. PML in Swoop Deborah L. McGuinness

35. Example Usage • DARPA’s PAL program – explaining cognitive assistant suggestions. • Video Deborah L. McGuinness

36. Explainer Strategy (for cognitive assistants) Present • Query • Answer • Abstraction of justification (using PML encodings) • Provide access to meta information • Suggests drill down options (also provides feedback options) Deborah L. McGuinness

37. Task State Database Architecture for Explaining Task Processing TaskLearner1 Task Manager (TM) TaskLearner2 TaskLearner3 Collaboration Agent Explanation Dispatcher TM Explainer TM Wrapper Justification Generator Deborah L. McGuinness

38. Task Explanation Ability to ask “why” at any point… Context appropriate follow-up questions are presented Deborah L. McGuinness

39. IWBrowser - Browse & Debug (TAMI) Deborah L. McGuinness

40. Multiple Interfaces… Browsing Proofs Deborah L. McGuinness

41. Browsing & Debugging Deborah L. McGuinness

42. Example Abstraction (using tactics) Deborah L. McGuinness

43. Intelligence Tool Explanation(similar to other applications that reason with statements that may not be 100% correct) Deborah L. McGuinness

44. Follow-up : Metadata Deborah L. McGuinness

45. Follow-up: Assumptions Deborah L. McGuinness

46. Explaining Extracted Entities (Techies) Sentences in English Sentences in annotated English Sentences in logical format, i.e., KIF Deborah L. McGuinness

47. Trustworthiness of Extracted Entities A trustworthy conclusion from IBM STAG KDD-model Annotator A highlytrustworthy conclusion from IBM EAnnotator The combined conclusion is highly trustworthy Deborah L. McGuinness

48. Estimated trustworthiness of the IBM extraction and integration components IBM Cross-Annotator Coreference Resolver 0.82 IBM Cross-Document Coreference Resolver 0.63 IBM EAnnotator 0.91 IBM GlossOnt 0.33 IBM JResporator 0.31 IBM KANI holdsDuring Relation Detector 0.20 IBM Knowledge Integrator 0.88 IBM Knowledge Structures Group's Relation Detector 0.94 IBM Statistical Text Analytics Group's ACE-model Annotator 0.80 IBM Statistical Text Analytics Group's KDD-model Annotator 0.73 IBM TAF/Talent plus a collection of miscellaneous TFST grammars 0.78 IBM Talent time annotator 0.83 Deborah L. McGuinness

49. Trustworthiness of Ramazi report From FBI From intercepts From CIA Deborah L. McGuinness

50. Trustworthiness of revised Ramazi report This text fragment changed from “Neutral” to “Trustworthy” after it was revised by an analyst (the phone number was corrected). Deborah L. McGuinness