SICoP Presentation A story about communication

1. SICoP PresentationA story about communication Michael Lang BEA Revelytix May 2, 2007

2. Interoperability • This track concerns making an SOA as managable and interoperable as is possible • Description • Abstraction • Governance • Communication • Community based Communication

3. Agenda • Communication in Indonesia • Schemas and Languages • Semantic Technology • ETL and data warehousing • Roadmap • Transitioning from here to there • Vocabulary Management • Semantic Data Services

4. Indonesia

5. Indonesia in 1960 • Indonesia is comprised of several thousand islands • 340 different languages/dialects are spoken • Indonesia is not an integrated country • The President decided that Indonesia needed a common vocabulary so that the groups could communicate with each other

6. Indonesia Today • The language/vocabulary was developed and is now in widespread use • Indonesia is interoperable today

7. IT Systems • Each system is an island and it has its own language • We call these languages schemas • If we want to be able to speak among systems we need a common vocabulary • Key concept #1 • Language = Vocabulary = Schema • The language is used by a community

8. Languages • Features • A way to represent concepts and the relationships between concepts • A way of representing facts about the world • A way of adding concepts, relationships and facts without breaking the language • A syntax or grammar that shows how the language is structured and functions

9. Why are there Data Warehouses • The Sales community uses the schema of the CRM system to capture facts and concepts • Manufacturing uses the schema of the ERP system • Finance wants to forecast trends • Invent a new language that has some of the concepts and facts of both systems as well as additional concepts • Build a new system

10. XSD as a Language • XSD is a great way to define the terms and syntax for a community • And it is a language, and it can be extended • But, systems processing XML documents containing the data are expecting the structure to be fixed according to an XSD • If the structure is different the system must be changed • Address • Street • City • Zip code • Address • Street • Apt number • City • Zip • Address • Street1 • Street2 • City • Postal Code

11. XSD as a Language • The purpose of XSD is to specify syntax • The representation of the data is determined by the schema • IT systems that process XML count on this • XSD has serious limitations as the technology for achieving interoperability • Every time the Language is extended the systems must be recoded • New communities cannot enter into the vocabulary • adding their own concepts • Or extending existing concepts

12. RDFOWLSPARQL Semantic Technology

13. Semantic Technology • The first fundamental change in Information Management since the RDBMS was developed in the early 1980’s • Key concept #2 • The “data” exists independently of any schema • data = instances = facts

14. RDF RDF is uniquely positioned as a machine readable lingua franca for representing relational, XML, proprietary, and most other data formats • BUT, it is not a language • There is no schema • It is a way of making data accessible to ANY schema defined using OWL or RDFS

15. RDF • RDF uses a graph structure • Can more accurately represent the entities being modeled • Can represent objects with complex structures directly without exposing implementation techniques, e.g. hierarchical structures • Data schemas do not need to be determined a priori • New data from disparate sources can be integrated seamlessly on the fly by merging two (or more) RDF graphs • RDF models data’s content and meaning, rather than just its structure or serialization

16. RDF is Schemaless • The schema can be defined using RDFS or OWL • Since the data is independent of the schema any language can be developed and deployed • Seamlessly incorporate new data sources, regardless of schema • Business logic and user-interface logic far more resilient in the face of unexpected data and schema evolution

17. Creating Languages with OWL Structuring RDF with Ontologies • The Web Ontology Language (OWL) allows the declaration of rules to define and describe RDF vocabularies. • OWL can be used to define the language for any community • OWL models can be extended without breaking the system that is consuming the data

18. SPARQL SPARQL: Web-aware semantic query • SPARQL is a SQL-like language for querying distributed RDF graphs • SPARQL queries emphasize: • Distributed data. Multiple data sources can be queried at once because SPARQL addresses graphs by URI. • Ragged data. The SPARQL OPTIONAL keyword lets users explore heterogeneous data in a single query. • Unpredictable data. The ability to query for predicates and information about predicates makes SPARQL ideal for exploring new and unexpected data.

19. Myth: None of this Works • Not true • Many tools are capable of producing RDF • Some are good at representing relational data as RDF • There are several good OWL modeling tools available • Jena and Sesame are open source reasoner/SPARQL engines • Many commercial products are incorporating support for semantic technology now

20. Roadmap

21. How to get from here to there • Here • Currently SOA’s exchange data as XML where the document structure is determined by an XSD and the query language is XQuery • There • SOA’s exchange data as RDF and query using SPARQL

22. Web Services Web Services Semantic Alignment of Data Services You can haveone or moreof these Mapping Vocabularies A & B Mapping Vocabulary “same as” or “same class as” Vocabulary A Vocabulary B generate describe the RDF generate describe the RDF XML Messages (in RDF XML) XSD XSD Describe the structure (elements & attributes) Describe the structure (elements & attributes) XML Messages (in RDF XML) reference reference WSDL WSDL describe describe

23. combine extract extract extract Web Services XML Messages (in RDF XML) RDF Content RDF Contentfrom allResponses RDF Content combine combine RDF Content Using Service Responses KNOWN FACTS

24. Context Based Integration “Semantic Interpreter” or “Semantic Message Translator” Small wrapper around Jena Vocabularies (OWL) Composed at design-time submit produce QUERY (SPARQL) KNOWN FACTS NEXT SERVICE REQUEST MESSAGE Designed to obtaindesired messagefor next service call Composed from previousmessages in a SOAtransaction plus assertions(facts) obtained fromother sources

25. Transition Plan • Start developing Community of Interest based Vocabularies as OWL models immediately • Generate XSD’s to facilitate the use of existing infrastructure directly from the OWL models • Begin a program of representing service based information as RDF • Build applications that access the RDF representation using the previously developed OWL models through SPARQL/reasoner engines

26. Conclusions • Semantic Technology is the only natural integration technology available • It is game changing in delivering capability • The road ahead is apparent if not quite obvious yet • Commercial tools are available and more are coming soon • It is time to get started

27. Vocabulary Management

28. BPM Customer Portal Partner App Employee Portal EAI / ESB ? DB DB DB XML XML Structured and Unstructured Data Designing Semantic Data Services

29. BPM Customer Portal Partner App Employee Portal EAI / ESB Semantic Data Services DB DB DB XML XML Designing Semantic Data Services Structured and Unstructured Data

30. Vocabulary Management • Step 1: Extract semantics from existing data • Import, tokenize, assign definitions, …

31. DB DB DB XML XML Vocabulary Management Step 1: Extract semantics from existing data

32. Vocabulary Management • Step 1: Extract semantics from existing data • Import, tokenize, assign definitions, … • Step 2: Create bootstrapped vocabulary • Discover terms, concepts, duplications, similarities, …

33. OWL OWL OWL Vocabulary Management Step 2: Create bootstrapped vocabulary

34. Vocabulary Management • Step 1: Extract semantics from existing data • Import, tokenize, assign definitions, … • Step 2: Create bootstrapped vocabulary • Discover terms, concepts, duplications, similarities, … • Step 3: Evolve vocabulary collaboratively • Make meaningful for people using natural language • Use formalisms for machines via OWL • Web-based collaboration and sharing

35. Vocabulary Management Step 3: Evolve vocabulary collaboratively

36. Vocabulary Management • Step 1: Extract semantics from existing data • Import, tokenize, assign definitions, … • Step 2: Create bootstrapped vocabulary • Discover terms, concepts, duplications, similarities, … • Step 3: Evolve vocabulary collaboratively • Make meaningful for people using natural language • Use formalisms for machines via OWL • Web-based collaboration and sharing • Step 4: Use vocabulary • Used by people and tools

37. Vocabulary Management Step 4: Use vocabulary to understand

38. BPM Customer Portal Partner App Employee Portal EAI / ESB OWL ? ? ? ? DB DB DB XML XML Vocabulary Management Step 4: Use vocabulary to design services Semantic Data Services Structured and Unstructured Data

39. Vocabulary Management Step 4: Use vocabulary to design services

40. Designing Semantic Data Services Vocabulary Management: • Bootstrap vocabularies with existing data sources • People collaborate on creation of vocabularies • People can understand vocabularies • Tools can use vocabularies (OWL artifacts) to produce semantic data services Next: • Deploy and use semantic data services

41. BPM Customer Portal Partner App Employee Portal EAI / ESB Semantic Data Services Knoodl Mapping with MatchIT BEA AquaLogic Data Services Platform (ALDSP) DB DB DB XML XML Designing Semantic Data Services BEA AquaLogic Workshop OWL Import Structured and Unstructured Data

42. SICoP Knowledge Reference Model The point of this graph is that Increasing Metadata (from glossaries to ontologies) is highly correlated with Increasing Search Capability (from discovery to reasoning).