1 / 32

Technologies for distributed systems

Technologies for distributed systems. Andrew Jones School of Computer Science Cardiff University. Need to bring data together. To achieve breadth (e.g. coverage of more organisms) To achieve depth (e.g. more complete data on individual species). Merging.

xena
Download Presentation

Technologies for distributed systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Technologies for distributed systems Andrew Jones School of Computer Science Cardiff University

  2. Need to bring data together • To achieve breadth(e.g. coverage of more organisms) • To achieve depth(e.g. more complete data on individual species)

  3. Merging 1. The original databases are physically copied into a new combined database. 2. The user interacts with the new combined database.

  4. Linking 1. Original databases remain separate, but accessed via a single system such as a portal 2.The user interacts with an access system which does not itself contain data. When the user requests data, it is fetched from the appropriate database.

  5. Basic problems to solve How to deal with data that is: • On various database management systems • Distributed across various machines • Distributed across various machines of various types • Based on various schemata (i.e. not all data expressed in the same form) Also, how to resolve data quality problems: • taxonomists vary in their opinions • large taxonomic treatments are generally inconsistent • individual databases generally have mistakes • (So we need tools to help biologists detect andresolve such problems, such as LITCHI – nottoday’s topic!)

  6. Essential elements of solution • Ways of setting up communication between components • Ways of expressing data suitably for it to be communicated between components • Ways of describing and finding components such as data sources

  7. Setting up communication Possibilities include: • CGI (Common Gateway Interface) – style HTTP requests(A standard for communicating requests to Web servers) • Z39.50(A standard for digital libraries) • Web Services • DiGIR

  8. CGI-style HTTP requests • Simple way of passing parameters • in one variant (GET), parameters expressed as part of the URL, e.g. http://www.ildis.org/LegumeWeb?genus~Sabinea&species~florida (NB: POST preferred) • Result: an HTML page (see next slide)

  9. “CGI” approach: strengths & limitations • Easy to set up • Not good for complex data • HTML is basically aformatting language, for sayinghow documents should be displayed, not what they contain • But we can pass around XML too • E.g. SPICE • Also HTTP is the basis of SOAP (see later)

  10. SPICE • Species 2000 Interoperability Cooperation Environment • Allows choice between • HTTP GET/XML response (essentially the CGI approach, but retrieving XML) • CORBA • Uses wrappers to transform to common data model & SPICE protocols

  11. User(Web browser) User(Web browser) .......... CORBA User server module (HTTP) CAS (Common Access System) CAS knowledge repository ‘Query’ co-ordinator Wrapper (e.g. JDBC) Wrapper (e.g. CGI) .......... (In some cases, generic) CORBA ‘wrapping’ element of GSD wrapper GSD GSD SPICE architecture

  12. Z39.50 • A standard for digital libraries • (Most library systems are built around this standard) • For interoperability in client-server architectures • Standardised sets of attributes (items of data)

  13. Z39.50 strengths & limitations • Standard for digital libraries • Works well for certain widespread, agreed data standards (‘profiles’) • Very restrictive if you want to add on things like extra security • Useless in cases where a data standard doesn’t yet exist

  14. Web Services Service directory (e.g. UDDI) • Web services provide a simple way of making software available on the Internet. • All the communications in this diagram are SOAP messages Directory query Register service description (WSDL) Query responses (WSDL) Service Provider XML service request, based on WSDL Service Consumer XML service response, based on WSDL

  15. DiGIR • Proprietary approach, especially designed for specimen records • Uses Darwin Core data model • The following slide is the DiGIR team’s high-level architecture diagram ...

  16. XML (eXtensible Mark-up Language) • Flexible mark-up language • Like HTML, but tags describe the document’s contents, not how it’s to be displayed. • XML is the basis of SOAP: ‘language independent’, i.e. a good data interchange format.

  17. Simplified Species 2000 example <!DOCTYPE SPECIESLIST[ <!ELEMENT SPECIESLIST (SPECIES*)> <!ELEMENT SPECIES (GENUS, EPITHET, AUTHORITY?)> <!ELEMENT GENUS (#PCDATA)> <!ELEMENT EPITHET (#PCDATA)> <!ELEMENT AUTHORITY (#PCDATA)> ]> <SPECIESLIST> <SPECIES> <GENUS>Vicia</GENUS> <EPITHET>Faba</EPITHET> </SPECIES> <SPECIES> <GENUS>Sabinea</GENUS> <EPITHET>punicea</EPITHET> <AUTHORITY>Urban</AUTHORITY> </SPECIES> </SPECIESLIST>

  18. Dimensions of interoperability • System • Syntactic • Structural • Semantic

  19. Syntactic interoperability – some problems • Differences in machine-readable aspects of data representation (formatting), e.g.

  20. Syntactic interoperability – some solutions • Typically fairly easy to write converters between formats • “Wizards” (if we’re going to do data preparation first) • XSLT (transforming between XML documents holding same information in different formats)

  21. Structural interoperability – some problems • Representational heterogeneity that involves data modelling constructs • Schematic heterogeneity • For example …

  22. Structural heterogeneity example Database 1 Database 2

  23. Structural interoperability – some solutions • Database views • XSLT (to some extent) • Metadata & ontologies (associate terms in data sources with those in a shared vocabulary) • “Wrapping” to map between heterogeneous data sources and a shared representation (common data model)

  24. Semantic interoperability – some problems • Specimen distribution data example • Database A holds data for Vicia faba • Database B holds data for Faba faba • Descriptive data example • Database A: leaf length varies from 25.4 to 76.2 mm • Database B: average leaf length 2 in

  25. Semantic interoperability – some solutions • It’s not an entirely solved problem! • Useful general techniques: • Use of ontologies (defining relationships between terms, e.g. units) • Mapping functions • Attached metadata • … • Domain-specific techniques • “Synonymy server” • LITCHI (as an integration tool) • …

  26. Ontologies • Agreed terminology • Relationships between terms • Example use: integrator can associate terms in a source database schema with those in an agreed federation schema

  27. Summary • Interoperation among distributed resources is essential for ‘added value’ • Techniques exist for dealing with • communication between heterogeneous systems (e.g. Web Services; wrapping) • communication between systems with heterogeneous data (e.g. ontologies) • But not all the problems aresolved!

More Related