Interoperable Data Formats for Production Systems

1. Interoperable Data Formats in Production Systems Dave Denault & Brian ScarlettIICWG October 2005

2. Presentation Content • Defining interoperability • The advantages of interoperability • Achieving interoperability; • Interoperability with sameness • Interoperability with interfaces • The need for standards • Functional requirements; • Core interchange technologies • Open Web mapping standards (WMS)

3. Defining Interoperability • The International Organization for Standardization (ISO) defines interoperability as; “The ability to communicate, transfer and process data among functional units in a manner that requires the user to have little or no knowledge of the unique characteristics of these units.”

4. Defining Interoperability • But we will use the less formal definition of; “Enable the largest possible audience to access and integrate the data with the least amount of effort and time.”

5. Reasons for Interoperability • To simplify the dissemination of data and services to partners and clients. • To facilitate improved data usability in addition to basic accessibility. • To promote collaboration both within and external to the organization by; • Accelerating the research process • Encouraging knowledge discovery • Keeping the way clear for “future questions and approaches”

6. Reasons for Interoperability • Get the data out there and make it simple to locate for everyone. • Make the data as easy to work with as possible – minimize data handling. • Document the data sufficiently such that others can work with the data in new and unanticipated ways. In other words;

7. Interoperability via Sameness • A simple solution is to have all data providers work within the same technical environment. • This solution typically requires; • Use the same commercial software • Use the same applications • Use the same operating systems • Use the same database designs • Use the same internal data formats, etc

8. Interoperability via Sameness • Many organizations have demonstrated that “sameness” can be impractical… • More time-consuming to implement. • Can be difficult to maintain over time. • Slower to adapt as business evolves. • Some business requirements are unique and support by all data providers can become a burden.

9. Achieving Interoperability • An alternative solution is to support heterogeneous technical environments which communicate via “interfaces”. • Interfaces determine what can pass in/out of an environment. • What does this require? • Standards • Standards • And more standards • These standards must be well-defined and complete and universal.

10. Interoperability and the Web • The World Wide Web (W3 or Web) is a good example of how standards can work. • The Web is supported by the WWW Consortium (W3C) which has the mandate; • “To develop interoperable technologies to lead the Web to its full potential.” • Prior to 1994, incompatible HTML versions created problems when using Web pages. • W3C promoted a set of core principles and components to be supported by everyone. • W3C helped to create the current Web.

11. Interoperability for IICWG • Basic functions required in an interoperable environment include • #1: Core interchange technologies that define machine-readable and machine-interpretable data. • #2: The deployment of interactive geospatial data using Internet accessible and open standards.

12. Interoperability for IICWG • #1: Core Interchange Format • Used between data providers. • Would also support external partners such as researchers and modellers. • With full metadata, the data is self-describing (e.g. MANICE attributes) • Replaces the multiple exchange formats used now (e.g. coverages, shapefiles, ASCII files, etc)

13. Interoperability for IICWG • #2: Standards for Interactive Geospatial Data; • Used primarily for sharing data with clients and the public. • Clients do not need to have expensive software to use the data (Web browser is possible) • Could also be used for sharing data with partners since most GIS software can consume these services too.

14. Core Interchange Format • Without a core interchange format, data conversions are often required. • But reducing the amount of this manipulation is desirable since; • Data conversion is costly to implement and maintain. • Dataset-specific software is not always available to all users. • Data quality can be compromised during conversion (losses, alterations, misinterpretations)

15. Interchange Formats • SIGRID-3 is an example of an existing interchange format. • SIGRID-3 is a specification for; • Vector data objects (polygons) • The attributes of data objects • Metadata for the objects • Other emerging standards include the Geographic Markup Language (GML) • GML uses XML to express geospatial vector features and their attributes.

16. Interchange Formats (NAIS) • Although intended for archival purposes, SIGRID-3 is being considered as the interchange format for NAIS. • Some extensions to the SIGRID-3 specification are being proposed to fulfill this role within NAIS. Interchange Format (e.g. SIGRID-3) Canadian Ice Service Internal system (e.g. ISIS) National Ice Center Internal system (e.g. SIPAS)

17. Interactive Geospatial Data • Standards already exist to provide simplified access to distributed geospatial datasets. • These standards are actively supported by the Open Geospatial Consortium (OGC) • 280+ commercial, government and research organizations are involved. • The OGC encourages development of standards for geospatial content and processing and exchange.

18. OGC Web Map Service (WMS) • Compliance with OGC standards enables users to exchange and apply information directly across; • Different platforms (e.g. UNIX, Microsoft Windows, Linux, Mac OSX) • Different applications • A key aspect of the OGC standards is that they are “open standards”. • Open = non-proprietary with free distribution (i.e. no royalties or fees)

19. OGC Web Map Service (WMS) • The Web Map Service (WMS) uses HTTP, the basic protocol of the of the Web, to issue requests. • The response is a conventional pictorial format such as JPEG or PNG, allowing standard Web Browsers to function as client applications. • Most commercial GIS software and many free applications/toolkits also support WMS (or intend to)

20. OGC Web Map Service (WMS) • Currently supporting WMS; • ESRI ArcGIS Desktop (e.g. ArcView, ArcInfo, ArcExplorer, ArcGlobe) • ESRI ArcGIS Server-side (e.g. ArcIMS) • Autodesk MapGuide • Intergraph GeoMedia WebMap • The GeoServer Project • IONIC Red Spider Web • Mapinfo MapXtreme • Oracle MapViewer 10g • Refractions uDig viewer (GeoInnovations) As of October 2005: 61 products fully compliant 240+ products being tested

21. OGC Web Map Service (WMS) Client/Partner/Public Web Browser (e.g. Firefox, IE,) Data Provider: Internal GIS systems (e.g. SIPAS, ISIS) Web Server compliant with OGC WMS No plug-ins required Geospatial Data Stores (e.g. Geodatabase, Shapefiles, …) External GIS systems (e.g. ArcGIS) and Viewers (e.g. uDig) Local layers + WMS layers

22. OGC Web Map Service (WMS) Access to Geospatial Data Indirect Access to Geospatial Data Client or Partner or Public Combination Access to Geospatial Data Web Server (WMS) Web Server (WMS) Web Server (WMS) GeoSpatial Data Geospatial Data Geospatial Data

23. Summary • Interoperability encourages the distribution and access of data with a minimal amount of user effort. • The basis for interoperability is standards, as shown by the Web. • Interchange formats would support system-to-system flows of data. • Web mapping standards (e.g. OGC WMS) would support system-to-user flows of data.

24. www.opengeospatial.org www.esri.com/software/standards www.refractions.net Interoperable Data Formats in Production Systems Thank you

25. Interoperability Presentation SIGRID-3 Shapefiles From Canadian Ice Service External GIS Viewer Refractions uDig Web Server compliant with OGC WMS The GeoServer Project External GIS systems ESRI ArcGIS 9.1 Local Data: Shapefiles Coverages Imagery External Web Server compliant with OGC WMS