Metadata & Interoperability

1. Metadata & Interoperability ENVE/CE 424/524

2. Metadata • They describe the entire contents of a data set • metadata are data about data • the documentation and handling instructions for data • Metadata are what make data useful • without documentation and handling instructions data would have no value to a user • without metadata it would be impossible to find data in a library or on the WWW

3. What does metadata tell you? "Using a standard to create metadata enables the GIS community to speak in like terms. It also sets the basis for software development by standardizing items and item descriptions. This has enabled the development of cataloging and indexing tools as well as search engines which can pose queries against collections of metadata."

4. Why use metadata? • We all create metadata informally, as notes or reports • We all use data that we didn't create • When you're trying to use someone else's data, which is easier to interpret, informal notes, a voluminous report, or a formal standard? • Use the history of word-processing as an example. It used to be that you never knew what software would be on any given computer... Wordstar, WriteNow, WordPerfect, Word... Maybe you got familiar with using your favorite word processor, but if you had to use another person's machine, there'd be no guarantee that the software you were familiar with was on it. Gradually, word-processing software evolved to the point that different word-processing programs applied the same standards for creating and formatting, and furthermore most programs can automatically read and open a document created in a different format. Establishing standards for word-processing has made it possible to access and use documents regardless of specific formats, programs, or platforms. • Like any powerful tool, the metadata standard requires some time and effort to learn, but it is a worthwhile investment in the long run.

5. What does metadata allow you to do? • Automate the process of searching for data • compare using a library catalog • Determine the fitness of a data set for use • particularly regarding quality • Handle data effectively • e.g., format • Identify notable data contents • e.g., to find images of an interesting hurricane

6. Metadata can be Expensive to Generate • They represent a high level of abstraction • and may need an expert to define • How do you reach consensus? • I say road you say street. • But the benefits are substantial • metadata make it possible to find data sets, and use them effectively • they allow the benefits of investments in data to be realized

7. The U.S. FGDC Standard • To be useful, metadata must adhere to widely accepted standards • Most widely used standard is the US Federal Geographic Data Committee’s Content Standards for Digital Geospatial Metadata (CSDGM) • Defined by a committee of U.S. Federal agencies • Now widely used worldwide • The basis of a new international standard • Potentially several hundred items for one data set • but easily boiled down to a much smaller number • Defines the items that should be included in a data description but not how they should be formatted or structured

8. Federal Geographic Data Standard

9. FGDC Metadata Example

11. The Dublin Core Standard • Devised by the digital library community • Suitable for any type of data, geospatial included • easily extended to include essential items for geospatial data • e.g., the latitude and longitude limits of the data set’s coverage • The idea is to establish a minimum set of metadata elements that define the fundamental properties of a dataset.

12. Elements of Dublin Core Title Author or Creator Subject or Keywords Description Publisher Other Contributors Date Resource Type Format Resource Identifier Source Language Relation Coverage Rights Management

13. International Organization for Standardization (ISO) • TC 211 - Standardization in the field of digital geographic information. • Aims to establish a structured set of standards for information concerning objects or phenomena that are directly or indirectly associated with a location relative to the Earth. • These standards may specify, for geographic information, methods, tools and services for data management (including definition and description), acquiring, processing, analyzing, accessing, presenting and transferring such data in digital/electronic form between different users, systems and locations. • The work shall link to appropriate standards for information technology and data where possible, and provide a framework for the development of sector-specific applications using geographic data. • ISO and FGDC work together to homogenize their standards.

14. eXtensible Markup Language (XML) The use of XML provides a reference framework for encoding of nested data structures and a means to test them. Unlike HTML, which mixes content tags such as <title> and <body> with presentation tags such as bold <b> and italics <i>, XML provides strictly for the structure of information entities using beginning and ending tags. Tags can be user defined User communities define their standard tags and apply them to the structure of metadata standards.

15. ArcGIS Metadata

16. Geolibraries • Repositories of data that can be searched for data covering geographic areas of interest • this was very difficult in a conventional library using a card catalog • each data set in a geolibrary is identified with a geographical footprint • in a search, footprints are matched to the area of interest defined by the user

17. The Alexandria Digital Library The user picks an area of interest by interacting with a map, specifying latitude and longitude limits, or giving a place name. The library returns all data sets whose footprints match the query area, and which match other criteria also supplied by the user.

18. National Spatial Data Infrastructure (NSDI) • ‘the technology, policies, standards, and human resources necessary to acquire, process, store, distribute, and improve utilization of geospatial data’ Source: Presidential Executive Order #12906 (1994): 'Co-ordinating Geographic Data Acquisition and Access: The National Spatial Data Infrastructure' W. Clinton. The vision of the NSDI: to assure that spatial data from multiple sources (Federal, State, and local governments, academia, and the private sector) are widely available and easily integrated to enhance knowledge and understanding of our physical and cultural world. • Establish standards • Improve access to and sharing of data • Develop national digital geospatial data framework

19. National partnerships via NSDIs • The problem: • Data duplication commonplace – so waste occurs • Ad hoc data sharing has many difficulties • Data often tailored to one application • Best data often collected in greatest detail at local level but not accessible to regional or national folk • Indexes/metadata to available geospatial data unknown until recently • No general protocols for any of this until NSDI. The solution involves: Federal government (many agencies) State government Local government Private sector – contractors, value-adders, exploiters Not for profit organizations Public

20. A global partnership – the need ‘In any area of the world, a number of core data types are required to assess environmental conditions and develop strategies that can lead to long term sustainable human development. For many areas of the world, these data do not exist at the scales required to support these activities. Development of such data sets is labor-intensive and costly and, although these data sets could support a wide variety of applications, no single use can generally justify the full cost of development.’ Htun (1997)UN Regional Director for Asia

21. Governments and the private sectors • National governments own and control national mapping agencies • All such mapping produced to national specifications until recently • New private sector providers: • Produce imagery for anywhere in world • Produce road databases • How do we get these to work together?

22. A Global Spatial Data Infrastructure? • Difficult enough to get national players to work together… • Is GSDI a process, a general framework or a product? • Who are the stakeholders? • Who needs it? (military doing what they need themselves?

23. The Global Mapping Project http://www.iscgm.org/html4/index.html

24. Interoperability Interoperability has a range of meanings but all focus on the ability to move easily from one system to another. Parallels the push for open systems - the intention to adhere to vendor-neutral computing standards • What is required: • Data producers must ensure that their data are readily accessible and understandable • User must be able to identify and locate relevant information • Queries to dispersed sites must be formulated in a manner meaningful on both data user and provider ends • Geodata from one source must be capable of being integrated with data from another • Display and analytical functions must available

25. The Open GIS Consortium (OGC) • The Open GIS Consortium (OGC) is a not-for-profit, international consortium whose 250+ industry, government, and university members work to make geographic information an integral part of information systems of all kinds. • Operates a Specification Development Program that is similar to other Industry consortia (W3C, ISO, etc.). • Also operates an Interoperability Program (IP), a global, innovative, partnership-driven, hands-on engineering and testing program designed to deliver proven specifications into the Specification Development Program. OGC Vision A world in which everyone benefits fromgeographic information and services made available across any network, application, or platform. OGC Mission To deliverspatial interface specificationsthat are openly available for global use.

26. Geospatial One-Stop Geospatial One-Stop makes it easier, faster, and less expensive for all levels of government and the public to access geospatial information. The Geospatial One-stop is one of 24 e-government initiatives sponsored by the Federal Office of Management and Budget to enhance government efficiency and improve citizen services. • E-Government (E-gov) Initiative • Component of President Bush’s Management Agenda • One of 24 E-Government initiatives • Part of the Government to Government Portfolio • Vision: the geographic component for e-gov • Accelerate implementation of the National Spatial Data Infrastructure (NSDI)

27. Establish integrated Framework Data Content Standards Use metadata to inventory, document and publish Framework data holding in NSDI Clearinghouse Publish metadata for planned data acquisition and update for Framework data Prototype and deploy enhanced data access and web mapping services for Federal Framework data Establish a Federal Portal as a logical extension to the NSDI Clearinghouse Network GeoSpatial One Stop Tasks

28. customer middleware common packaging services Providers of Theme X • automated ad-hoc requests • data and mapping via web services • web discovery, portrayal, and order or access • support of common content models and extended models • support of neutral encoding methods • data packages can be validated against a common reference • data translation shared between provider and customer Data services Past and Future customer Providers of Theme X • human request handling • ftp data access • web discovery and order • documentation of local content models • use of proprietary formats • little adherence to enforceable standards • data translation largely the responsibility of the customer

29. Portal is an online access point for geospatial data, maps of data, and metadata about data Distributed Information: data stored at provider sites, not in Portal Portal uses open standards for interoperability Geospatial web service specifications Metadata and content standards Portal builds upon NSDI Clearinghouse by providing direct access to data Web-based user interface for human access Application interface for automated GIS/DSS access Geospatial One-Stop Portal Concept

30. Future Distributed Information System Federal Users Civilian Users National Policy Tribal Users Makers Commercial Users State Users Local Users DoD Users International Users replicated access points NSDI One-Stop Portal State Governments Other Federal Agency Local Governments Commercial Sector Services Data Bases The National Map DoD Tribal Governments

31. Article Review • Give full citation of the article (also URL if online) If not online, provide a copy of the article. • Include an article summary - a long-worded rewrite is not necessary. Make sure you hit the main points. What was the take-home point of the article? • Compare the article to analysis methods presented in class. Did the authors deviate or use a more advanced method. What were their assumptions? Why did they or didn’t they use “standard” methods discussed in class and lab? • 4. State what you learned from the article/chapter. How did this help you with your project? With understanding concepts covered in lab or lecture?