Registration and Harvest IIB Presentation May 1 , 2014

1. Registration and HarvestIIB PresentationMay 1, 2014

2. GEOSS and Registration • The ‘pledge’ of data or services to GEO requires the act of registration • Registration collects basic information about the resource and its context (e.g. related SBAs, availability, extent) called “metadata” • Data or system owners can register metadata with GEOSS: • If you have never described your dataset or service with metadata, use the form at geossregistries.info • If you have a catalog of metadata that is not registered with data.gov, register the catalog with geossregistries.info

3. You may be registered already • You don’t need to do anything if: • You have created metadata for your dataset or service in an agency, national, or professional catalog (i.e. INSPIRE, data.gov.uk) and it is already registered with GEOSS • The data or service resource is already described with a current DIF or SERF in the CEOS IDN • But… • Be sure to include any search tags (i.e. geossdatacore, EO Vocabulary) in your metadata, however, and provide a context for any URLs in the metadata as to format or protocol

4. Proposed Changes • General concern about the complexity of registration and the fact that items can be registered either in CSR or the DAB • DAB functionality now includes support for harvest of remote collections as well as distributed search • GEOSS Clearinghouse function can be deprecated • A single registration facility is required to pledge (self-descriptive) metadata, catalogs, or services for processing/integration with the DAB – along with a ‘last resort’ catalog to collect full metadata by form

5. Process • Consolidated Requirements for the GCI were prepared in advance of selection of a GEO Web Portal. Included: • Common requirements across components to enable interoperability • Component-specific requirements • Executable tests to verify items that could be tested for assessment • UML diagrams were developed to describe the use cases, required information classes, and interactions between components and beyond to access of pledged resources • DAB is just now being included in the architecture of GCI, along with requirements, and its interaction with the rest

6. Data Publishing in GEOSS 4 Users (browser) Client Software CSW, opensearch geoportal.org GEOSS DAB* *GEOSS Discovery and Access Broker GEOSS Portal 3 Standards Registry (SIR) 5 also registered Component and Service Registry National or Community Metadata Catalogs Links to access data share type codes geossregistries.info form 2 Options Data.gov Agency catalog • Create metadata • Save in catalog • Indexed by DAB • Search via Portal • Link via metadata IDN URLs to data, maps, web services (HTTP) 1 Data Systems Metadata Editor / Manager Structured Metadata Agency Metadata and Data Services

7. GEOSS Current Architecture Access GEOSS User GEO Web Portal GEOSS Common Infrastructure • GEOSS Registries: • Standards & Interoperability • User Requirements • Best Practices Wiki • Semantic Monitoring Services Discovery and Access Broker* (DAB) Clearinghouse (Catalog) Centralized EO Inventory Clients GENESI*, CWIC*, FedEO* harvest Component & Service Registry (CSR) Register real-time search metadata links to Resource Providers Member and PO Resources Agency Services and SW Applications: Web services, Applications. Community Portals Official Catalogs

8. GEOSS Proposed Architecture Access GEOSS User GEO Web Portal GEOSS Common Infrastructure • GEOSS Registries: • Standards & Interoperability • User Requirements • Best Practices Wiki • Semantic Monitoring Services Discovery and Access Broker* (DAB) Centralized EO Inventory Clients GENESI*, CWIC*, FedEO* Users harvest or search Collections Register real-time search metadata links to Data Resource Providers access Member and POResources data Create and manage Metadata Official Catalogs Agency Assets

9. Contacts and Systems Registration • Contact management is required, capturing the affiliation of individuals to their immediate organizations and also with GEO Members and Participating Organizations • GEO DAB requires a minimum of configuration information to effect search and access functions on pledged Discovery Assets – metadata, catalogs, and self-descriptive services • New focus on registering and configuring these Discovery Assets that provide access to structured metadata as a means to find data, services, applications, models, documents, websites, etc. • This fundamentally simplifies the function and scope, like a “Yellow Pages”

10. Contact information • Originally envisioned in the architecture was a mechanism to capture and manage GEO Member and Participating Organization (M&PO) information • Suggest the ability to also manage individual user information and associations with GEO M&PO (Could we use OpenID to manage email-based identities?) • Users in this system would then login to register Discovery Assets and tracking of pledged resources by organization - to be maintained for internal reporting • Do not want to manage significant Personally Identifiable Information (PII) that is not already managed elsewhere • Could still allow non-affiliated registrations…

11. Discovery Assets • Discovery Assets that the DAB can configure include URLs to: • Individual metadata files • Collections or series of metadata files • Metadata catalog services (CSW, opensearch, OAI-PMH) • Service endpoints (OGC WxS, OPeNDAP, RESTful) • Potentially also websites/pages that are self-descriptive using microcodes or Schema.org tags • Entries hosted in the Contributor’s Catalog

12. Discovery Asset Registration • Only a few fields of information need to be collected: • URL to the resource • Description of the resource • Protocol used (HTTP, REST, OGC WxS, OPeNDAP, FTP) • Format of the resource (file type: i.e. XML, json, KML, GeoRSS, netCDF, etc.), directory • Schema reference of resource, if applicable • Contact email • (Organization link, GEO MP link, date-time stamp possible) • These will allow for configuration of harvest/access in DAB • Reference to standard protocol and format standards should be drawn from a published codelist in the SIR (OSGeo work)

13. Contributor’s Catalog • Currently in CSR there is the ability to form-enter metadata when there is no community, national, or organizational catalog otherwise available • Suggest the establishment of a form-based ISO metadata editor and a catalog for the GCI to host these records and also the registration of Discovery Assets • This would also enable metadata management for few, individual records by end-user publishers • This would need to be added to the architecture (green bubble) diagram and UML models. • This would be realized through a single form/catalog to hold full records and Discovery Assets

15. Roles and responsibilities • Articulate DAB functions in the Consolidated Requirements • New core registration and catalog function proposed to be designed jointly between USGS, GMU, and CNR with review and collaboration with the GCI Providers (IN-03) • Propose to eventually deploy at the GEOSec for them to administer GEO M&PO enrollment and host the Contributor’s Catalog • Registrations will trigger notifications to the DAB operators • Need a mechanism to track registrations, their configuration within the DAB, and notification to publishers • May need to create or use an external OpenID Provider (like Google or Facebook Connect) to support single sign-on

16. Proposed GEOSS Workflow Register user Relative to GEO M&PO, agencies, use email address as key identifier Create description of online data, service, or other resource using formal but simple metadata Describe Contribute descriptions to a Community Catalog in a country, COI, or international (i.e. CEOS IDN) – or Contributor’s Catalog for DAB harvest Publish Discover Search for information via Portal UI or catalog API (opensearch or CSW) Access Access information through described APIs / URLs on the resource

17. Issues • GEOSS Portal and DAB developments have been done outside the existing Task framework, stimulated by GEOSec exigencies • DAB has not been introduced and integrated with the existing GCI architecture • Roles and repsonsibilities for NextGen capabilities have not been clarified

18. Next Steps • Use the GCI Providers calls and membership of IN-03 to document the GCI architecture to include the DAB and nextGen registry • Simplify and approve an updated Consolidated Requirements doc and reflect on the architecture and relation to overall architecture • Deploy updated GCI Components to streamline registration (nomination to DAB) and collect metadata as a catalog of last resort • Develop training materials to promote propser use of GEOSS (let’s get beyond even mentioning GCI)