1 / 31

Data Integration Progress and Guiding Principles

Data Integration Progress and Guiding Principles. Disciplines, generalization, and open-access. David Blodgett – dblodgett@usgs.gov USGS Office of Water Information Center for Integrated Data Analytics. Outline. Data Integration Disambiguation Barriers to moving Forward.

kamana
Download Presentation

Data Integration Progress and Guiding Principles

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Integration Progress and Guiding Principles Disciplines, generalization, and open-access. David Blodgett – dblodgett@usgs.gov USGS Office of Water Information Center for Integrated Data Analytics

  2. Outline • Data Integration Disambiguation • Barriers to moving Forward. • Anecdotes, everyone loves anecdotes! • Principles to go Forward!

  3. Disclosures • I’m a water guy. • I‘m a millennial. • I assume Internet. • I’m a Badger. • … Forward!

  4. Data Integration – Disambiguated. Integration is the act of combining multiple things into a whole.

  5. Data Integration – Disambiguated. What makes something integrated? How different do things need to be to count? Do you just need to combine things?

  6. What kind of data integration is needed for decisions? Integrated Search Visual Integration Multi-source Data Ingest Jeff.deLaBeaujardiere@noaa.gov Application / Decision Driven Model of Data Integration Slide Credit: Jeff de La Beaujardiere Data Consolidation Data Bundling Data Fusion in the Cloud? Data Warehouse

  7. What kind of data integration is needed for decisions? Integrated Search Visual Integration Multi-source Data Ingest Jeff.deLaBeaujardiere@noaa.gov Data Consolidation Data Bundling Data Fusion in the Cloud? Data Warehouse

  8. A general model for data integration.

  9. Service Orientation On local machines, we run software. List, introspect, summarize, transform, integrate. Can scan the entire domain of the data! A service may do any or all of these things. Software on the server can summarize the domain and range of its holdings. (ie. Deliver Dynamic Metadata)

  10. Web Service – So what? Software on the server can summarize the domain and range of its holdings.

  11. Generalized Aspects of Data Services Various Communities’ Interchange International Standards. Discipline specific linked to other disciplines.

  12. Practical Barriers ‘I don’t know how to use the required software.’ ‘The software I need is really expensive.’ ‘The information I need is a big mess.’ ‘The information I need is really big.’

  13. Understanding Barriers ‘The information is in a language I don’t know.’ ‘The information is in a format I’ve never seen.’ ‘The taxonomy used doesn’t work with mine.’ ‘I’m not sure if what I’m seeing is a data quality issue or real.’

  14. Defensive Barriers ‘I collected this data and want to publish on it.’ ‘People won’t interpret my data correctly.’ ‘I don’t want to be liable for decisions made.’ ‘This data’s quality is too low to stand behind.’

  15. Square Pegs and Round Holes Coverages and Features A grid cell IS NOTa point measurement!!!

  16. Scale Discontinuity

  17. Anecdotes!...Because they are instructive!

  18. Water Quality Portal USGS, EPA, USDA Joint service providing water quality and other environmental monitoring data. http://www.waterqualitydata.us

  19. Integrated Ocean Observing System

  20. Weather Underground 42K Current Conditions Weather Stations!

  21. Geo Data Portal Data Integration Framework Weather Common architecture for access and processing multiple environmental data resources! Landscape Climate Center for Integrated Data Analytics: Nate Booth, Tom Kunicki, Dave Blodgett, Jordan Walker, Ivan Suftin, I-Lin Kuo.

  22. Enabling Technologies….

  23. ____.data.gov – Big Win! Data access typeis a first class citizen! Includes both human and machine metadata. Machine-interpretability is an expectation. Content management systems and catalogs are becoming data service providers!!!

  24. Forward!

  25. Principle #1: Data Object Patterns We must continue to identify and model the common patterns our data adhere to. Non-interpretive content / attributes should be provided by service ‘methods’. These patterns must transcenddiscipline or implementation.

  26. Principle #2: Domain Semantics. Semantic relationships are necessarily governed by a given scientific domain itself. This is Foundational to all additional interdisciplinary concerns.

  27. Principle #3: __ - Agnostic Standards Standards, specifications, and best practices must be ____ - agnostic. A standard can be implemented using any technology, in any discipline. eg. WaterML2 -> TimeSeriesML

  28. Principle #4: Identity Management Uniqueness can’t be taken for granted and must be curated very deliberately. You are not your location. Neither is a place. Foundational to linking any and all information to an entity.

  29. A few thoughts to leave you with… Maps are metadata. Index-based data access is dead. A Geospatial database should be coherent without it’s spatial table.

  30. Summary A standard is an established generalization. Scientific disciplines govern their semantics. Open-access (the internet) must be a given.

More Related