1 / 20

Introduction

Introduction. Two components Data Exchange Infrastructure (DEI) Spatial Data Model (SDM) Together, they make up “NCDDC Middleware”. Introduction to NCDDC Middleware - what it does and why. Richard Owens. What Is DEI (High-level overview).

markku
Download Presentation

Introduction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction • Two components • Data Exchange Infrastructure (DEI) • Spatial Data Model (SDM) • Together, they make up “NCDDC Middleware”

  2. Introduction to NCDDC Middleware - what it does and why Richard Owens

  3. What Is DEI (High-level overview) DEI is a client/server framework by which applications and data repositories can interact. The goal of the framework is to solve the problems of Data Access and Data Representation. DEI is a CORBA-based set of interfaces which are used to locate and transfer geospatial data. It is a client/server architecture which uses CORBA's IIOP (uses TCP) as its transport protocol.

  4. Data DEI Concepts • Gateway • Server component that adapts DEI interfaces to a specific • data source • Data source dependent • DEI Client • Any software that uses DEI CORBA interfaces to • communicate with gateways CORBA Client Gateway

  5. DEI Concepts (continued) • Data Flavor (formats) • Like a MIME type

  6. What DEI Is Not Complete • Lacks generic query processing • Poor support for very large datasets Data Archive/Storage Data Format • Supports them, though Web services (graphic here)

  7. History Originally Navy NRL funded project Now being developed by NCDDC Current Status Java based gateways via Java GDK Current focus C++ gateways Original focus, now lagging behind Java Future Mature Java GDK, then C++ GDK Add generic query processing Add support for very large datasets Open source

  8. What is the SDM • Object-oriented library that is used to represent and manipulate spatial data objects. • 3D • Some other dimensional capabilities, too • Structured Grids • Regular, Rectilinear, Curvilinear, Projected • Unstructured Grids • Coordinate Systems • Cartographic, Cartesian, Cylindrical, Spherical • GIS-like structures • Points, Lines, Polygons, etc.

  9. What is the SDM (continued) • Java • ~160 classes • ~1050 Unit tests

  10. Key Features • Called the “Field Spatial Data Model” by its developer because it uses a ‘field’ concept, which relates spatial objects to data. • Fundamental storage is multi-dimensional array. • Decoupling of data storage from its geometry and topology. • Requires basic metadata (quantity, units, validity, accuracy, etc.) for all data. • Aids in interoperability

  11. Key Features (continued) • Small, consistent API for data access. • The SDM hides all implementation details of data representation behind its abstract interfaces. • Decouples many issues regarding storage and internal representation of the data, while providing a consistent interface to it. • Data may be logically represented one way and physically stored another way. • Many manipulations may be done without actually having to have the data physically in memory.

  12. Key Features (continued) • Externalized format for representing datasets • XML based (FDML) • Can be generated without the use of the SDM.

  13. DATA DATA MESH FIELD ARRAY ARRAY Field Concept

  14. What SDM Is Not Transport mechanism Storage/Archival Magic. Won’t solve all problems. Just a data format Complete

  15. History • EPA project • Ported from Eiffel to Java in 2002

  16. Future • Documentation • FY03 • Text data (done) • Generalize dimensional support beyond existing 3-D • (to 2-D, 1-D or N-D) • Investigate Time dimension support (complete in FY04) • Application Layer Services • Common Query mechanism • Subsetting (parameters, spatial) • Format support (NetCDF, Shapefile) • FY04 • Aggregation of data types • Time dimensions support - including subsetting in • Application Layer • Open Source

  17. NCDDC Middleware Objective Develop a middleware architecture that can be used to tie data and applications together across a local or wide-area network in a way that is independent of the data and application platforms, operating systems, and formats.

  18. NCDDC Middleware Defined • “NCDDC Middleware” is the fusion of DEI with SDM • DEI • Data Transport • SDM • Data Representation • DEI + SDM (“NCDDC Middleware”) • Spatial data representation distributed over a network • Common form for data, and the means for transporting it

  19. DEI/SDM, HTTP/HTML Analogy Transport HTTP, DEI Content HTML, SDM

  20. Benefits • Gateways can be created that can transport descriptions of the data with embedded links to the actual data – the actual transport of the data is done piecewise, “on demand”. • Can have datasets that are larger than memory. • Caching mechanism improves performance. • Data is transported in binary form • Sun’s XDR format • Foundation for netCDF • Fast and efficient • Since DEI is integrated within the SDM, clients can use SDM interfaces to access gateways without the need to code to DEI’s CORBA interfaces.

More Related