220 likes | 329 Views
User Working Group 2013. Data Access Mechanisms – Status 12 March 2013 http://podaac.jpl.nasa.gov. PO.DAAC Functional Areas. Data Access. Data Management & Stewardship. Provide intuitive services to discover, select, extract and utilize data .
E N D
User Working Group 2013 Data Access Mechanisms – Status 12 March 2013 http://podaac.jpl.nasa.gov
PO.DAAC Functional Areas Data Access Data Management & Stewardship Provide intuitive services to discover, select, extract and utilize data Preserve NASA’s data for the benefit of future generations Science Information Services Provide a knowledgebase to help a broad usercommunity understand and interpret satellite ocean data and related information
PO.DAAC Functional Areas Data Access Data Management & Stewardship Provide intuitive services to discover, select, extract and utilize data Preserve NASA’s data for the benefit of future generations Science Information Services Provide a knowledgebase to help a broad usercommunity understand and interpret satellite ocean data and related information
High-Level Functions Data Providers Information Providers Consumers Information In Data In Information Out Data Out High-Level Access Tools Ingest Web Portal Visualization Web Services & Publishing Direct Data Access Inventory Archive
High-Level Functions Data Providers Information Providers Consumers Information In Data In Information Out Data Out High-Level Access Tools Ingest Web Portal Visualization Web Services & Publishing Direct Data Access Inventory Archive
Applicable 2012 UWG Recommendations • Recommendation 24 – Harden tools: Closed. (Tool improvements have focused on stability, debugging, and updates, along with selected re-architecture to improve maintainability.) • Recommendation 25 – Improve linkages between tools and datasets: Closed. (Links to applicable tools and services now exist on the dataset information pages (access tab) in the beta site.) • Recommendation 27 – Harden HITIDE and add datasets: Partially Closed. (The HITIDE interface has been reworked and functionality has been standardized. Extraction methods are still being worked and datasets will continue to be added.) • Recommendation 28 – Decommission POET: Open. (Decommission is pending the addition of all datasets from POET into THREDDS. Updates and testing are in progress.) • Recommendation 29 – Add datasets in THREDDS: Partially Closed. (Applicable datasets from the recommendation have been made available. Datasets will continue to be added.) • Recommendation 34 – Report on how UWG Homework shaped FY13 activities: Closed. (The UWG homework responses aligned with our development goals, but additionally served as a vetting document for evaluating updates.)
Outline Purpose of this Presentation: Data Access Mechanisms Status • Provide a functional overview and architectural response Presentation Outline: • Technology mapping and infusion • Current status + Outlook of each functional area: • Direct Data Access • High Level Access Tools • Web Services
Technology Mapping (Infusion Process) Lessons-Learned New Technologies User Needs Standards New Approaches PO.DAAC Vision Cloud Needs Pool ESDIS Goals Selection Infusion “Technology on purpose” Process for Infusion andDeprecation Leveraging ESDSWG Tech Infusion Working Group’swork Eye to the outside (Other DAACs, other agencies, etc.)
Direct Data Access Data Providers Information Providers Consumers Information In Data In Information Out Data Out High-Level Access Tools Ingest Web Portal Visualization Web Services & Publishing Direct Data Access Inventory Archive
Direct Data Access: FTP • Standard way of doing business • Our most popular means of access • High degree of user-dependency
Direct Data Access: OPeNDAP • Widely used in the Earth Science communities 25
Direct Data Access: THREDDS • Popular capability in Oceans and Climate communities • Specifically requested by the UWG 29
Direct Data Access: FastCopy • File transfer mechanism requested by specific partners • Alternative to FTP
Web Services & Publishing Data Providers Information Providers Consumers Information In Data In Information Out Data Out High-Level Access Tools Ingest Web Portal Visualization Web Services & Publishing Direct Data Access Inventory Archive
Web Services: Consolidated Web Services (CWS) • New paradigm for PO.DAAC (unsilo’ing) and our Users • Better exposes PO.DAAC’s services • Allows users to quickly write their own clients • Provides flexibility and agility New 25
Web Services: Publish / Subscribe • New paradigm for users • No longer have to crawl the FTP site • Improves client’s ability to customize their processing New 25
High Level Access Tools Data Providers Information Providers Consumers Information In Data In Information Out Data Out High-Level Access Tools Ingest Web Portal Visualization Web Services & Publishing Direct Data Access Inventory Archive
High Level Access: Dataminer • Selected for extraction capabilities • Developed externally • Limited ability to modify software
High Level Access: HITIDE • Fills need for Level 2 extraction • Refactors and expands upon Dataminer'sfunctionality New 27 24 34
High Level Access: LAS • Fills need for Level 3 capabilities • De-facto standard in the Ocean Science community • Helps with decommission of POET New
High Level Access: POET • Provides access and extraction of Level 3 data • Getting expensive • LAS is a more broadly accepted alternative 28