1 / 26

PDS4 Project Report

PDS4 Project Report. PDS MC F2F Columbia, MD Dan Crichton April 2-3, 2013. PDS4: The Next Generation PDS. PDS4 is a PDS-wide project to upgrade from PDS version 3 to version 4 to address many of these challenges An explicit information architecture

vinnie
Download Presentation

PDS4 Project Report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PDS4 Project Report PDS MC F2F Columbia, MD Dan Crichton April 2-3, 2013

  2. PDS4: The Next Generation PDS • PDS4 is a PDS-wide project to upgrade from PDS version 3 to version 4 to address many of these challenges • An explicit information architecture • All PDS data tied to a common model to improve validation and discovery • Use of XML, a well-supported international standard, for data product labeling, validation, and searching. • A hierarchy of data dictionaries built to the ISO 11179 standard, designed to increase flexibility, enable complex searches, and make it easier to share data internationally. • An explicit software/technical architecture • Distributed services both within PDS and at international partners • Consistent protocols for access to the data and services • Deployment of an open source registry infrastructure to track and manage every product in PDS • A distributed search infrastructure

  3. Label Schema Data Element Planetary Science Data Dictionary has Class Information Architecture Concepts Product Information Model Tagged Data Object (Information Object) Used to Create <Array_2D_Image>     <local_identifier>MPFL-M-IMP_IMG_GRAYSCALE…     <offset unit="byte">0</offset>     <axes>2</axes>     <axis_index_order>Last Index Fastest…     <Element_Array>         <data_type>UnsignedMSB2</data_type>         <unit>data number</unit>     </Element_Array>                <Axis_Array>         <axis_name>Line</axis_name>         <elements>248</elements>         <unit>not applicable</unit>         <sequence_number>1</sequence_number>     </Axis_Array>     <Axis_Array>         <axis_name>Sample</axis_name>         <elements>256</elements>         <unit>not applicable</unit>         <sequence_number>2</sequence_number>     </Axis_Array> </Array_2D_Image> Validates Expressed As Extracted/Specialized Describes Data Object

  4. Client A Client B Service Interface C Service System Design Approach • Based on a distributed information services architecture (aka SOA-style) • Allow for common and node specific network-based (e.g., REST) services. • Allow for integrating with other systems through IPDA standards. • System includes services, tools and applications • Use of online registries across the PDS to track and share information about PDS holdings • Implement distributed services that bring PDS forward into the online era of running a national data system • With good data standards, they become critical to ultimately improving the usability of PDS • Support on-demand transformation to/from PDS

  5. Summary of Progress to Date • Requirements in place (approved by MC 3/2010) • PDS-wide Architecture defined • Major reviews conducted • Initial PDS products defined using maturing PDS4 specification • LADEE, MAVEN teams developing labels • System builds grouped by purpose: build 1(a,b,c,d), 2 (a,b,c), 3(a) • Used to establish a rigorous release process • Includes deployment of software, data standards, documents and system integration testing • As stability has increased, we have moved to 6 month builds • Operational capabilities deployed • Registry and Harvest infrastructure in place at EN • Central catalog migrated to registry; High level search migrated • Information model released for limited use to select data providers • PDS3 data now being ingested into PDS4 system at EN • Beta test conducted by Atmospheres to get user input on PDS4 concepts (XML, Bundles, etc) • LADEE PDS4 Peer Reviews conducted • IPDA endorsement and plans to move to PDS4 • International implementation by PSA for Bepi Colombo underway • International access prototype demonstrated at IPDA 2012 SC

  6. PDS4 Software Development Lifecycle NASA Software Management Guidebook – NASA-GB-001 Evaluation by users as early as possible!

  7. Project Lifecycle – mapped to NPR 7120.8 per SDR1 Project Lifecycle Pre- Formulation Formulation Implementation KDP: Prelim Design KDP: Release LADEE/ MAVEN KDP: Project Plan & Arch KDP: Study Build 1: Prototype build Project Lifecycle Gates & Major Events 1d 1b 1c 1a Build 2: Release to LADEE/MAVEN Project Plan PDS4 Design PDS4 Prelim Architecture Begin Study Project Study/ Concepts Build 3: V1.0 of PDS4 Stsandards Operational Readiness Review (LADEE/MAVEN) (November 2011) PDSMC Preliminary Design (August 2009) Build 1b Internal Stds Assessment (Dec 2010) PDS System Review II (June 2011) Project Reviews PDSMC Concept Review (Dec 2007) PDSMC Arch Review (Nov 2008) PDSMC Impl Review (July 2008) PDS System Review (Mar 2010) Build 1c IPDA Stds Assessment (April 2011) Build 1d External Stds Assessment (Aug 2011) Phoenix Beta Test (Dec 2012)

  8. PDS4: Build Structure 8

  9. Build 2 ORR • Begin PDS4 label design for LADEE and MAVEN; Begin planning/testing migration • Deploy a PDS4 standard that supports the Policy on “Acceptable PDS4 Data Formats” • Transition the central catalog to the registry infrastructure • Deploy early PDS4 software tools and services

  10. PDS4 Planned Mission Support ISRO, JAXA planning PDS4 missions LADEE (NASA) BepiColumbo (ESA/JAXA) InSight (NASA) MAVEN (NASA) Osiris-REx (NASA)

  11. PDS3 Implementation Current Missions PDS3 Archive @ DNs Datasets + Products PDS3 Central Catalog PDS3 Pipeline PDS3 Ingest PDS3 Services PDS3 Metadata Index PDS Catalog Info Users PDS Portal

  12. Today Current Missions PDS3 Archive @ DNs Datasets + Products PDS3 Central Catalog PDS3 Pipeline PDS3 Ingest PDS3 Services PDS3 Metadata Index PDS Catalog Info PDS Catalog Info Users Updated PDS Portal toPDS4 Transform PDS4 XML Label PDS4 Registry PDS4 Metadata Index • PDS4 infrastructure deployed at EN; Central catalog migrated.

  13. Support for LADEE/MAVEN Current Missions PDS3 Archive @ DNs Datasets + Products PDS3 Central Catalog NOTE: PDS3 Services phased out overtime PDS3 Pipeline PDS3 Ingest PDS3 Services PDS3 Metadata Index PDS Catalog Info PDS Catalog Info Users Updated PDS Portal toPDS4 Transform PDS4 XML Label PDS4 Registry PDS4 Metadata Index PDS4 Archive @ DNs PDS4 Ingest PDS4 Pipeline PDS4 Services Build 4 New Missions (LADEE, MAVEN, O-Rex, Insight) PDS3 Data Migration • PDS4 infrastructure deployed at EN; Central catalog migrated. • Working towards acceptance of new PDS4 mission data

  14. Protégé Ontology Modeling Tool Requirements & Domain Knowledge The Information Model Driven Process & Artifacts PDS4 Information Model PDS4 Data Dictionary (ISO/IEC 11179) Filter and Translator Information Model Specification Registry Configuration Parameters PDS4 Data Dictionary (Doc and DB) Query Models This is given to data providers XML Schema (pds) XMI/UML XML Document (Label Template)

  15. Capabilities/Operational Deployment • Design and Generate PDS4 Products • Information Model 0310b frozen (build 3b) • COTS tools tested and in use to support PDS4 Schema development (internal to PDS) (build 2) • DN tools for generating PDS4 products developed at multiple nodes (build 2) • Validation • Validate tool developed for PDS4 label validation (build 2) • Harvest/Registration • PDS3 central catalog migrated to registry (build 3a) • CI tool in place to register PDS3 catalog data (done, e.g., MSL) in a PDS4 registry (build 3a) • Harvest in place to register PDS4 bundles and resources in PDS4 registry (build 3a) • Registration of PDS4 web resources (build 3a) • Registration of PDS4 bundles (build 3b) • Search/Access • Deployment of the PDS4 search service at EN (build 3a) • Generation of a PDS4 search index for PDS3 data (build 3a) • High level search of PDS3 data sets, PDS4 bundles, web resources, IPDA (build 3b) • Distribution/user tools (build 4) • This the need area for future builds, but is dependent on having good bundles … • We need to take advantage of the flexibility of XML in presenting PDS4 to end users

  16. Build 3b Deliverables* • Software System • Data Standards* • Registry Service • Harvest Tool • Validate Tool • Security Service • Report Service • Search Service • CI Tool • Upgraded portal search and page views to support PDS4 • Information Model • XML Schemas • Data Dictionary • Concepts Document • Standards Reference • Data Providers Handbook • PDS4 Example Products * Posted to http://pds.nasa.gov/pds4

  17. Build 3b Screenshot http://pds-engineering.jpl.nasa.gov/index.cfm?pid=145&cid=187

  18. Build 3b Finalization • March 2013 • Freeze the model • Synchronize the model and software • 29 March 2013 • Capture in CM; Deploy IM, documentation, software, test plans, etc to PDS4 build 3b site • April 2013 • EN regression testing to verify software against PDS4 examples • Compile test report • Pending no major issue, deploy Standards as V1.0 to support LADEE, MAVEN SIS finalization and IPDA SC

  19. Build 3b Testing Process • Engineering has developed a test plan (Emily Law will lead) • Test cases • Data product matrix • Engineering performs regression tests using example bundles and data products • Validate functional requirements L3 – L5 • Engineering will write a test report

  20. Beta Testing • PDS4 development has performed increasing levels of beta testing with users since build 1 • Latest beta test by Atmospheres included review of PDS4 labels, bundles and documentation Search Service Deployed for Build 3a Includes links to Archive Resources

  21. PDS4 Archive Support Pages • Cassini Archive Support page has been received well • In PDS4, we are developing archive pages (e.g. Phoenix) for our migrated data • We can add tools/services over time - this gets the data online • Those pages show up at the top of a search now with the new search service from build 3a

  22. IPDA Interactions • IPDA endorsement of PDS4 (July 2012) • PSA moving forward with PDS4 • Dave Heather leading a IPDA PDS4 implementation project • Will be at JPL in May 2013 for coordination meetings for Bepi Columbo • Search/access to international archives • Demonstrated as a prototype at IPDA SC and COSPAR in July 2012 • Leverages the PDS4 REST-based architecture • Will be operational with build 3b deployment 7th IPDA Steering Committee (July 2012) (ASI, CNES, ESA, ISRO, JAXA, NASA)

  23. Architecture Search Service supports the PDS and PDAP protocols enabling development of other portals and applications on this infrastructure. • Registered Objects: • Websites • Data Sets • Investigations, Instruments, etc. • Tools and Services Search results include mission support pages and other more specific search interfaces. Cross-Agency Search and Access

  24. IPDA Search Results Data sets and other results returned include products from NASA and PSA. PSA data sets currently link directly to a PSA web-based interface.

  25. Summary • Significant progress has been made and operational capabilities have been deployed since build 2b • We need stable data standards (v1.0) so we can move forward with LADEE, MAVEN and others, and focus on data distribution services. • We need to be producing bundles • That will push us forward • We have international alignment! • We should good progress at the July 2013 meeting

  26. Backup

More Related