Planetary Science Archive

1. Planetary Science Archive PSA User Group Meeting #1 PSA Archiving Standards PSA UG #1  July 2 - 3, 2013  ESAC

2. The PDS Standards • Within the PDS Standards, data are organized into volumes and data sets, collecting together observations of similar type, processing level, and/or from a specific mission phase or observation campaign. • All PSA data are compliant with NASA’s Planetary Data System (PDS) Standards • This is the first step in ensuring that the data stored in the PSA are as widely usable as possible and will remain so for many years to come • Widely accepted and understood within the planetary science community • Standards available online and downloadable as a ‘Standards Reference’ pdf file.

3. The PDS Standards • All PDS compatible data formats conform to a unified standard format incorporating documentation, calibration, and raw and processed data files. • The format distinguishes: • Meta-data Files • At the intermediate level are files which describe the circumstances and parameters of the data collection (for example: the instrument used, the observatory site, the type of data collected, etc.). These files are called "catalog objects" or just "catalog" files. • Volume/Dataset Description Files • These files are included on any distribution medium intended to be an entire, self-contained archive - for example, a set of DVDs or a ZIP file. The volume description files detail the organization of the data in the archive (in terms of directories, e.g.) and any additional documentation and software which might be included.

4. The PDS Standards • The lowest level is that of the data themselves. • Each file will have a PDS label either appended to the beginning of the file or, more often, in a separate but proximate file. "One file, one label" is the general rule. • The LBL file must contain a full description of the data object(s) • Allows for all PDS products to be read by existing standard software • Allows for long-term data usage with clear description of data to allow users to read files with their own software

5. PDS Resources • http://pds.jpl.nasa.gov • More information on the PDS and complete versions of the Standards Reference and Data Dictionary • The Data Dictionary can be found here • Supporting Software Tools can be downloaded here

6. A Few PDS Tools • http://pds.jpl.nasa.gov • Go to the Tools menu • NASA View • A useful tool for visualising labels and simple small data products • PDS Software Tools • Some general tools reading tables and manipulating PDS files.

7. NASA View Example • NASAView can open simple PDS compliant data • Click on File -> Open Object and select the product you want to see • You can also click on Label -> Full Label to view the complete PDS label with full information on the data product being viewed

8. ReadPDS • To retrieve the latest fully tested release or previous releases, check out the page: • http://pdssbn.astro.umd.edu/nodehtml/software.shtml • Within your IDLDE, select the File->Preferences and check that your path contains the ReadPDS. • ReadPDS is an IDL package that is maintained and distributed by PDS's Small Bodies Node (SBN). It allows most PDS compliant products to be read in to the IDL software.

9. ReadPDS 2 • To run the ReadPDS you also need to have the astrolib library installed. If not already done, you can retrieve the astrolib library at http://idlastro.gsfc.nasa.gov/ . • Read the aareadme.txt file distributed with the ReadPDS package. There is no further way to test your installation than to call a procedure within IDL. So, on your IDLDE command input, enter e.g.: • If the routine READPDS is compiled successfully, your installation is ok.

10. Planetary Science Archive PSA User Group Meeting #1 PSA Archiving Process PSA UG #1  July 2 - 3, 2013  ESAC

11. Top level interfaces and interactions • The full archiving process involves many interactions and interfaces • PSA-DH interaction with the PDS and IPDA for archiving standards and data requirements. • Data sets developed by PI teams with strong iterations with Archive Scientists in PSA-DH • Project Scientists and science team involvement during development of data sets and content requirements (Archive Plan) • Delivered data sets are peer reviewed by independent reviewers (PS and PSA-UG involvement) • After validation, data are passed to the SAT for ingestion. Strong iterations are required

12. Top-level Interfaces

13. Data Set Lifecycle

14. Top level data set Lifecycle • PSA-DH consult with PI teams during data set, documentation and volume preparation • Documentation and initial pipeline data are reviewed. • In the nominal delivery phase, regular deliveries are expected. These are: • Validated internally with PVV, QBTool, PVS, spot checks (future DVal) • Once validated, data are passed to SAT for import, ingestion and if applicable, the import of geometry information • Process is tracked through a Google spreadsheet (future Archive Process Controller)

15. Validation / Ingestion timescale • Overall turnaround time for Datasets Verification and Ingestion is between a few days and a couple of weeks • Depending on Dataset size and complexity, some people availability (mostly holidays) • No datasets “backlog” • Nominal situation – aim for a 1 week turnaround overall. • Situation continues to be monitored through • Datasets Google Spreadsheet • Datasets Verification and Ingestion statistics

16. Data set tracking spreadsheet