ESAC / ESTEC PSA status and interactions with PDS 26 th March 2010 Dave Heather

1. ESAC / ESTEC PSA status and interactions with PDS 26th March 2010 Dave Heather ESA-ESTEC, dheather@rssd.esa.int

2. Scope • Historical perspective • PSA general properties / services • Current mission archive status • PSA and PDS collaboration / interaction and issues • IPDA and interoperability

3. Definition and Purpose The Planetary Science Archive (PSA) is the initiative, the setup, the process and the implementation to preserve data from ESA’s spacecraft to planetary bodies, as well as supplementary information acquired in laboratories or ground-based observatories. The PSA is the primary resource for all scientific and engineering data from the European Space Agency’s planetary missions. • The prime objectives of the PSA are: • to support the experimenter teams in the preparation for the spacecraft and ground-based long-term archives • to enable and ensure the (long-term) preservation of these archives • distribution of scientific useful data to the world wide scientific community • provision of supplementary data services aiming to maximize the usage of planetary mission data and ease the scientific data analysis. http://www.rssd.esa.int/psa

4. Historical Perspective • GIOTTO Science Data Archive – initiated before we had a legal basis in the Agency. NASA participated via the International Halley Watch plus the MoUs signed for NASA support (DSN, Navigation). • ROSETTA Science Management Plan • ROSETTA ESA/NASA Memorandum of Understanding • Mars Express and Venus Express Science Management Plans • Bebi-Colombo Science Management Plan Rosetta Science Management Plan (ESA/SPC(94)37, page 28): “6. Provide the reduced and calibrated scientific data sets from his/her instrument in useable form to the RSOC for inclusion in the Rosetta Science Data Archive.” Page 42, RSOC reponsibilities: “the preparation of guidelines for science data archiving and - supported by the PI team - to create the Rosetta Data Archive.” Page 42: “5.4 Rosetta Scientific Data Archive … After this period, the scientific data products from the mission have to be submitted to RSOC in a reduced and calibrated form such that they can be used by the scientific community. RSOC will prepare the Rosetta Scientific Data Archive within one year of the receipt of the complete data sets from the individual Rosetta science investigations. Based on current technology, the archive would be distributed as a set of CD-ROMs. …” Mars Express Science Management Plan The Science Operations Centre will prepare the final Mars Express Scientific Data Archive (MESDA) within one year of the receipt of the complete data sets from the individual Mars Express Orbiter and Lander Module science investigations. Based on current technology and IMEWG recommendations on standards for scientific data of future missions to Mars, the archive would be distributed as a set of CDROMS based on the NASA Planetary Data System. BepiColombo, Science Management Plan Provide the scientific data (raw data, calibrated data, and higher level data), including relevant calibration products, to the BepiColombo ESA-JAXA archive in a format that will be agreed with the ESA SOC for application by the general science community. The ESA-JAXA science data archive, which will be compatible with the Planetary Data System (PDS), will be based on and part of the Planetary Science Archive (PSA) developed for Smart-1, Mars Express and Rosetta. ESA/C(2002)148: Rosetta ESA/NASA MoU “ARTICLE 10 - RIGHTS IN AND DISTRIBUTION OF SCIENTIFIC DATA … Following the period defined in 10.1 above, all scientific and ancillary Rosetta spacecraft data records will be deposited with the ESA Data Library, and NASA’s Small Body Node of the U.S. Planetary Data System (PDS) using PDS format. NASA will place the science data in a public archive in the Small Body Node of the U.S. Planetary Data System …” Giotto Mission The Giotto spacecraft was launched on 2 July 1985 to study comet 1P/Halley. Giotto encountered comet 1P/Halley on 13 March 1986 and came within 596 km of the nucleus. Giotto encountered comet 26P/Grigg-Skjellerup on 10 July 1992. Giotto instrument data are archived in co-operation with NASA’s PDS Small Bodies Node and were distributed on a set of CDROMs (“International Halley Watch Campaign”) A close collaboration with the PDS Small Bodies node was established and maintained up to today.

5. Data Rights and Duties • ESA Convention (SP-1271) • Resolution on the Rules concerning Information, Data and Intellectual Property ESA Convention, Article V In carrying out its activities under Article V, the Agency shall ensure that any scientific results shall be published or otherwise made widely available after prior use by the scientists responsible for the experiments. The resulting reduced data shall be the property of the Agency. ESA/C/CLV/Rules 5 (Final) “OWNERSHIP, ACCESS, USE AND DISSEMINATION OF RAW AND CALIBRATED DATA RESULTING FROM A PROGRAMME OR ACTIVITY OF THE AGENCY 1. OWNERSHIP OF RAW AND CALIBRATED DATA The Agency shall be the owner of all raw and calibrated data directly resulting from a payload flown in the context of an Agency’s programme or activity (excluding any data which are required for the control of the payload itself) including when the payload is furnished by a Provider.”

6. Planetary Science Archive Available since March 2004: http://www.rssd.esa.int/PSA Active development Datasets received from PI teams Needs to be Peer-reviewed by independent team Needs to be validated before being ingested into the PSA (check PDS format compliance) Around 13TB of data on hard disks Classical User Interface Map based interface for MEX Dataset Browser (not searchable)

7. PSA : one archive, several missions Giotto Huygens Rosetta Smart-1 Venus Express Mars Express

8. PSA-DH Personnel and Mission Contacts

9. Christophe Arviset ESA Jesus Salgado* INSA Inaki Ortiz* SERCO Ignacio Leon INSA Monica Fernandez INSA Neil Cheek SERCO ESAC Science Archives Team 2 FTEs for PSA Pedro Osuna* ESA User Interface Group Data Access Group

10. End Users, Services and Functionalities Offered • PSA Scientific Advisory Group • Helpdesk, Notification • Data Query and Retrieval • PDS Standard Support (currently only for Version 3!!) • Coordination and chair for Peer Reviews • Design, Production and Delivery of auxiliary data • SPICE conversion • Software consultancy and support • ESOC data long-term archive preparation • Science planning information, long-term archive preparation

11. PSA Advanced Interface • PSA supports 4 different views: • Query • Latest Results • Shopping Basket • Login/Register • (Request Monitor to come) Selection of Result Display Options Query Panel are ANDed when querying the database

12. Search Button Select Base Map Type (MOLA / VIKING…) Select Instrument / Detector 1. Click and drag to select your area of interest Zoom function / day/night projection 2. Press ‘Search’ and wait for footprints Access to ‘standard’ UI 3. Click on desired footprint Long / Lat information PSA UI – The Map Interface 5. Transfer at any time to the standard interface for advanced searches. Your search parameters will be remembered. 6. Return to the map browser, you can view day/night boundaries and sub-solar point for selected footprint. 4. Download directly or view label / docs etc.

13. PSA Browser Interface • Opening Screen Click on the instrument that interests you Select the data set you wish to look at Browse through the directories to locate the files you want Right click the product you want to save etc. to bring up the menu Left click to view directly (where possible)

14. Data Producers, Services and Functionality Offered • Data Archive Workshops on request (internal and external) • SPICE Workshops (on request – in co-ordination with NAIF) • Individual Archive Consultancy from early phase • Coordination of Mission Data Archive Working Groups (DAWG) • Coordination and support of Mission geometrical parameter information • Dataset Support/Development Tools • Product generation from telemetry to L2 (GDP) – used for C1 • Dataset validation • Dataset ingestion

15. PSA Data Producer Services • We offer a PSA Volume Verification Tool (PVV) • Data producer • Can verify the structure of the whole data set • Can verify the content of the ODL language • Can verify the references, catalogue structure, etc • Database • Can easily ingest data sets (standard delivery within one week) • PVV includes • Semi-automatic update of the tool • Direct connection to the PSA database (network based) • Always the latest dictionary • Support of former dictionary versions • PVV does not include • A guarantee of PDS compliance! • PVS: qualitative validation now operational!!

16. PSA Management Services • Statistics Tool (PST) • Currently does NOT account for FTData set Browser access. • Notification Management • Data Set and User Administration (PAT) • Dictionary Administration (PDT)

17. Current Status, Available Datasets • GIOTTO: data from comet Halley and Gripp-Skellerup • Comet Halley ground-based observations (Halley-Watch) • Comet Wirtanen ground-based observations • Mars Express: instrument and auxiliary data available (~11Tb) • HRSC Radiometric, Map-Projected (to July 2009) and DTM (to Dec 2006) data. • ASPERA NPI and ELS data up to June 2009. • OMEGA, SPICAM, MARSIS and MaRS data all available and deliveries continue. • PFS data only on FTP. • Calibrated data from several instruments now coming in. • Working group on Mars upper atmosphere

18. Current Status, Available Datasets • Huygens: instrument and housekeeping data available for all instruments • New GCMS data has just been sent • DISR re-ingestion planned with improved ‘processing’ • Venus Express: first data sets released Spring 2009 • VMC, SPICAV-SOIR, MAG available and regular deliveries. • First SPICAV data now available. • VIRTIS still closing out review, but available via ftp. • Rosetta: first data release still in preparation, pending PDS/PSA agreement • PDS collaborations ongoing • Validation is a major issue. • Steins fly-by joint PSA / PDS review last October last year. Very successful.

19. Current Status, Available Datasets • SPICE: Rosetta, MEX and VEX data sets released. New versions will be ingested after Easter. • Chandrayaan-1: data pipeline running at ESAC for CIXS and SIR2. • SMART-1: AMIE, SIR, D-CIXS and XSM data being prepared for release in May. • BepiColombo: data handling and archive support now taking shape. New approach for ESA. • Updated PSA WWW area – continuous improvements: • Tools • Documentation • FTP Access • http://www.rssd.esa.int/PSA

20. PSA and PDS Interactions • It was decided at an early stage that PSA data would comply with PDS Standards: • To maximise the cross-compatibility of ESA and NASA data • Existing and well established standard in the community • We have good contact and excellent support from several of the PDS Nodes: • MEX: Geosciences / PPI. MOU between ESA and NASA • Rosetta: Small Bodies Node (primary). International mission. • VEX: Atmospheres Node. Interoperability. Review support. • Ancillary data: NAIF. SPICE used for all ESA planetary missions.

21. PSA and PDS collaboration Rosetta issues • Rosetta first review took place Feb 2007 but data are still not available • Full ingestion (and hence compliance) required at PDS • How does one ensure full compliance? VTOOL? LVTOOL? TBTOOL? PVV? Others? REVIEW? FULL / OPEN PSA/PDS INVOLVEMENT IN ARCHIVE DEVELOPMENT CYCLE? ALL OF THE ABOVE… MORE!?

22. PSA and PDS Interaction -> IPDA / PDS4 • Development of our validation tool has highlighted several inconsistencies in the PDS Standards and the way in which various experts interpret them » IPDA • PDS and PSA are drawing upon the lessons learned on both sides of this working relationship in order to refine and streamline the Standards. • These ‘lessons learned’ are being used by the IPDA to help develop a refined set of IPDA recommendations for archiving that will allow for international archive interoperability and avoid similar issues in future. • Some issues also input to PDS2010 (PDS4) development. • Involved in PDS2010 System Review and in IPDA assessment of PDS4

23. IPDA - VEX Interoperability • VEX - First interoperability studies between PDS and PSA • Access to data remotely • Full search functionality • Transparent access to data on various servers • Planned for PDS data access on VEX

24. Open Issues • “Difficulty” to control Dataset delivery schedule • Affects PSA-DH schedule • Affects PSA-DT schedule • Incomplete deliveries to the scientific community as calibrated data deliveries are not available in time by some experimenter teams on the on-going planetary missions • Still not fully resolved • Insufficient resources within some of the experimenter teams • To be solved with new concept in Solar System Science Operations Division - but can only be done partially. Issue was addressed with delegations (SPC) on a number of occasions • Parallel activities in member states – we try to coordinate and focus resources

25. Open Issues • Standard evolution. With IPDA (all international partners), and maintaining our well established links with PDS, we follow PDS-Standard Evolution. • Peer Review handling – data are sometimes slow to be released through the formal procedure • Requires time and significant resources to perform review • Delay in DS ingestion into PSA • Data set ingestion turnaround • Manual process • Potential Remedy actions • Planning & monitoring through Dataset Spreadsheet • Try automate ingestion process? • Review Peer-review concept? • Immediately release the incoming Datasets as “provisional”?