Migrate only past missions & newly awarded missions ( InSight )

1. PDS Imaging Node PDS4 MigrationLisa Gaddis (USGS)Sue LaVoie (JPL)November 30, 2012PDS Management Council MeetingUCLA PDSMC November 29-30, 2012

2. Migrate only past missions & newly awarded missions (InSight) Justification: for active missions, community analysis is ongoing with existing software & tools Prioritize based on need for class development/follow-on missions and popularity Retain PDS3 labels & metadata within PDS4 structure Migrate mission datasets requiring no (or simple) data manipulation Retain copy of any data requiring manipulation Migrate metadata only of mission datasets requiring data manipulation Justification: complex data migration is too risky Retain PDS3 data format & label Create PDS4 labels and metadata to enable complex data product searches across all holdings Decisions/Action required by MC & Data Modelers Policy allowing waiving of data migration for complex legacy data sets Dictionary terms/keywords enabling description of legacy formats Modify IMG systems to support both PDS3 & PDS4 data sets & formats PDS4 Migration Strategy PDSMC November 29-30, 2012

3. Process Identify data set for migration Create/update Discipline Classes, provide input to PDS-wide Classes Map PDS3 to PDS4 labels and ancillary files/data Create schema Modify existing software tools & scripts to automatically create labels, collection and bundles Generate Sample Bundle(s) Verify Iterate Informal IMG internal review & testing Load Bundle into IMG Registry Identify next data set ….. Repeat iterative process until Data Model and Classes are “final”, refining all along the way. Then perform the full data set migration on priority data sets PDS4 Migration Approach PDSMC November 29-30, 2012

4. PDS4 Migration - Top Priority Data Sets PDSMC November 29-30, 2012

5. PDS4 Migration – Waive Data Format Change PDSMC November 29-30, 2012

6. Clementine 750nm Basemap (Chris) Sample Bundle validated, reviewed by EN, updated and redelivered for ingestion Folders in bundle: data (6 map projected single band images & ISIS detached labels), document (.txt, .doc, .htm, .pdf versions of “volinfo” suite), miscellaneous (delivery & transfer manifests), xml_schema Cartography Class - Incorporating Federal Geographic Data Committee (FDGC) model. Still needs fine tuning Tools used: oXygen XML Editor, IMG Generate Tool, EN Validate Tool, ad hoc scripts Mars Pathfinder (MPF) IMP EDR (Elizabeth) Sample Bundle validated and delivered to EN for review and feedback Folders in bundle: data (200 files), browse (browse for each data file), context (instrument, instrumet host, investigation, resources, etc.), document (EDR SIS), geometry (gazetteer), xml_schema Imaging Instrument Parameter Class – in good shape for this data set & Clementine data set Tools used: oXygen XML Editor, IMG Generate Tool, EN Validate Tool, ad hoc scripts Migration Progress PDSMC November 29-30, 2012

7. Clementine UVVIS 5-band Mosaic (Chris) Multispectral data set Preliminary Bundle layout defined Mars Pathfinder (MPF) RVRCAM (Elizabeth) Tables and Engineering data sets Phoenix OM, RAC, SSI EDR (Amy) Precursor for InSight (launch 2016; Geo is lead node) Landed mission products and metadata Phoenix, MER, MSL heritage Work closely with Geo throughout Class work: Camera Model, Landed Geometry, Mars InSitu, likely more Migration – Next Up PDSMC November 29-30, 2012

8. Accomplishments Installed all PDS2010 software on IMG operational machine, including Registry, Registry UI, Harvest, and Validate Used Validate tool on test bundles prior to delivery to EN for review Continued development of IMG Generate Tool (label generation) to support ongoing data migration activities Created internal IMG Wiki to track PDS4 migration details, progress, lessons learned, open items, versions of dictionaries and schemas used, etc. Plans Harvest test & completed Bundles into IMG Registry Work with Engineering Node to communicate with central Registry System & Tools Status PDSMC November 29-30, 2012

9. Back-up PDSMC November 29-30, 2012

10. Imaging Node: PDS4 Data Migration • IMG manages a lot of complex data with a wide variety of formats • 203 data sets, ~13 M products • Many of these are not PDS4-compliant • 90% of IMG data sets require • Assessment & revision of classes of keywords • Discipline Classes • PDS-wide Classes • Manipulation of data to achieve compliance • 25% of IMG data sets • But, we assume that migration of all data sets is not required PDSMC November 29-30, 2012

11. IMG Data Migration: Data Manipulation • Data Manipulation • 25% of IMG data sets • 52 of 203 data sets • Potentially very time consuming and costly • Process: • Engage experts • Software development & testing • V&V • Peer Review? • Release • Issues: • Retain PDS3 version – forever? PDSMC November 29-30, 2012

12. IMG Data Migration: Classes • Class development will require • IMG for Discipline-specific classes • Cross-node participation for many others • Classes needed for IMG - prelim analysis (not all included on chart) • Calibration – PDS-wide • Compression – PDS-wide • Geometry • Camera Model – IMG • Surface Geometry – PDS-wide • SPICE Geometry – PDS-wide • Cartography – IMG • Imaging Instrument Parameter – IMG • Telemetry Parameters – IMG • Command Parameters – IMG • Processing History – IMG • Imaging Instrument Compression – IMG • Mosaic – IMG • Anaglyph (?) - IMG • Spectral Qube (?) - IMG PDSMC November 29-30, 2012

13. Past Missions PDSMC November 29-30, 2012

14. Active Missions PDSMC November 29-30, 2012