1 / 19

FY10 Data Releases & Volumes

PDS Imaging Node FY10 Mid-Year Report Lisa Gaddis (USGS) Sue LaVoie (JPL) March 25, 2010 PDS Management Council Meeting Washington, DC. FY10 Data Releases & Volumes. IMG Time Budget: Weekly Average. Mar10 Snapshot.

errin
Download Presentation

FY10 Data Releases & Volumes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PDS Imaging Node FY10 Mid-Year ReportLisa Gaddis (USGS)Sue LaVoie (JPL)March 25, 2010PDS Management Council MeetingWashington, DC PDSMC March 25-26, 2010

  2. FY10 Data Releases & Volumes PDSMC March 25-26, 2010

  3. IMG Time Budget: Weekly Average Mar10 Snapshot • Mission Interface (Archive and Baseline) activities comprise ~72% of our time budget • Management, Technical Operations for IMG and PDS (incl. PDS2010 WGs), data restorations comprise the remaining ~28% PDSMC March 25-26, 2010

  4. Data Preservation & Usability: IMG Data Integrity PDSMC March 25-26, 2010

  5. Data Accessibility: Delivery, Services • Imaging Node Web Site • Mission-specific interface points to online image data via delivery services and other Node Collections, documentation, data release calendars, home page, PDS-wide search • Photojournal • ‘Best of’ NASA image collection released through projects, media & news events • Captioned images used by media, scientists, educators, public • Map-a-Planet (MAP) • Web Map Services (WMS) interface delivers derived products from PDS data • Widely used map bases for other delivery systems within and outside of PDS • Planetary Image Atlas • Serves image data from all past & current planetary missions, updated as deliveries are received • Single- and multi-mission searches and bulk download • Map-based searches using Google Web Toolkit (GWT) • Unified Planetary Coordinates (UPC) Database • Delivers image metadata with unified coordinates and improved geometry • Geometry ‘backbone’ for Atlas, ODE (GEO), ISIS3, Google Mars/Moon, others • Map-based interface (PILOT) provides db access, footprint plots • Supports only selected PDS image collections used for cartography and mapping PDSMC March 25-26, 2010

  6. Imaging Node Web Site Status: • Updates to look & feel, usability • Pages widened for easier readability and to accommodate more information • Improved navigation: more intuitive links & easier to find things • Standard templates created for pages to improve readability and maintenance • Expanded ‘HELP’ • New content • SISes and other documentation added for all missions • Mission pages & software/tool links for Mariner 9 & 10, LRO, KAGUYA (SELENE) • News on data releases Plans: • Continue to create scripts for a fully automated web site • Continue to create and/or distribute tutorials for accessing PDS data for each mission supported • Continue to add missions and supporting information, as well as pointers to data and services, as new data releases are received http://pds-imaging.jpl.nasa.gov/ PDSMC March 25-26, 2010

  7. Planetary Image Atlas Status: • New missions added • Database, metadata, thumbnail/browse, user interface • LRO (LROC, LAMP) – LROC feeds Atlas via OODT from ASU • Voyager, Galileo • Completed modification of Atlas to support Cassini Mission Ops & MER Mission Ops • Completed integration of IAU Gazetteer of Planetary Nomenclature with Atlas to enable feature-based searches • Metrics: 5.3 TB/month downloaded, 14,381 visits/month Plans: • Google Maps-based searches for more targets • Footprints on maps • Image calibration • XML version of PDS labels • Multi-mission searches for products from both orbital & surface missions • Integrate with PDS2010 services: Ingest & Preparation, Inventory & Monitor • Missions to Add • Viking Orbiter & Lander, Clementine, Magellan, Mariner 9 http://pds-imaging.jpl.nasa.gov/search PDSMC March 25-26, 2010

  8. Photojournal Status: • Space ImagesiPhone app released (free app hosting hundreds of images; Photojournal provides backend functionality) – jointly funded by JPL Media Relations • Upgraded all 4 operational systems and 1 development system (hardware, database and tomcat servers) • Added Social Networking links (Twitter, You-Tube, Flickr, etc) • Added RSS feed and ‘Share’ capabilities • Metrics: >2 TB/month downloaded; ~100 images/month added Plans: • Redesign database schema and disk storage directory approach to improve performance due to huge data volume • Redesign as a 3-tier web application, enabling more interactive experience • Enable secure upload and preview of images prior to quality control and publishing • Add batch download option from the Favorites page • Provide a mobile-friendly site • Update Space Images iPhone app to include animations (jointly funded) http://photojournal.jpl.nasa.gov/index.html PDSMC March 25-26, 2010

  9. Map-A-Planet (MAP) Status: • New data/capabilities added • Clementine UVVIS derived & binned color (P. Lucey) elemental abundance (FeO, TiO2) • User calculation of Clementine UVVIS elemental abundances, band depth, etc. • MESSENGER/MDIS USGS mosaics • Software and interface improvements • Implemented “custom function” simple math application • Metrics: 122 GB/mo downloaded, ~70000 page hits/mo Plans: • Add planetary map data • Cassini+ global mosaics of Enceladus, Rhea, Tethys, etc. • Cassini VIMS mosaic of Titan (Brown/Barnes) • Derived (Lucey) lunar mineral maps (olivine, pyx, etc.) • Earth-based radar of Moon (B. Campbell) • THEMIS IR global Mars mosaics (ASU) • Merged Clementine and Lunar Orbiter ‘albedo’ image (Hare) • Moon Mineralogy Mapper mosaics as available • Software and interface improvements • Implement Polar Stereographic and Equidistant image views • Allow user definition of map area using center lat/long and bounding box http://www.mapaplanet.org/ PDSMC March 25-26, 2010

  10. UPC Database/PILOT Web Interface Status: • Data Added • Cassini ISS for Saturnian satellite surfaces (now being tested) • Viking Orbiter 1 & 2 for Mars • Database holdings before additions: • ~1.5 M images • MGS/MOC NA & WA, MO/THEMIS-IR, MRO/HiRISE & CTX, Clementine/UVVIS & NIR • Moved db to new compute cluster to speed ingestion and improve access times • Metrics: 199 MB/month downloaded, 456 page hits/mo Plans: • Automate ingestion of data into the UPC • Leverage data integrity processing and db to identify new files to add • Use the MOAB clustering software to schedule and manage work • Add new data: • Lunar Orbiter III, IV, Vl; LROC NAC, MESSENGER MDIS • Push-frame imagers: • MRO MARCI, THEMIS VIS, LROC WAC • Cassini VIMS • Mars Express HRSC • Clementine HiRes, LWIR • Mariner 9, 10 http://pilot.wr.usgs.gov/ PDSMC March 25-26, 2010

  11. Mission Interface: Data Nodes, Etc. • HiRISE Data Node (UofA, Tucson, AZ) • New 5-year proposal, new 2-yr Extended Mission (to late FY12) • ISIS3 support for JPEG2000 image format, I/O (funded by other projects at USGS) • THEMIS Data Node (ASU, Tempe, AZ) • Transition Plan currently in draft form • LROC Data Node (ASU, Tempe, AZ) • Extensive testing and support for 3/15/10 data release (~40 TB of LROC data!) • ~120 TB of data storage hardware in-hand (56 TB in FY10) and planned for purchase (64 TB in FY11) for one copy • SDSC ‘deep archive’ transfer • 27 TB NAS shipped 3/11/10 • ~25 TB of HiRISE RDRs, v.1 (superseded data) PDSMC March 25-26, 2010

  12. Lunar Orbiter Digitized Film Archive (USGS) Chris Isbell is IMG POC Archive design & draft volumes in place online Constructed frames (primary product) as PDS image files (w/ PDS labels) Digitized film strips, geojpegs, browse images now online Draft ancillary products and documentation available Peer review planned for April/May, 2010 ~750 GB volume Lunar Orbiter Image Recovery Project (ARC) Rafael Alanis is IMG POC ~10 TB expected: 7.5 TB for high-res, 2.5 TB for medium-res images Several telecons with Dennis Wingo RE PDS labels and image formats Apollo Digitized Film Archive, Metric (ASU) Patty Garcia is IMG POC Have one (non-archive) copy of Apollo Metric camera data (ISIS3 cubes, ~30 TB) from ASU at USGS Will soon transfer original raw scan TIFF files from ASU to Flagstaff (awaiting NAS) Drafted archive plan ~75 TB expected Restorations (1/2) PDSMC March 25-26, 2010 12

  13. Apollo Sample Image Archive (JSC) Patty Garcia is IMG POC First delivery of raw scanned data to USGS(~3 TB) Continued to work with JSC personnel to define archive products ~25 TB expected Kaguya/SELENE (JAXA) Lisa Gaddis is IMG POC Plans for improved/more detailed user instructions for data from LISM instrument package Multiband Imager, Spectral Profiler, Terrain Camera One copy, ‘safed’ data on NAS ~100 TB by FY11 Yukio Yamamoto is DARTS (online archive) contact Restorations (2/2) PDSMC March 25-26, 2010

  14. LPSC, Mar 2010 Alice Stanboli, Patty Garcia, Chris Isbell, Scott Akins Abstract & poster presented on MAP status Slideshows and demos of Atlas, MAP, UPC, etc. ~30 mins talking to Kristen Erickson of NASA HQ NLSI 3rd Lunar Forum, July 2010 Possible, not yet planned Education and Outreach IMG and GEO booths at LPSC 2010 PDSMC March 25-26, 2010

  15. Extra Slides Follow PDSMC March 25-26, 2010

  16. Data Collections: Hardware at IMG March 2010 • IMG hardware and computing infrastructure, guided by industry best practices and PDS requirements, is secure and scalable • Servers are redundant and configured as failover clusters • RAID-5 & RAID-6 storage provides failover at disk and node level • Systems at USGS and JPL provide ‘hot backup’ to each other, ensuring continuity of operations • IMG hardware capacity will need to increase ~4 to 8X this volume within 5 years PDSMC March 25-26, 2010

  17. Data Collections: IMG Data Volume • IMG, PDS, and NSSDC will soon be the caretakers of an unprecedented volume of digital data! • Requires significant expense for hardware, maintenance, data integrity, system monitoring, etc. • FY10 Cost of 1 TB of redundant online storage and assoc network = $750 (initial purchase) • FY10-12 archive growth of ~100 TB/yr will require purchase of redundant storage at both USGS and JPL, at an estimated Total Cost $150K/yr Archive Elements (Est.): • FY09: 73 TB (HR, TH, All IMG) • FY10: 170 TB (HR, TH, LRO, Ap) • FY11: 375 TB (HR, TH, LRO, Ap, K) • FY12: 478 TB (HR, LRO, Ap) • FY13: 500 TB (LRO, Ap) • FY14: 512 TB (LRO) Note:Includes only primary volumes at USGS and JPL. DN storage, partner copies, and deep archives at NSSDC and SDSC will ensure 3-copy redundancy. PDSMC March 25-26, 2010

  18. Lisa Gaddis, PI USGS Sub-Node Sue LaVoie, Co-I JPL Sub-Node USGS Technical Leads: Patty Garcia: Mission Interface Chris Isbell: Data Delivery Bob Sucharski: Collections JPL Technical Manager: Alice Stanboli S. Akins M. Bailen J. Barrett K. Becker P. Geissler R. Raub • R. Alanis K. Boggs • J. Diehl A. Leung • J. Padams L. Plesea • E. Rye THEMIS Data Node (ASU): Philip Christensen, PI Kimm Murray, Manager LROC Data Node (ASU): Mark Robinson, PI Ernest Bowman-Cisneros, Manager HiRISE Data Node (UA): Alfred McEwen, PI Rod Heyd, Manager Organization March 2010 PDSMC March 25-26, 2010

  19. Magellan: retrieve single-look images Not an archive product; only exist on safed SAR-EDRs readable by VEXCEL-created UNIX SAR Processor Software Attempting to recompile software on new systems Mariner 9: assist in access to SDDPT version of EDRs Desire to incorporate into ASU’s JMARS Provided access to work-in-progress PDS-compliant products and documentation Expert Assistance: User Calls for Help PDSMC March 25-26, 2010

More Related