1 / 35

Lisa Gaddis (USGS) and Sue LaVoie (JPL)

PDS Imaging Node Status Mission Interface, Archive and Delivery Services PDSMC April 3 & 4, 2008 Washington, DC. Lisa Gaddis (USGS) and Sue LaVoie (JPL). Overview: Imaging Node (IMG) Activities (FY08+). Mission Interface Data Nodes : THEMIS (ASU), HiRISE (UA), LROC (ASU)

anais
Download Presentation

Lisa Gaddis (USGS) and Sue LaVoie (JPL)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PDS Imaging Node StatusMission Interface, Archive and Delivery ServicesPDSMC April 3 & 4, 2008Washington, DC Lisa Gaddis (USGS) and Sue LaVoie (JPL) PDSMC April 3, 2008

  2. Overview: Imaging Node (IMG) Activities (FY08+) • Mission Interface • Data Nodes: THEMIS (ASU), HiRISE (UA), LROC (ASU) • Cassini (ISS, VIMS, RADAR) • Mars Exploration Rovers (Cameras – Descent, Haz, MicroImager, Nav, Pan) • Mars Global Surveyor (MOC) • Mars Reconnaissance Orbiter (HiRISE, CTX, MARCI, ONC) • Messenger (MDIS) • Mars Express (HRSC) • Phoenix (MECA-OM, MECA-AFM, RAT, SSI, ‘MARDI’) • Chandrayaan-1 Moon Mineralogy Mapper (M3) • Lunar Reconnaissance Orbiter (LROC, LAMP) • LCROSS • Mars Science Laboratory (MASTCAM, MAHLI, MARDI) • Systems: Online Systems & Support Software • Online access to planetary image data (~9 TB JPL, ~7 TB USGS, ~5 TB ASU, ~18 TB UA = ~39 TB) • Atlas II, Map-a-Planet, Photojournal (~2 TB/mo of MER, Cassini, MRO) • Deliver improved ancillary data (pointing, position, geometry) • Unified Planetary Coordinates (UPC) Database • Baseline Services • Provide expert assistance to PDS and users for cartographic and scientific data analysis • Restorations include Lunar Orbiter, Apollo cameras • PDS ‘Community Service’ • Serve as PDS representative on NASA ‘cartography’ working groups PDSMC April 3, 2008

  3. Data Delivery Services • IMG Web site • Enhanced automation for faster & easier updates • Continued sprucing per advice from Mark Rose (Ames) and users • Mission-specific interface points to online image data and other Node collections • Map-a-Planet • Web Map Services (WMS) interface delivers derived PDS products • New: (Moon) Clementine NIR, Lunar Orbiter, Lunar Prospector; (Mars) MGS TES albedo and thermal inertia; (Jupiter/Io) Galileo SSI color • Atlas II (aka ‘New’ Planetary Image Atlas) • Includes image data from all past and current planetary missions • Updated monthly with Cassini, THEMIS, HiRISE, MER, etc. • Single- and multi-mission searches and bulk download • Map-based searches now being implemented (Google Toolkit) • Will use improved geometry data from UPC • Unified Planetary Coordinates (UPC) Database • Delivers image metadata with unified coordinates and improved geometry • Geometry ‘backbone’ for Atlas II, ODE (GEO), ISIS, (ILIADS?) etc. PDSMC April 3, 2008

  4. Map-A-Planet • Capabilities and Features: • Delivers derived PDS map products • Viking Orbiter Mars Digital Image Mosaic (MDIM) v. 2.1 • MGS/MOC WA (256 p/deg), MOLA (128 p/deg) • Clementine UVVIS and NIR mosaics (11 bands) • Uses WMS to make maps of spatially referenced data from geographic info in labels • Multiple output file formats • GIS ‘world file’ and common image formats • ISIS detached labels as input and output • Multiple image projections • Simple cylindrical • Mercator • Sinusoidal • Polar Stereographic (implementation in progress) • 32-bit output formats (new!) • MGS/TES albedo and thermal inertia data • Products linked to NASA Nomenclature database and USGS planetary maps See http://mapaplanet.org OGC WMS – Open Geospatial Consortium Web Map Service(s) PDSMC April 3, 2008

  5. MAP: In Development • Add option for full image download • Add JPEG2000 format for output files • Uses OGC Geospatial Data Abstraction Library (GDAL) for data translation from PDS • Smaller file sizes for large image products and mosaics • HiRISE and LROC • Output ISIS formatted files from PDS input • Create ISIS detached label to augment PDS label for products • Part of USGS ‘integrated Web services’ plan to simplify access to PDS products and allow ‘on-the-fly’ processing • Will directly link PDS and ISIS products to a variety of desired output formats • Uses OGC-compliant s/w and data formats to ensure interoperability • New data added (or soon to be added): • Mars MOLA (128 pixels/deg) global mosaic (highest available resolution) • Mars MOC (256 pix/deg) WA global mosaic • Mars TES global derived data (albedo, thermal inertia @ 8 pix/deg) • Lunar Prospector derived Fe, Ti, Th, H, etc. (5 and/or 0.5 deg) • Clementine NIR 6-band global mosaic • Lunar Orbiter photographic global mosaic (512 pix/deg) • Merged Galileo SSI and Voyager color mosaic for Io See http://mapaplanet.org PDSMC April 3, 2008

  6. Atlas II • Atlas II has 4 distinct layers and functions • Data Access via Web Service (uses OODT) • Performs searches across all data sets in the system • Handles semantics translation (e.g., East or West longitude, SCET vs FDS) then searches metadata from repository’s OODT Profile Server or SQL database • User Interface via ‘Forms’ • The human-interface part that allows tailored requests • Processing Capability • Uses JPEG2000 software and server • Facilitates HiRISE and LROC deliveries • Retrieval (uses OODT) • Data download via OODT Product server or ‘wget’ to Web Service • Shopping-cart style bulk data ordering • Allows geographic data searches OODT - Object Oriented Data Technology See: http://pdsimg.jpl.nasa.gov/Missions/index.html PDSMC April 3, 2008

  7. Atlas II Status and Plans Status: • New missions supported • Cassini, MER, MGS, Messenger, MRO (HiRISE, CTX, and MARCI) now operational • Support for both mission-specific and multi-mission queries • The same code base is used for both • Common parameters of each logical grouping (data set) are dynamically found and displayed and searched across missions Plans: • System modifications • Migrate meta-data from Sybase to MySQL to increase performance • Schema optimization • Configuration file to integrate with Atlas II and OODT • Add data processing capability (calibration, mosaicking, geometric processing, etc.) • Map-based searches • Integration with UPC • Data migration from Atlas I web-interface to Atlas II • FY2008: Voyager, Viking Orbiter, MPF, Clementine, Magellan See: http://pdsimg.jpl.nasa.gov/Missions/index.html PDSMC April 3, 2008

  8. Unified Planetary Coordinates (UPC) Database • A geometry database system to unify coordinate systems for planetary image data • Designed in part to resolve a problem with coordinate system disparities in older planetary image data (esp. for Mars) • Provides significantly improved image positional and geometric information • Calculates geometric parameters incrementally for each image • Uses 3-D surface to project images where possible { What were they thinking? Planetographic – Angle between the equatorial plane and a line that is normal to the reference ellipsoid Planetocentric – Angle between the equatorial plane and a line from the center of the ellipsoid PDSMC April 3, 2008

  9. Corner Footprint UPC Database • A ‘geometry database’ for PDS data • Reads position & pointing from PDS image labels • Designed to be updated and expanded as needed • Converts to unified coordinates • Planetocentric latitudes • 0 to 360 degree positive east longitudes • Mars/MGS: IAU 2000 Mars Coordinates • Calculates ‘improved’ pointing and location data • Corner, edge positions every X lines, samples • Images projected onto 3-D surface model where possible • Why? • Improves data search & retrieval (geographic, feature-based) for PDS image data • Enhances user ability to identify, combine, correlate and analyze cross-mission PDS data • Can easily be used by Atlas II, ODE and other search tools within and outside PDS (OGC-compatible) • Used to identify overlapping image sets to seed control point and match point files • Will significantly enhance speed and quality of production of image mosaics in ISIS from data such as HiRISE and LROC UPC Footprint Example: MOC WA Footprint PDSMC April 3, 2008

  10. UPC Status • Database Population (Phase II Development) • MGS/MOC WA, NA population complete (237,842 images) • Clementine UVVIS and NIR in progress (957,144 images) • Upgrades to hardware now underway to improve through-put time • Others staged for population • MRO/CTX (5,020 images) • MRO/HiRISE (157,727 images) • Viking (63,015 images) • THEMIS IR and VIS now undergoing camera model improvement in ISIS • To be added to UPC when completed (Aug 08?) • Lunar Orbiter in progress • Camera model (ISIS3) and PDS-compliant label being developed • Preliminary version of UPC released to JPL (IMG) and Wash U (GEO) • MGS/MOC WA and NA database • Workshop held at USGS on March 27, 2008 • MapServer interface operational • Includes WMS, WFS and Raster support • Allows users to specify desired acquisition parameters • Displays detailed image footprint and basic information • Now being tested with live, direct access to UPC database PDSMC April 3, 2008

  11. UPC Footprints: MGS/MOC UPC MOC Cumindex MOC guinness@wunder.wustl.edu Offset is ~20 km 96 lon E, -36.5 lat PDSMC April 3, 2008 11

  12. Atlas II and UPC Integration PDSMC April 3, 2008 12

  13. Baseline Services: Restorations • Restoration Activities • Clementine Near-Infrared (NIR) • Peer Review complete for Release V. 01 of Clementine near-infrared 6-band lunar mosaic (100 m/pixel) • Complements UVVIS cube, completes full 11-band Clementine multispectral dataset • Final release of V 1.0 to PDS expected May 1, 2008 • Apollo Photo Digitization (Metric, Panoramic, Handheld Cameras) • USGS collaborators working with PI Robinson to evaluate scanned products to ensure quality and usability • 80 TB of data expected! • Working with ASU to create mission, instrument, and data catalog files for PDS archive for Apollo image data • Lunar Orbiter Digital Data Archive • Developed tiling scheme and labels for LO global mosaic and projected frames • Working to create mission, instrument, and data catalog files for PDS archive of LO data • Viking Orbiter (VIS) • Decompressed the full collection and loaded the database with the resultant metadata for AtlasII searches. • AtlasII configuration and release pending • Support for Lunar Data Providers • Support data providers (as needed) who respond(ed) to NASA PMDAP and/or LASER announcement of opportunity PDSMC April 3, 2008

  14. Baseline Services: PDS-Wide • Standards and ‘PDS2010’ • Participating in ongoing testing of Label Validation Tools (Garcia, Isbell) • Participating in Physical Media Working Group ‘Tiger Team’ (Isbell) • Participated in ‘PDS2010’ ‘Data/Information Model’ and ‘User’ Tiger Teams (Isbell, Gaddis, et al.) • PDS Community Service • Represented PDS at LPSC meeting in March 2008 • Presented abstract and poster on PDS lunar data holdings in IMG (Garcia) • Representing PDS on NASA Working Groups (Gaddis) • Goal is to clarify and/or develop recommendations for standardization of cartographic products from PDS data • Lunar Mapping Project (ESMD; T. Lavoie, MSFC) • Lunar Geodesy and Cartography Working Group (ESMD; Archinal, Chair) • Geologic Mapping Sub-Group of the PGG Cartography and Geologic Mapping Working Group (L. Bleamaster, Chair) • ‘Planetary Cartography’ Working Group? (B. Bussey, Chair?)----TBD PDSMC April 3, 2008

  15. THEMIS DATA NODE (TDN) Principal Investigator: Phil Christensen Location: Arizona State University (ASU), Tempe, AZ Operations In full operational status since 2001 Data Holdings ~5 TB now available to community Full copy of data online at USGS (updated quarterly), served via Atlas II Data Releases Four per year (#21 to #24 in FY08); last in Jan 2008, next in April 2008 (this week!) Data Access ‘Map Tool’ Image Browser shows coverage for MO/THEMIS, VO, MGS/MOC NA, MRO/CTX & HiRISE JMARS (Java-based) is mission planning tool for data visualization and integration THMPROC Web site provides customized, processed THEMIS VIS and IR image products ISIS geometric processing s/w used (now undergoing refinement at USGS) All of these tools will benefit from UPC database Data Nodes PDSMC April 3, 2008 15

  16. Data Nodes HiRISE DATA NODE (HDN) • Principal Investigator: Alfred McEwen • Location: University of Arizona, Tuscon, AZ • Operations • Now in full operational status, Eric Eliason is manager • MRO Extended Mission proposed to NASA (1 Mars Year, FY10 and FY11), ICD now in revision • Transition to PDS of data holdings will be facilitated by MRO/HiRISE data mission holdings in place at USGS • Data Holdings • 18,500 gigabytes (18 TB) now available to community • Data Access • JPEG2000 JPIP server allows efficient browse/roam/pan/zoom, used in the HDN and via the AtlasII • HiRISE Web Site (http://hirise.lpl.arizona.edu; tailored for HiRISE data products) and AtlasII (for generic search capability) provide access to all released products • Data also served by JMARS (ASU) and ODE (GEO/WashU) • Data Release • MRO project data release #1 in June 2007, #2 in September 2007, #3 in December 2007, #4 in March 2008 • Color RDR products delivered October 2008 • ~5 Months ahead of schedule! • Establishes procedures and guidelines that will benefit Data Node for LRO Camera PDSMC April 3, 2008

  17. LRO Camera (LROC) DATA NODE (LDN) Principal Investigator: Mark Robinson Location: Arizona State University (ASU), Tempe, AZ Operations Plan modeled after HDN Operations, Ernest Bowman-Cisneros is manager Transition to PDS of data holdings will be facilitated by LRO/LROC data mission holdings in place at USGS Data Holdings LROC EDR Product SIS in review (starts this week!) LROC RDR Product SIS in development (near final), review will be combined with EDR review Up to ~60 TB planned for Exploration Phase NAC EDR/CDR – 36 TB WAC EDR/CDR – 24 TB RDRs – 3 TB Data Access Web access, map-based data searches, parameter search, direct browse capability See site for Apollo data, for example: http://apollo.sese.asu.edu/ Data Release Quarterly deliveries planned (higher frequency expected after first release) LRO to transition from NASA Exploration Systems to Science after Yr 1 Data Nodes PDSMC April 3, 2008 17

  18. Extra Slides Follow PDSMC April 3, 2008

  19. PDS IMG: Sub-Node Services Breakdown (FY08+) PDSMC April 3, 2008

  20. What is the UPC? • Starts from but does not replace the many existing instrument-specific databases • UPC Ingestion Pipeline uses ‘Conductor’ software developed at UA for MRO/HiRISE • Process scalable to unlimited number of processing systems (now using 8 Linux and Mac OS X machines) • Uses local and/or remote (retrieved via wget) PDS-formatted source files • Contains pre-computed geometry information for images, based on • ISIS and NAIF geometry subroutines • Best SPICE data available (including adjusted SPICE from geodetic solutions) • Single set of planet, coordinate, and other relevant definition files • The UPC can and will be recomputed • To incorporate new images or new SPICE • To change the definition of useful data (ranges of i, , , and resolution) • To include additional search variables such as optical depth • Users can specify desired ranges for • Latitude, longitude, elevation or Cartesian coordinates (x, y, z) • Resolution • Incidence, emission, and phase angles • Instrument, filter, etc. • Time-of-day • Other variables defined by user PDSMC April 3, 2008

  21. What is the UPC? (2) • Three areas of functionality • Availability and management of SPICE data • Four types and/or sources • Predicts • Nadir • Reconstructed • C-Smith-ed (“refined” SPICE products) • Generation of database for all instruments • Starts with best available SPICE data • Requires support in ISIS 3 • Primarily a Camera Model • Adaptable to mission diversity and peculiarities • Numerous ingestion sources • PDS labels (MER, Viking) • Cumulative index tables (MOC) • ISIS 2 & 3? • Others • Diverse DB query tools • Applicable to different needs • Data processing, auto-seeding, mosaicking, cartography • PDS data searches • MGS/MOC dataset is one of the most improved PDSMC April 3, 2008

  22. The UPC Database • Builds on SPICE database system derived from original mission kernels • Candidate images initially identified by UPC • Contents and/or paramaters designed to be refined and updated periodically in ISIS system • Results can be provided back to user community in the form of new SPICE kernels for included images • Takes advantage of best available DEMs and SPICE kernels as well as advances in ISIS computing capabilities and techniques • Full functionality and power of UPC developed and distributed in the ISIS system to science community PDSMC April 3, 2008

  23. Why Use the UPC? • External users benefit from: • Availability of image data via web interface • Compatibility with web GIS standards • ISIS public availability exports most internal capabilities to scientific community • Access to a growing library of data products • Access to most recent SPICE/Control Nets • Anticipate a large number of high-quality community-generated products • Provide system and/or data to other institutions like the PDS PDSMC April 3, 2008

  24. Footprint Example: HiRISE Corner Footprint UPC Footprint Original UPC PSP_002158_2035_RED4_0 PDSMC April 3, 2008

  25. Corner Footprint UPC Footprint Footprint Example: MOC-WA ab102402 PDSMC April 3, 2008

  26. UPC GIS Capabilities • UPC provides binary predicates (or Geometry Relationships) using GEOS (JTS) functions and extensions in the database backend (PostGIS) • Binary predicate operations take two geometries (e.g., MULTIPOLYGON) and returns a Boolean value indicating if the geometric relationship between them exists • Examples: • Equals(geometry,geometry) • Intersects(geometry,geometry) • Contains(geometry,geometry) PDSMC April 3, 2008

  27. Point MultiPoint Polygon MultiPolygon GeometryCollection LineString LinearRing MultiLineString UPC GIS Geometries PDSMC April 3, 2008

  28. Intersection Union Difference Intersection of LineString and Polygon Symmetric Difference GIS Analysis Methods PDSMC April 3, 2008

  29. GIS Predicate Example • This example shows results of LineString and Polygon predicate functions PDSMC April 3, 2008

  30. UPC Population Process • Initial population of the UPC is compute intensive • A compute cluster of sorts is utilized to distribute processing tasks • The process of downloading, computing, collecting and inserting the data into the UPC is called a pipeline • Each process within the pipeline is called a segment of the pipeline PDSMC April 3, 2008

  31. UPC Population (2) • The pipeline is developed using the Conductor software developed by the Planetary Image Research Laboratory (PIRL) at the University of Arizona • http://pirlwww.lpl.arizona.edu/software/Conductor.shtml • Conductor is a Java application for managing queues of source files to be processed by sequences of procedures • The queues for each segment of the pipeline are actually a set of two (MySQL) database tables • One table contains the source file, the second contains the procedures required to process each source file completely through the segment • One or more segments may exist in a single pipeline • The last procedure in a segment passes the output source to the next segment in the pipeline PDSMC April 3, 2008

  32. UPC Population (3) • Segments of a pipeline are run independently from one another on each compute node in the cluster • Segments are distributed across one or many compute nodes • One or more copies of the same pipeline segment may be run on a single node • This system is scalable within nodes and across many platforms • We are currently run up to 12 different “nodes” around the clock • These nodes are both Linux and Mac OS X systems PDSMC April 3, 2008

  33. UPC Population (4) • The UPC pipeline is something of a hybrid • It has 4 segments for each instrument in the UPC database • Typically each instrument has its own Download and ISIS Ingestion segment • All instrument data is processed through the GeomProc (geometric data) and Publish (database insert) segments PDSMC April 3, 2008

  34. MOCNA DownloadMOC MOCWA THEMISIR DownloadTHEMIS THEMISVIS GeomProc Publish DownloadHiRISE HIRISE DownloadCTX CTX DownloadCLEM CLEMENTINE UPC Population (5) PDSMC April 3, 2008

  35. UPC Population (6) • Segment design has several criteria • Complexity of instrument • Some PDS EDRs do not have INSTRUMENT_ID • Number of images in instrument dataset • MOC has ~238,000 images, 96,618 of them are NA • Clementine has ~1.8 million • Ingestion into ISIS differs by instrument • SPICE initialization may also differ • Geometric processing and database publication (insertion) is consistant or sufficiently generalized PDSMC April 3, 2008

More Related