290 likes | 432 Views
GES DISC Services. Push Harder? Be Careful? Change Direction? What about adding ______?. Discovery Services. Mirador Development scaled back to sustaining engineering level External Search (in Test mode TS1) Technically successful, but... Usability-challenged Start and stop date/time
E N D
GES DISC Services Push Harder? Be Careful? Change Direction? What about adding ______?
Discovery Services • Mirador • Development scaled back to sustaining engineering level • External Search (in Test mode TS1) • Technically successful, but... • Usability-challenged • Start and stop date/time • Total number of hits • Uniform sort order • Duplicates • Usability: Simplicity vs. Features (esp. Services) • Mirador Usability Sounding Board? • mail list for queries on usability quandaries
Number of Users* - March 2011 *OK, not really. It’s the number of distinct IP addresses
The quality of AIRS data varies considerably Version 5 Level 2 Standard Retrieval Statistics
Quality Schemes can be complicated Hurricane Ike, viewed by the Atmospheric Infrared Sounder (AIRS) Air Temperatureat 300 mbar PBest : Maximum pressure for which quality value is “Best” in temperature profiles
Current user scenarios... • Nominal scenario • Search for and download data • Locate documentation on handling quality • Read & understand documentation on quality • Write custom routine to filter out bad pixels • Equally likely scenario (especially in user communities not familiar with satellite data) • Search for and download data • Assume that quality has a negligible effect Repeat for each user
The effect of bad qualitydata is often not negligible Hurricane Ike, 9/10/2008 Total Column Precipitable Water Quality Best Good Do Not Use kg/m2
DQSS replaces bad-quality pixels with fill values Original data array (Total column precipitable water) Mask based on user criteria (Quality level < 2) Good quality data pixels retained Output file has the same format and structure as the input file (except for extra mask and original_data fields)
DQSS Status + Plans • Operational for AIRS L2 Standard Retrieval • Nearly operational for MODIS Water Vapor • Next: MODIS Aerosols, MLS Water Vapor • Next: ??? • Also, OPeNDAP Gateway nearly reader to front-end DQSS • Allow OPeNDAP access to DQSS-served data.
OPeNDAP* • Remote access to data: no need to download • Access at fine granularity • Variable • Array regions • Stride • Present HDF data as netCDF/CF • Enhances Tool Usability • Reformatting: ASCII, netCDF *OPeNDAP = OpenSource Project for a Network Data Access Protocol
Who Uses OPeNDAP? • Industrial-strength scripters looking for subsets • Thick client users • GrADS, Panoply, IDV, McIDAS-V, Ferret • Internal Systems • Giovanni • MapServer • Simple Subset Wizard
OGC* Standards - WMS • Web Map Service (WMS) • URL request: returns map image • Implemented with open-source MapServer • Giovanni also supports WMS • Consumers: • AIRS NRT page • Google Earth • GIS programs • IDV • Giovanni *OGC = Open Geospatial Consortium
OGC - WCS • Returns “coverages”: data variables in NetCDF/CF1 • Used by other systems • DataFed • Giovanni • Atmospheric Composition Portal • Simple Subset Wizard
Subsetting • Semi-custom tools for some products • Reuse HSE libraries from UAH • Reuse Lats4D from A. DaSilva • Usually HDF in -> HDF out • Implemented as REST* URLs • Subsetting at time of download • Subsets are implemented as user requests come in • Areas where we should proactively develop subsetters?
~100 Subsettable Datasets • AIRS Radiances (channel), L2 Retrievals (variable), L3 (spatial+variable via SSW) • MLS L2 (spatial+variable) • TOMS L3, OMI L2-L3 (spatial+variable), OMI L2 • TRMM L3 (spatial+variable) • Models (spatial+variable) • Did we miss any (that shouldn’t be missed)? • Should all SSW subsets be offered in Mirador?
Format Conversion • Custom code for some L3 and L2 datasets • HDF -> netCDF/CF • Improves usability in tools • Moving toward external tools where possible • OPeNDAP • Lats4d: based on GrADS
Simple Subset Wizard • Desired: “Just give me the data from time 1 to time 2 for this spatial box”. • Current: “search for granules, view granules, select granules, select subset option, re-enter spatial box...” • ESDIS-funded technology infusion effort • DEMO
G3 Evolution to Agile Giovanni (G4) • Factors driving evolution • G3 architecture was never completed • No workflow engine • Cost of adding significant features is too high • Architecture is too brittle
Key G4 Goals • Reduce cost and time to add new features • Improve performance over G3 • Support external maintenance of external data
Evolution Plan • Implement new projects in Agile Giovanni (G4) • Aerostat ACCESS project • Point data in database, bias corrections • Year of Tropical Convection (YOTC) • Level 2 data • Community-based Giovanni • Externally maintained portals and data • Implement G4 features to meet existing G3 functionality • Migrate G3 instances to G4 portals
Roads Not Taken Not Taken • Giovanni 3 enhancements • ISO 19115 Metadata • Document architecture • Mirador features and usability revamp • Persistent locators • Unique identifiers • Giovanni Evolution • DQSS • Atmospheric Composition Portal • Simple Subset Wizard • Community-based Initiatives • Mirador External Search • Expanding data services
Agile Giovanni Architectural Features • Model-view-controller • Semantic Web underpinnings • Variable-centric, not dataset-centric • Code reuse: Kepler, YUI, JCache, MapServer