Global Precipitation Measurement (GPM)

1. Global Precipitation Measurement (GPM) GV Data Exchange Protocol Mathew Schwaller GPM Formulation Project Ground Validation Manager mathew.r.schwaller@nasa.gov

2. Objectives for this Presentation • Examine the assumptions and issues surrounding a data exchange protocol for international global precipitation measurement (GPM) ground validation • Identify many problems associated with defining and maintaining a standard data exchange protocol • Propose an “applications-based approach” to interoperability among international GPM ground validation sites

3. Some Assumptions Assumptions • Each GV site will have its own unique set of instrumentation optimized for its local users • Not possible or desirable to standardize site instrumentation • Each GV site will have its own data system for archive and distribution • These data systems will be optimized for local users • All GV sites make voluntary contributions of local data and information, with the goal of validating Global Precipitation Measurement instrumentation, algorithms and products • There is no central funding authority (definition of voluntary) • There is no central authority to manage voluntary contributions Issues • There will be a relatively small number of applications that will utilize the data from an international consortium of GV sites • At present, there are no applications that fall into this category • Is it possible for a band of volunteers to adopt a common data exchange format leading to global validation products?

4. Graphic Illustration of the Assumptions application(s) • Each site will have its own unique instrumentation • Each site will have its own unique data system • Only a few shared/global applications are anticipated • Is it possible to adopt common data model, data exchange attributes, formats, protocols, and middleware for generating global validation products? middleware data exchange protocol data system GV site-1 GV site-2 GV site-n instrumentation

5. The Scaling Problem • Coordination costs rise exponentially as partners are added • Any n partners can form up to (2n-1) possible subsets • Specifying GV site attributes, and coordinating shared attributes among varying sites quickly becomes a logistical nightmare! • The more successful you are (the larger the number of GV participants) the time and $$ spent on coordination rises exponentially • If unchecked, data coordination costs could eat into the GV budget for instrumentation, measurement and analysis

6. The Problem of Site Bias • Each site will have its own set of instruments, measurement protocols, calibration procedures, analysis methods and data products • There is no way to coordinate or dictate instrumentation and operations commonality among all GV sites • It should be possible, but may be very difficult, to agree on a common data product: there may be many interpretations and representations of “instantaneous rainfall” for example • Even if common data products are found (e.g., reflectivity), it will be difficult to agree on a common method for measuring and reporting the errors associated with the product • We need to assume that the measurement error of any data product will vary from one GV site to another, and that there will be (unknown) bias from site to site

7. The Problem of Incentive • There are a number of costs that each GV site must consider before entering into an international GV consortium • Data conversion costs: product content and format will likely vary from site to site • Data reliability costs: product measurement error and bias vary at each site, and they may not be well characterized • Utilization costs: no global applications at present that can use the data even if it were available • There is certainly value in generating a globally consistent data set for precipitation validation, but… • The value of participating in a GV consortium must be greater than the cost

8. An Ideal Exchange Protocol • Maximizes incentives for voluntary participation • Provides some value to each participant • Minimizes cost for participation • maximizes re-use of existing resources at local sites • Minimizes coordination costs • Compensates for or otherwise quantifies within-site error and site-to-site bias

9. Typical 3-Tier Architecture • In many data system architectures, applications are separated from data sources by “middleware” • Middleware strengths: • Common interface for applications programmers to access data objects and services • Resolution of location/access information about data objects and services • May provide workflow services for complex tasks • Weaknesses: • Middleware needs to be designed, developed, tested, maintained • “Longer term, the project could still falter if the International Virtual Observatory middleware standards make it too expensive for institutions to prepare their survey data” applications middleware data sources

10. Example from space sciences: the International Virtual Observatory Space Sciences Middleware Applications Data Sources

11. Recommendation • At this stage: focus on the applications, let the protocol follow the applications • Define applications/algorithms for validating global precipitation that are interesting and useful • Implement the application(s) at cooperating partner sites in the international GV consortium • Initially, this will require developing a custom interface for each site • As the number of global GV applications grows (beyond 2 or 3), work on a common data exchange protocol should be reconsidered Replicated application application-A application-A application-A Custom interface for each data source data source-1 data source-2 data source-n

12. Example GV Application • Matching ground and space-based (PR, DPR) data for statistical validation Algorithm resamples coincident ground and space-based radar observations (Bolen and Chandresekar, 2000; Liao et al., 2001) TRMM PR GPM DPR ground radar

13. Prototype in Development • Resampling prototype provides statistical comparison of ground and space radar reflectivity • Good agreement high in the storm (where PR/DPR attenuation effects are minimal) indicates good relative agreement between PR/DPR and the GV radars • Good agreement near the surface indicates that the PR/DPR attenuation-correction algorithms are working well • Can be extended to comparison of precipitation rate, DSD and other variables • Prototype will focus on TRMM PR comparison with NOAA NSR-88D radars • Starting with one or two NSR-88D sites • Will evaluate the possibility of scaling the prototype to all 158 WRS-88D radars • Will evaluate the possibility of scaling to other S-band radars from other US and international sites

14. Conclusions • Focus on global applications that can exploit GV data from an international consortium of providers • These applications will help define the requirements for a GV data protocol • Once the applications requirements are understood a number of possible frameworks for data exchange may be considered, for example: • CEOS/GEOS/CEOP: focus on catalog services • Open Geographic Information System (OGIS): focus on web services • National/International Virtual Observatory (and others): focus on Grid storage and computing • Whatever the framework, the GV data exchange protocol must address practical issues of scalability, site bias, and incentives for participation