RDA Survey 2005: User Insights and Data Needs Analysis

1. RDA Survey 2005 CISL Executive Meeting 1 August 2006

2. Outline • Basics • General Quantitative Summary • Response to Three Questions in more Detail • Who is using what from the RDA? • Assessment of Open-ended Questions • Mapping to Work Underway and Futures • How to gain more from user surveys

3. Basics • User pool • Registrants on RDA online server • No restrictions on who can register • Must register to get data - general information (metadata) does not require registration. • Year group 2005 only, 3914 invited to survey • Respondents • 476, about 12% • Open period, 45 days (4/276/12)

4. Basics • Divide user community by affiliation • Enables response evaluation for separate groups • Observations and Assessments • ~ 60% are Non-U.S users • + international hub for data • + supports policy of ‘free and open access’ • + leverage in forming international data exchanges • use resources (hardware and network) to serve world 180 98 71 38 30 22

5. Basics • Seven questions with fixed responses • Choose one of five levels of satisfaction • Very Dissatisfied,Dissatisfied,Neutral,Satisfied,Very Satisfied • Simple assessment, add (N + S + VS), % satisfied 97%, Content of the RDA for your research (410) 95%, Accessing RDA data from the DSS server using FTP or web browsers (405) 90%, Accessing RDA data directly from the NCAR Mass Storage System * (215) 96%, RDA metadata, documentation, and supporting information from the DSS server (369) 94%, Data specifically prepared for you by the DSS staff, including FTP downloads, tapes, and other media (227) 95%, Consultation and assistance from the DSS staff (224) 96%, Discovering RDA data using search, catalogues, and project information from the DSS server (353) (nnn) = total respondents * Potentially some bias?

6. Observations and assessment • single strong mode - S • would like to see stronger VS, especially for US University

7. Observations and assessment • good strength in S and VS, users like simple access • not much variability between affiliations, good US University approval

8. Observations and assessment • good strength, and not much difference between S and VS • high variability between affiliations in VS, good US University approval

9. Who is using what? • Observations and assessment • generally users tap the RDA for multiple types of data • good balance in the major categories, led by reanalyses

10. Assessment of Open-ended Questions • Three questions - generated ~ 50 pages of text • Many positive comments, good, - affirms our efforts • But, we want more! • assess the comments that had suggestions, collectively, map to DSS developments underway and determine relevant future directions • Process • Four staff and SJW read, digested, and discussed - look for the bigger message - very subjective • Didn’t over analyze - many interpretation are possible

11. Assessment of Open-ended Questions Question 1 • List your preferred methods for discovering data (e.g. Google search, local site search, catalogs, etc). • Weak, leading question • All modes are preferred, plus, directives from colleagues, and discovery from other websites • Observations and actions • Users can now use the UCAR Google search from the RDA website, since March 2005 • Restricted to RDA results • Need to strengthen ‘meta tags’ on RDA documents • Some are not finding data that we do have (comments and small VS on bar chart) • Well into a metadata project with features - 18 months • Tightly constrained standard - GCMD with mapping to CF • Database (MySQL) stored and accessible • Enables fast and accurate browse and search • Allows progressive deployment on a large archive • Roll out in 2006

12. Assessment of Open-ended Questions Question 2 • List any datasets your projects need that should be added to the RDA. • Higher spatial and temporal resolution - everything! • More near real-time • Atmospheric • NEXRAD precipitation mosaics for CONUS • High resolution global forecast winds • Chemical components, aerosols • Time series indices (AO, NAO, etc) • Ocean • Reanalyses • Weekly SST, monthly SSS • Ocean bottom composition • Sea ice (thickness, velocity) • Satellite • TMI, TRMM (precipitation) • Jason-ERS-TOPEX • GOES Images • Hydrology • River flow • Soil moisture • Land surface, vegetation

13. Assessment of Open-ended Questions Actions • Added NCEP Global Ocean Data Assimilation System Reanalysis, 1980 -> • Will added higher resolution atmospheric datasets (analyses and forecasts) • TIGGE • Improvements to IDD derived archive (real-time data delivery) Current (161 MB/day, 50 GB online) ModelCoverageResolution Init timeLength/IntervalSize RUC CONUS 81x81km Hourly 12h/3h 53mb/day NAM CONUS 81x81km 00,12 60h/12h 23mb/day GFS N Hem. 381x381km 00,06,12,18 120h/6h 11mb/day GFS Global 5x5deg 00 240h/12h .19mb/day ENS N. Amer. 1.5x1.5deg 00/12 84h/6h 69mb/day ECMWF Global 5x5deg 00/12 168h/24h 5.2mb/day New (17 GB/day, 500 GB online) ModelCoverageResolutionInit timeLength/IntervalSize RUC CONUS 40x40km Hourly 12h/1h 490mb/day RUC2 CONUS 20x20km Hourly 12h/3h 3.4GB/day NAM CONUS 12x12km 00,06,12,18 84h/3h 2.0GB/day GFS Global 0.5x0.5deg 00,06,12,18 84h/3h 4.4GB/day GFS Global 1.0x1.0deg 00,06,12,18 180h/3h 6.5GB/day GFS Global 2.5x2.5deg 00,06,12,18 192h-384h/12h 300mb/day ECMWF Global 5x5deg 00/12 168h/24h 5.2mb/day

14. Assessment of Open-ended Questions Other Actions Discovery: From improved metadata, create ‘better’ topic specific catalogs • Example, complete prioritize reference catalog for RDA precipitation datasets • Related references at other sites, e.g. TRMM Satellite data at NASA • Potentially, share catalogs via OAI-PMH (Open Archives Initiative Protocol for Metadata Harvesting) Generally, put more data online • FNL Analysis (very popular with MMM and WRF users) • Currently only one-year, boost to full record - starting in 1999 • Others - when action supports many users and helps DSS Stay proactive in building content • Status quo (search out additions, collaborate on data projects, etc) • Open an online user suggestion box?

15. Assessment of Open-ended Questions Question 3 • Is your science limited by the research data available? If yes, what data or services are needed? • Issues that confuse this user group • Data access methods • RDA server, MSS, special requests, SCD data access computing accounts, CDP • Cost for data • Much free, some not, various legacy policies, differences across affiliation • Site navigation Solution (in part), reduce fragmentation of the information across the web site, create clear concise documents in a single place, bring navigation to modern look and standard.

16. Assessment of Open-ended Questions Question 3, continued • Is your science limited by the research data available? If yes, what data or services are needed? Wish List • All data available in ASCII and netCDF • More direct linkages with field program data - need better UCAR-wide discovery (CDP). • Easy access to older datasets, which are generally not online • Direct access to the MSS • Formats that can directly couple to regional and global models

17. How to gain more from user surveys • Transition to surveys designed by professionals • RDA questions are too ad hoc • There is much speculation in the interpretation • The discussions are interesting but summary is very subjective • Want metrics for two subjects • Past usage satisfaction • Quantifiable clues for the future • Need to apply correct statistical tests • Account for sample size • Determine confidence intervals and/or thresholds

18. Summary • Very pleasing overall results for the RDA • Our developments are aligned to address the bigger issues, in most cases • Other suggestions give us awareness for future work • The RDA is a node of international data collaboration • Certainly will not run short of tasks to work on in my life time.

19. Some people like us! Underlines by SJW • I have been extremely happy with the data and services provided. Your organization is an efficient, friendly, affordable and responsive alterative to NCDC. I have absolutely no complaints. Keep up the good work • Very good and friendly service. I consider year back global free data as a big achievement especially if to compare with ECMWF or with NZ meteorological data services. It's very helpful for small research groups all over the world. Good luck • Great selection of browsable datasets. Have used this over the past few years to locate hard-to-find data elsewhere • I am very happy with the service and also the response from the staff. At one occasion…... I got a very prompt and helpful response. I hope such an excellent service can continue to be supported • A very good free service that we in Europe could not do without. I only wish that our own institution provided the same service • The US philosophy of open access is very commendable. Good work! • Absolutely no complaints a great service and a great benefit to the wind industry. well done! • Very good work. I am satisfied with the service and the data and consider that the services are highly needed by the scientific community. Thanks