American Customer Satisfaction Index (ACSI) Survey Results - 2009

1. American Customer Satisfaction Index (ACSI) Survey Results - 2009 HDF/HDF-EOS Spring Meeting Carol.L.Boquist@nasa.gov 04/01/2010

2. Contents • 2009 EOSDIS Survey Flow • ACSI Responses * • Comments Legend on survey flow charts: Blue boxes designate survey areas White boxes contain rating questions Yellow boxes contain the number of respondents Arrows extending outside blue boxes denote skips Boxes outlined in red or red text denote suggested changes Green outlines and boxes added for HDF/HDF-EOS * Note: The purpose of the survey is to assess user satisfaction with the data and services of the Earth Observing System Data and Information data centers. The survey is conducted by CFI Group under a contract with the Federal Consulting Group (FCG) located within the Department of Interior's National Business Center. The Federal Consulting Group, Executive Agent in government for the ACSI, works with OMB to ensure adherence to applicable Federal regulations such as the Privacy Act and the Paperwork Reduction Act.

3. Measurement Timetable * After a spam issue was identified with receipt of email at one of the data centers, email invitations were resent and data collection was extended.

4. Data Collection 3,842 responses were received 3,842 responses were used for modeling Those who answered for more than one data center: Two: 108 Three: 15 Four: 2

5. User background and interests 3842 • How did you become aware that you could acquire Earth science data from NASA? (select any that apply) • •Colleague • •ECHO Ware House Inventory Tool (WIST)/ EOS Data Gateway • •Global Change Master Directory (GCMD) • •NASA or Data Center Web Site • •NASA Sponsored Research/Data Provider/Affiliate • •Science Conference/Workshop/Meeting • •Scientific Literature • •University (Graduate School, Course work, Classroom, Professor, Lecture, etc.) • •Web Search • Other (please specify) Where are you currently located? drop down list Use ISO 3166, http://www.iso.org/iso/english_country_names_and_code_elements Included USA, “Taiwan, Province of China” changed to “Taiwan” • For which general areas do you need or use Earth science data and services? (select any that apply) • Atmosphere • Cryosphere • Land • Human dimensions • Ocean • Space Geodesy • Calibrated radiance • Other (please specify) 3842 Continue to User background and interests 2 2009 EOSDIS Survey Flow - CLB

6. Satisfaction – within USA vs. outside USA 71% of respondents are outside of the USA in 2009 vs. 68% in 2008. Respondents outside the USA have a slightly lower overall Satisfaction score with EOSDIS (76 outside vs. 77). Compared to last year there was a score increase for those respondents within the USA and a score decrease for those respondents outside the USA.

7. User background and interests (cont.) 3842 • For which areas do you need or use Earth science data and services? (select any that apply) • Agriculture • Air quality • Atmospheric Composition • Carbon Cycle • Climate • Climate Change • Cryosphere – sea ice • Cryosphere – snow • Ecological forecasting • Ecosystems • Land Cover • Land Use • Modeling (please specify) • Natural Disasters • Natural Hazards • Ocean Color • Ocean (sea surface height, sea surface temperature, etc.) • Population • Public Health • Resources (Forestry, Mining, etc.) • Socioeconomics • Solid Earth • Space Geodesy • Space Weather • Sun-Earth Connections • Sustainability • Water resources • Water & energy • Weather • Other (please specify) 3842 Have you done any of the following <from data center>: searched, requested, ordered, visualized, and/or downloaded data or data products? No: Skip to Documentation 167 Yes: Continue to Search 3675 2009 EOSDIS Survey Flow - CLB

8. Respondent Information For which specific areas do you need or use Earth science data and services? Demographics (when comparable) remain fairly consistent with 2008 * Multi-select question; Language to question was changed slightly in 2009; Modeling was asked as a separate question prior to 2008.

9. Search 3675 • How did you search for the data products or services you were seeking? • Data center’s or data-specific specialized search, online holdings or datapool (Please specify, e.g., DADDI, Giovanni, GISMO, GLOVIS, HyDRO, LAADS, Mercury, Mirador, MISR, MIST, NOESIS, OPENDAP, POET, SEADAS, SNOWI, URSA, WebGIS, etc., ,…) • Direct interaction with user services personnel • Global Change Master Directory • Internet search tool (e.g., Google Earth, Google) • Warehouse Inventory Search Tool (WIST)/EOS Data Gateway (EDG) • Other (please specify) • Did not search (skip to Order) 3505 • Were you searching for multiple datasets occurring at the same time or place? • Yes • No Please comment on your experience with the search method you used. Please include whether you need to search for multiple datasets occurring at the same time or place. • Using a 10-point scale, on which “1” means “Poor” and “10” means “Excellent,” please rate the tool … • Ease of finding data • Ease of using search capability • How well the search results met your needs 3505 170 Continue to Order 3675 2009 EOSDIS Survey Flow - CLB

10. Order 3675 3601 • How frequently during a year do you request/order/download data products? If the order is an automatic subscription, please choose the frequency. • Once • Daily • Weekly • Monthly • Quarterly • As needed (please specify approximately how many times numerically) • Did not order (skip to Documentation) • Did you use a subsetting tool as part of the process of requesting/ordering/downloading the data or was a subsetting tool part of the subscription process? • Yes, by band • Yes, by channel • Yes, by geographic area • Yes, by geophysical parameter • Yes, by both geographic area and geophysical parameter • Did not use a subsetting tool • Using a 10-point scale, on which “1” means “Poor” and “10” means “Excellent,” please rate … • Ease of selecting data products • Description of data products • Ease of requesting/ordering data products 3601 Continue to Delivery 74 Skip to Documentation 2009 EOSDIS Survey Flow - CLB

11. Delivery 3601 • How was your data delivered? • FTP immediate retrieval from online holdings • FTP retrieved after order • FTP via subscription • http-based download from Web • http-based batch download from Web (wget) • Web-based visualization tool • Other (Please specify) • How long did it take for you to receive your data products? • Immediate retrieve • Less than an hour • Less than a day • 1-3 days • 4-7 days • More than 7 days • Using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… • Convenience of delivery method • Timeliness of delivery method • Which method of data delivery do you prefer? • FTP immediate retrieval from online holdings • FTP retrieved after order • FTP via subscription • http-based download from Web • http-based batch download from Web (wget) • Web-based visualization tool • Other (Please specify) 3601 Continue to Format 2009 EOSDIS Survey Flow - CLB

12. Delivery Over half said their data came from MODIS (same in 2008) 27% said ASTER (18% in 2008) *Multi-Select =Significant Difference vs. 2008 Impact=0.5

13. Format 3601 • In what format(s) were your data products provided to you? (select any that apply) • HDF-EOS/HDF • NetCDF • Binary • ASCII • GeoTIFF • JPEG, GIF, PNG, TIFF • OGC Web services (WMS, WCS, WFS, etc.) • GIS (e00, shp, etc.) • KML, KMZ • CEOS • Don’t know • Other (please specify and/or comment) • What format(s) would/do you prefer? (select any that apply) • HDF-EOS/HDF • NetCDF • Binary • ASCII • GeoTIFF • JPEG, GIF, PNG, TIFF • OGC Web services (WMS, WCS, WFS, etc.) • GIS (e00, shp, etc.) • CEOS • KML, KMZ • Other (Please specify another format or comment on specific version, etc.) HDF-EOS/HDF: 1686 GeoTIFF: 1707 HDF-EOS/HDF: 2217 GeoTIFF: 1193 HDF-EOS/HDF&GeoTIFF: 596 • Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… • Ease of using the data product in the delivered format • Overall quality of the data product • Overall usability of the data product 3601 Continue to Usage 1 2009 EOSDIS Survey Flow - CLB

14. Format data products were provided Format preferred HDF-EOS/HDF 58% HDF-EOS/HDF 44% GeoTIFF 31% GeoTIFF 44% ASCII 15% ASCII 20% JPEG, GIF, PNG, TIFF 13% GIS 18% Binary 10% JPEG, GIF, PNG, TIFF 17% NetCDF 9% NetCDF 15% GIS 6% Binary 13% Don't know 3% KML, KMZ 9% KML, KMZ 3% Other preferred format 3% CEOS 2% OGC Web services 3% Other format 2% CEOS 2% OGC Web services 1% Number of respondents 3842 Number of respondents 3842 Product Quality In 2008, 74% said products were provided in HDF-EOS and HDF and 41% said they were their preferred method. *Multiple responses allowed

15. Product Quality Improved significantly this year Impact=0.4 =Significant Difference vs. 2008

16. Usage 3601 • What types of data did you get? (select any that apply) • Socioeconomic data • Satellite data • In-situ measurements (field campaigns, land validation, lightning, etc.) • Model data • Other (Please specify) • What instrument data did you get? (select any that apply) • AIRS • Altimetry (QuikScat, JASON, etc.) • AMSR-E • ASTER • CERES (Terra, Aqua, TRMM) • DORIS • GLAS (ICESat) • GNSS • MISR • MODIS (Land) • MODIS (Atmosphere) • MODIS (Ocean) • MOPITT • OMI • PR/TMI/VIRS (TRMM) • SAR (ERS, JERS, RADARSAT, PALSAR) • SLR/LLR • TES • VLBI • Other (Please specify) 3373 228 3601 Continue to Usage 2 2009 EOSDIS Survey Flow - CLB 16

17. Usage (cont.) 3601 • How many people are using or will use the data you received? • 1 • 2-4 • 5 or more • Thinking about your most recent … • Did you use software tool(s) to work with the data (e.g., format conversion, analysis, visualization, etc.?) • Yes (Please specify which tool or tools you used to work with the data.) • No, I couldn’t find what I needed (please specify what you were looking for) • No, I couldn’t understand how to use it (please specify what you were trying to use) • No, didn’t need (Need to add in 2010 survey) • Are you generally finding what you want in terms of type, format, time series, etc.? • Yes • No (Please specify and/or comment on what you want but are not finding.) Y: 3368 N: 233 3601 167 3842 From Did not search/order/../download Did you look for or get documentation? From Searched but did not order 74 No: Skip to Customer Service Yes: Continue to Documentation 1049 2793 2009 EOSDIS Survey Flow - CLB 17

18. Documentation 2793 • What documentation did you use or were you looking for? • Instrument specifications • Science algorithm • Product format • Tools • Science applications • Data product description • Production code • Other • Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate… • Overall quality of the document (i.e., technical level, organization, clarity) • Extent to which the data documentation helped you use the data • Was the documentation • Delivered with the data • Available online • Not found (Skip to Customer Service) 2554 3842 Have you requested assistance from <Data center name>’s user services office during the past year? 239 From did not look for Documentation 1049 No: skip to ACSI 2762 Yes: Continue to Customer Service 1080 2009 EOSDIS Survey Flow - CLB

19. Product Documentation Was the documentation… Delivered with the data (18% vs. 16% in ‘08) Available online (73% vs. 70% in ‘08) Not found (9% vs. 14% in ‘08) What documentation did you use or were you looking for? Data product description 45% Product format 16% Science algorithm 14% Instrument specifications 7% Tools 6% Science applications 6% Production code 1% CSI for those whose documentation was not found is 69 vs. those who got it delivered with the data (78) or online (78). =Significant Difference vs. 2008 Impact=1.0

20. Customer Service 1080 • Was it • By phone • By E-mail • Both by phone and e-mail • Think about the user services staff you interacted with when you requested assistance from <Data center name> user services. On the same scale from 1 to 10 where 1 means “Poor” and 10 means “Excellent,” how would you rate the user services staff on… • Professionalism • Technical knowledge • Accuracy of information provided • Helpfulness in selecting/finding data or products • Helpfulness in correcting a problem • Timeliness of response Continue to ACSI 1080 2009 EOSDIS Survey Flow - CLB

21. Customer Support Did you request assistance from the Data Center’s user services staff during the past year? No=72%. Of those who said yes, 84% used email, 3% used the phone, and 12% used both phone and e-mail. 89% (91% in 2008) were able to get help on first request. These respondents continue to have a significantly higher CSI (81) than those who did not (65). =Significant Difference vs. 2008 Impact=1.6

22. 3 ACSI standard Outcomes 1080 Skipped Customer Service 2762 Using a 10-point scale on where “1” means “Not at all likely” and “10” means “Very likely,” how likely are you to recommend <Data center name> to a colleague? 3842 Using a 10-point scale on which 1 means “Very Dissatisfied” and 10 means “Very Satisfied,” how satisfied are you with the data products and services provided by <Data center name>? Using a 10-point scale, on which “1” means “Not at all likely” and “10” means “Very likely,” how likely are you to use the services provided by <Data center name> in the future? Using a 10-point scale on which 1 now means “Falls short of your expectations” and 10 means “Exceeds your expectations,” to what extent have the data products and services provided by <Data center name> fallen short of or exceeded your expectations 3842 Continue to Reported a problem Now, imagine an ideal provider of scientific data products and services. How close does <Data center name> come to that ideal organization you just imagined? Please use a 10- point scale on which 1 means “Not at all close to the ideal,” and 10 means “Very close to the ideal.” 2009 EOSDIS Survey Flow - CLB

23. Reported a problem Closing 3842 Have you ever contacted <Data center name> to report a problem? 3842 No Do you have any additional comments or suggestion about possible improvements to data products, services, tools, documentation, or the websites that you would like to share? Are you finding what you need on our websites? (please comment) 2965 Yes 877 Using a 10-point scale on which “1” means “handled very poorly” and “10” means “handled very well”, please rate how well the problem was handled. You have reached the end of the survey. Please click on the "Finish" button below to send your responses to CFI Group's secure database. • Were you able to get the help you needed on your first request for assistance? • Yes • No Your survey responses have been received. NASA appreciates your input and will use this feedback to better serve its customers. 3842 23 2009 EOSDIS Survey Flow - CLB

24. 2004 2005 77 77 75 75 78 82 80 78 81 81 79 74 73 73 71 73 73 75 72 75 76 73 71 Customer Satisfaction Results N=1016 N=1263 N=2857 N=2291 N=2601 N=3842 2007 2006 2008 2009 ACSI 74 (+/-) 0.9 (+/-) 0.6 (+/-) 0.4 (+/-) 0.5 (+/-) 0.5 (+/-) 0.7 Overall satisfaction How satisfied are you with the data products and services provided by [DAAC]? Expectations To what extent have the data products and services provided by [DAAC] fallen short of or met your expectations? ATTRIBUTES Ideal How close does [DAAC] come to the ideal organization?

25. Customers over Multiple Years No significant differences were see between 2008 and 2009 for those who have answered the survey over the last four years. For those answering the survey over multiple years, scores have seen some movement

26. Customers over Three Years For those answering the survey in 2007, 2008 and 2009, Product Documentation saw a statistically significant score increase from 2008 to 2009.

27. Customers over Two Years For those answering the survey in 2008 and 2009, there are many statistically significant score increases.

28. Areas of Opportunity These areas have remained consistent year over year. Top Improvement Priority Product Search (75) Product Selection and Order (76) Product Documentation (77)

29. Summary • NASA EOSDIS has made significant improvements versus last year in multiple areas (Product Documentation, Quality and Customer Support) • Product Selection/Order saw a small but significant decrease this year • Product Search, Selection and Order continue to be the top opportunities for improvement • Documentation continues the trend with a higher impact this year and also continues to be a top opportunity • Customer Support continues to be high impact for those who require it. Imperative to maintain the strong level of service. • Ensure that those who are providing customer support realize how important it is and how it affects satisfaction.

30. Comments • Verbatim comments are supplied in separate word documents. • In what format(s) were your data products provided to you? (select any that apply) • Other (please specify and/or comment) • Did you use software tool(s) to work with the data (e.g., format conversion, analysis, visualization, etc.?) • Yes (Please specify which tool or tools you used to work with the data.) • No, I couldn’t find what I needed (please specify what you were looking for) • No, I couldn’t understand how to use it (please specify what you were trying to use) • Do you have any additional comments or suggestion about possible improvements to data products, services, tools, documentation, or the websites that you would like to share? Are you finding what you need on our websites? (please comment)