Cyberinfrastructure Requirements and Best Practices

Cyberinfrastructure Requirements and Best Practices Lessons from a study of TeraGrid Ann Zimmerman Research Assistant Professor UM School of Information OGF Workshop, May 27, 2009

Outline Background Challenges People Methods used Analysis Conclusions

Background: Learning from TeraGrid • NSF-funded study to examine: • The TeraGrid collaboration • user needs and requirements • impact on research practice & outcomes • education, outreach & training activities • Research Team • Tom Finholt, PI; Ann Zimmerman, co-PI • Magia Krause, PhD student

Key Questions: User Needs • What factors affect users’ computing needs and requirements? • What factors affect users' behavior as it relates to their use (or non-use) of TeraGrid/HPC? • How are the needs of users expected to change over the next five years?

Key Questions Continued • Where do users currently spend time that does not count as doing science? • What research questions do they want to answer but currently cannot? What are the barriers?

Data Collection June 2006-May 2007 • 7 site visits, including 4 TeraGrid sites • Interviews (n=~90) • Participant observations • User workshop • Document analysis and review • Surveys • Survey of current TeraGrid users • Surveys of tutorials at TG ’06 & TG ‘07

TeraGrid Planning Process • The goal of the planning process was to • developoptions for delivering TeraGrid resources and services • based on the diverse needs of science and engineering communities • http://www.teragridfuture.org/

Challenges Heterogeneous users Potentially thousands of users Distributed environment

People included in the study • Current TeraGrid users • “Target” TeraGrid users • Non-users • Science gateway developers • Cyberinfrastructure “experts” • TeraGrid personnel

Methods

Workshops • Strengths • Learn by seeing users interact • Gain both broad & detailed information • Relatively efficient • Challenges • Require careful & creative planning (pre- and post-workshop) • Invite 3-4 times as many people as you want to participate in the workshop

Interviews • Strengths • Gain detailed information • Information informs survey development • Challenges • Time-consuming (to conduct & to analyze data) • Resources limit the number of people who can be interviewed

Survey • Advantages • Findings are generalizable across a larger population • Inexpensive (in $ not in person time) • Challenges • Developing a good survey is hard • Getting a good response rate takes a lot of effort

Participant Observation • Advantages • Learn about all the factors that affect the ability to serve users • Learn about user needs from a variety of sources • Challenges • Time-consuming • Capturing and analyzing data

findings

Components of User Behavior • The nature of the research problem • Alignment between infrastructure and scientific practice • Computational readiness • Ease of use

Achieving Transformative Science • “Easy” things can be show stoppers • Many complexities to manage • virtual organization • diverse user needs • changes in science

Conclusions • Different methods provide different kinds of information • Involve more than users in your study • Current methods are effective, but time-consuming and resource-intensive • New methods required. For example, “mine” and analyze sources of information (wikis, user support logs and databases, user sites)

More information • TeraGrid evaluation study reports & Planning Process workshop reports • http://deepblue.lib.umich.edu (browse for documents by Ann Zimmerman) • Other TeraGrid Planning Process materials • http://teragridfuture.org

Acknowledgments • TeraGrid • Research participants • NSF grants OCI-0602525 and OCI-0724300

Contact me! • Sorry I couldn’t be here! • Contact me at: • asz@umich.edu • http://www-personal.si.umich.edu/~asz/index.html

Cyberinfrastructure Requirements and Best Practices