S tudy of U ser P riorities for e -Infrastructure for e- R esearch (SUPER)

1. Study of User Priorities fore-Infrastructure for e-Research (SUPER) Steven Newhouse Jennifer Schopf Andrew Richards Malcolm Atkinson NeSC Workshop - February 2007

2. We have a dream… • A usable, useful, and accessible e-infrastructure for researchers across a wide variety of disciplines • Existing and new e-infrastructure facilities and services integrated into a coherent whole • To increase the use of the existing e-infrastructures by a >10 by 2010 NeSC Workshop - February 2007

3. Turning Dreams to Reality! • Identify issues that are: • Short-term (6-18 months): • Actions within existing funding streams • Longer-term (3-5 years): • Actions that need new/renwed funding streams • Inform roadmaps for collaborative research: • Organisations: OMII-UK, NGS, DCC, … • Funders: RC UK, JISC, JCSR, … • Not the place to identify solutions NeSC Workshop - February 2007

4. Methodology & Coverage • Face to face interviews: • Newcastle, Glasgow, Edinburgh, Oxford, Cambridge, UCL, Reading & on-line survey • Covered 45 people from over 30 projects • ~30% EPSRC • ~30% BBSRC, MRC and JISC • Remainder: DTI, EU, Wellcome, AHRC, ESRC, NERC & PPARC, University funded NeSC Workshop - February 2007

5. Projects by Funders NeSC Workshop - February 2007

6. Major Common Topics • Distributed file management and policy • Tools to support dynamic Virtual Organisations • Long-term project support: • Tools, services, training and consultancy • Provision of authentication, software licensing, and reliable consistent environments • User Interaction with e-infrastructure services NeSC Workshop - February 2007

7. File management and policy • Growth of simulation based science on grid resources • Files needed as input & output (not DBs) • From your desktop and remote resources • Files need meta-data as to content & source • Ideally automatic annotation & provenance • How long must/should they be stored for? • Who can have access and when? NeSC Workshop - February 2007

8. Dynamic Virtual Organisations • Current VO models are relatively static and driven from the centre, e.g. VOMS • Need to be more end-user centric • The resources and services I can access: • Through collaboration, membership, position, … • VO composition & relationships change • Tools and operating models need to reflect this • May need to record activity within a VO • e.g. contracts, deliverables, NeSC Workshop - February 2007

9. Projects need Support • Teams: ‘people the most valuable asset’ • But managing distributed teams as one is very hard • Manage different cultures, organisations & incentives • Services: ‘stand on shoulders’ • Software: Use what’s out there: • NGS: Core very low-level infrastructures • Community Resources: MIMAS, EDINA, myGrid, … • People: Need access to experts • Training & Consultancy: ‘rapidly gain knowledge’ • Hard to get all the skills in one person  MUST TRAIN NeSC Workshop - February 2007

10. User Oriented Operational Issues • Authentication • Certificates adopted by service providers • Very difficult for many end-user communities • Deployment of many wrappers around certificates • Licensing • Growing use of third party commercial applications, e.g. Matlab • Use my license on remote machine • Currently very hard (impossible) • Consistent Environments • Without consistency very hard to deploy licensed applications on demand • Reliability • If services not reliable, might as well not be there. Better if not! NeSC Workshop - February 2007

11. User Interaction withe-infrastructure Services • Interactions MUST match the user • Technical Expertise • Normal Environment • Command line shells • Traditional ‘expert’ interface to systems • Scripting Environment • From within basic shells: Bash, Tcsh, … • Application Environments: Perl, Matlab, Python, … • Workflows • Portals All underpinned through a common API NeSC Workshop - February 2007

12. Minor Issues • Sustainability • FEC is making collaboration harder • Need to ensure usage is accounted for • File Replication • End to End Security • Firewalls still the ‘blunt instrument’ of choice • Scheduling • A vital requirement for only a few groups NeSC Workshop - February 2007

13. Conclusions • Software • Much has been prototyped… but not finished • There are still areas that need experimentation • Policy • Need ‘better joined up’ ness through best practice • Data, VOs, Environments, … • Community Support • Technical consultancy for all users of e‑infrastructure services from people • Self-help training materials and hands-on tutorials delivered by trainers for common tools NeSC Workshop - February 2007

14. Acknowledgements • Funding • E-Science Core Programme & JISC • Day release • OMII-UK, NGS, JISC, Globus Alliance • Contributors • Interviewees for their honesty & flexibility • Comments from the community • Feedback: s.newhouse@omii.ac.uk NeSC Workshop - February 2007

15. Technical Breakouts • Virtual Organisation Infrastructures • Life Cycle • Entities that need to be managed & accounted for • E-Infrastructure Support Service • Training • Consultancy • E-Infrastructure Deployment • Consistent environments • User Interaction • File & meta-data Management • Mechanisms • Policy • Replication NeSC Workshop - February 2007

16. Breakout Considerations • Who are the early adopter/active communities? • To gather detailed requirements • How uniform are the requirements within the community? • Are there gaps? Revise emphasis? • Targets over the next 12 months • Longer term? NeSC Workshop - February 2007