Using the VL-E Proof of Concept Environment

1. Using the VL-EProof of Concept Environment Connecting Users to the e-Science Infrastructure David Groep, NIKHEF

2. Virtual Laboratory for e-Science (NL) • To boost e-Science by • the creation of an e-Science environment • and doing research on methodologies • To carry out concerted research • along the complete e-Science technology chain, • ranging from applications to networking, • focused on new methodologies and reusable components.

3. Data Intensive Science/ Bio- Informatics Medical Diagnosis & Imaging Bio- Diversity Food Informatics Dutch Telescience VL-e Application Oriented Services XXXXXXXX Grid Services Harness multi-domain distributed resources Virtual Laboratory for e-Science

4. VL-E in a nutshell • Experiments become more complex • more than just coping with the data • Computer is integrated part of the experiment • support the experimental process end-to-end Application Needs (pull) Experiment validation Papers and associated data Provenance meta-data Information modeling Data/Resource Collection Access … Technology (push) … Grid Resource Sharing Web Networks

5. parameters/settings, algorithms, intermediate results, … software packages, algorithms … Parameter settings, Callibrations, Protocols … raw data processed data presentation acquisition processing sensors,amplifiers imaging devices,, … conversion, filtering,analyses, simulation, … visualization, animationinteractive exploration, … Rationalization of the experiment and processes via protocols Metadata The Experimental Process experiment interpretation Much of this is lost when an experiment is completed.

6. Combining data sources Key element for all users: Data Combination • From different organisations • data ownership preserved • data correctness maintained by preventing ‘forks’ • Extracting common meaning • need for workflow definition and ontologies in collaborative experiments

7. Combining data in Cognition Science • Collaborative scientific research • Information sharing • Metadata modeling • Allows for experiment validation • Independent confirmation of results • Statistical methodologies • Access to large collections of data and metadata • Training • Train the next generation using peer reviewed publications and the associated data

8. Combining Acquisition and Simulation • Robert: kun je hier een mooi plaatje voor maken? Het lijkt me de goede plaats om ook in-silico experimenten even te noemen

9. Role of the Proof-of-Concept (PoC) • Platform for user application development • Provisioning network & grid infrastructure • stable releases of common tools • tested ‘external’ middleware • stable releases of internal developments • Support for users & dissemination • infrastructure installations • end-user helpdesk • on-site aid in migration

10. Bio- Informatics Medical Diagnosis & Imaging Bio- Diversity Data Intensive Science/ Food Informatics Dutch Telescience Stable, reliable, tested Cert. releases Grid MW & VL-software Flexible, test environment Characteristics Flexible, ‘unstable’ Virtual Lab. rapid prototyping (interactive simulation) Test & Cert. Grid MW & VL-software Compatibility Application development Usage NL-Grid production cluster Central mass-storage facilities+SURFnet NL-Grid Fabric Research Cluster DAS-2, local resources Initial compute platform VL-e Rapid Prototyping Environment Environments VL-e Certification Environment VL-e Proof of Concept Environment PoC Release n Release Candidate n+1 Developers Heaven/Haven Integration tests Functionality tests Adventurous application people GT3.2 + * LCG2.x + SRB + LCG2.x + others Tagged Release Candidates Download RepositoryPoC InstallerCluster Tools Developer CVSNightly buildsUnit tests stable, tested releases externalmiddlewareproducts

11. Involving Users • Training via tutorials on middleware • good attendance, but slow uptake later on • On-site support in integration • good technology update, but people intensive • User driven integration: application pull • rapid update, good attendance • requires an ICT scientist to work long-term with the domain scientists to recognize and extract generic elements

12. Tutorials • Grid, LCG2 tutorials • Hands-on event series‘Grid Admin Nerd Group’ ‘After Sales Service’ • Documentation • User help-desk (by phone & mail) User Experience: nice, but information quickly ‘lost’

13. On-site support • EMUTD exampleMaurice to provide image & input • Effective use of EDG/EGEE tools for job submission, SRB for data access User experience:problem effectively solved!but with high manpower investment by PoC

14. Application pull Application Specific Part Application Specific Part Application Specific Part Potential Generic part Potential Generic part Potential Generic part Virtual Laboratory Application Oriented Services Management of comm. & computing Management of comm. & computing Management of comm. & computing Grid Services Harness multi-domain distributed resources Application Pull VL-E methodology

15. Can we keep our users content? • Take care of grid & generic aspects • collaboration community building & security • policy-constraint & dynamic resource sharing • Software Integration • there are many tools already … ‘just integrate them’ • but only wide deployment will show the weaknesses • Make it work • consistent software engineering practices • hide changes lower layers by use of standard interfaces • Easy-to-use installers (PoC Installer, Quattor) • and teach us how to scale up to a grid service provider

16. http://www.vl-e.nl/