1 / 33

NA4: User Community Support and Expansion

NA4: User Community Support and Expansion. C. Loomis (CNRS/LAL) NA4 Activity Manager EGEE-III First Review, 24-25 June, 2009. Activity Overview. NA4: 19%. Tasks. TNA4.1: Support Virtual Organization Support (VOS) Application Porting Support (APS) Direct User Support (DUS)

berke
Download Presentation

NA4: User Community Support and Expansion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NA4: User CommunitySupport and Expansion C. Loomis (CNRS/LAL) NA4 Activity Manager EGEE-III First Review, 24-25 June, 2009

  2. Activity Overview NA4:19% NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  3. Tasks TNA4.1: Support • Virtual Organization Support (VOS) • Application Porting Support (APS) • Direct User Support (DUS) TNA4.2: Strategic Discipline Clusters • High Energy Physics (HEP) • Life Sciences (LS) • Earth Sciences (ES) • Grid Observatory (GO, CS) • Computational Chemistry (CC) • Astronomy & Astrophysics (AA) • Fusion (F) TNA4.3: Activity Coordination • Activity Management • Regional Coordination NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  4. User Community Organization Domain Domain Clusters VO VO VO Grid Auth. User User User User User User Community User User User User User NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  5. Community & Use Consistent doubling every 12-18 months. EGEE-II = Y1 EGEE-III = Y2 Around 13000 Registered Users CIC Portal: http://cic.gridops.org/ Accounting Portal: http://www3.egee.cesga.es/ NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  6. CPU Utilization by Domain CPU Use: 1K-SI2K-Month 112 Registered VOs 171 “Visible” VOs 23 “Core” VOs 4167 “Core” Users 2x increase overall HEP largest users / contributors AA/ES/OTH show strong increase NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  7. Applications http://appdb.eu-egee.org Alt. link: http://grid.ct.infn.it/egee_applications/ NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  8. Virtual Organization Support VO Management Developments • Improving the VO registration information • Integration of collaborative tools with VO information • Expansion of SAM testing framework for non-LHC VOs Documentation and Support Provision • Links to VO documentation: https://twiki.cern.ch/twiki/bin/view/EGEE/VOSupport • Liaison between operations and VO managers VO Tools Identification • Identified problems with VOMS functionality • Worked in collaboration with JSPG on changes to policies related to VO management NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  9. Application Porting Support Consultancy and Porting • 15 applications ported; ~10 applications being ported Training • Group collects, prepares, reorganizes training materials and offers those as customized training packages for users • Direct participation in NA3 training events Provision of Infrastructure Services • Group leader is VO manager for the NA4 • Partly responsible for Application Database Public Relations • Writing of success stories of ported applications to increase visibility and to help others with similar applications • “EGEE App. Porting Support Group” won Best Demo prize at EGEE’08 NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  10. Application Porting Support http://www.ldps.sztaki.hu/gasuc/ 15 applications ported 10 applications being ported NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  11. Direct User Support Ticket Handling • DUS support unit part of GGUS since mid-September • Have taken 2-person, 2-week shifts to treat tickets • The number of tickets assigned to DUS has been small Documentation and Use Cases • Reviewed and accessed existing documentation • Writing new documentation to fill identified gaps • Working with clusters and other teams to improve their documentation NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  12. High Energy Physics Ganga/DIANE, AMGA, Dashboard • Used by 1000s in HEP, strong adoption by other communities • Ganga/DIANE tutorials: NSS IEEE, Helsinki, BalticGrid • Dashboard tutorial: UF4/OGF25 • CERN Training for Trainers Grid validation for LHC data taking: CCRC’08, STEP’09 • 4 expts., 3 grid infra., 100s of sites, O(PB) data • Sustained 4GB/s CERN  T1, O(100K)jobs/day NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  13. Technology Transfer Collab. HEP and Fusion clusters (since EGEE’08) • Porting of specific fusion applications using Ganga/DIANE • Results of the collaboration shown during UF4/OGF25 Lattice QCD (in production since 2007) • Running autonomously on a daily basis using Ganga/DIANE • Sustained rate of 1000 concurrent jobs, 750 CPUs and more than 20TB of data transferred GEANT4 simulation toolkit (since 2005) • Widely used by astroparticle physics, medical applications, radiation studies, as well as HEP • Validation of the new releases performed regularly on EGEE grid, again using Ganga/DIANE NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  14. Life Sciences Major calculations: • WISDOM (http://wisdom.eu-egee.fr/) • System biology on cancer data • Genetic linkage analysis for disease loci • Identification of causes for coronary artery diseases Tooling Support • AMGA • Medical Data Manager • MOTEUR • Taverna2 Plug-In NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  15. Nature Genetics Article Genome-wide haplotype analyses of complex human diseases • Study the impact of DNA mutations on human coronary diseases • Very CPU intensive analysis to study the impact of correlated (double, triple) DNA mutations EGEE grid deployment • 1926 Coronary Artery Disease patients; 2938 healthy controls • 378,000 Single Nucleon Polymorphisms = local DNA mutations • 8.1 million combinations tested in less than 45 days (instead of >10 years on a single Pentium 4) Results in Nature Genetics Mar. 2009 (D. Tregouet et al) • Major role of mutations on chromosome 6 was confirmed. NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  16. Earth Sciences User and application support Dissemination • Session at European Geosciences Union (EGU) in 2008 • Special issue of journal with 12 peer-reviewed papers • 2 PhDs and 5 papers based on Geocluster results from EGEE Specialized tools • Data distribution, file explorer, storage access systems, workflow tools, … NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  17. Footprint Pesticide risk assessment and management in Europe • FP6 EU project • BRGM France + 14 partners in 9 countries Creation of a large database including 4 million scenarios (climate, soil, pesticides, …). Successful results with the first 2 million scenarios obtained with EGEE running around 4800 jobs/day. Exploitation of database and results by all partners. • Creation of one SME in France for agriculture consultancy. NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  18. Grid Observatory Created the Grid Observatory Portal • Store and publish monitoring information for analysis Reaching out to CS community: • EGEE’08: Grid Community Meeting • UF4/OGF25: Joint session “From Grid Monitoring to Analysis” • Grid Meeting Autonomic Computing (GMAC’09) at ICAC NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  19. Grid Observatory Portal http://grid-observatory.org/ NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  20. Computational Chemistry Analysis of grid licensing models. Expanding membership • Training of young researchers • Availability of necessary software packages Tooling: • Chempo, Charon, ECCE, Wien2K • Parallel version of GAMESS NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  21. Generator: generates input files with different parameters (currently 4 input) Executor: executed as many times in parallel as many parameters are generated by “Generator” Collector: collects all output files into a single TAR file Computational Campaigns Chemical reactions • N + N2, O + O2 and F + HD • Thermal rate coefficients Nanotube modeling P-Grade port of ABC program NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  22. Astronomy & Astrophysics Development of active community • Large number of applications ported to the grid • Focused training and dissemination Tooling: • Management of parameter sweep applications • Scheduling and bookkeeping systems • Visualization Interaction with EuroVO NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  23. Planck Satellite Launched 14 May will be in L2 orbit in early July. INAF: Ported full LFI missionsimulation to EGEE IFCA: Ported several codes toLFI Data Processing Center: • Mexican Hat Wavelet filters • Multi-frequency Matrix filters • Matched Multi-filter code NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  24. Fusion Application porting • 9 applications have been ported to give relevant scientific results Tooling: • Data mgt. tool development with goal of multi-machine analysis • GIF Portal for launching Generic Algorithm-based applications • Use of Kepler workflow engine (bridge to EUFORIA project) NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  25. Fusion Developments MaRaTra ASTRA ISDEP MC code follows ion trajectories inside plasma • Self-consistent plasma profiles: intro. of non-linear effects • Divertor Studies: Map of 3D fluxes on wall of device • Tokamak geometry • Ion heating ASTRA-MaRaTra • First complex fusion workflow betweenapplications running on different platforms • ASTRA: SGI Application • MaRaTra: Grid Infrastructures NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  26. Activity Coordination Activity Management • All milestones and deliverables have been achieved • Maintain the RESPECT program • Encouraged community interaction via the User Forum • Encourage participation in meetings through “travel money”: • Financed 6 people to attend EGEE’08 and UF4 • Sponsor of the GMAC’09 workshop • Contributed to EGEE  EGI migration via SSC Workshops • Collaboration with MathWorks for MATLAB on the grid Regional Coordination • Design, implementation, and filling of Application Database • First line support, document review, etc. • Liaison activities increasingly important as EGI approaches NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  27. RESPECT Identify third-party software that works well with gLite. • http://technical.eu-egee.org/index.php?id=290 Simplified Access • P-GRADE, Ganga, Migrating Desktop, g-Eclipse, i2glogin, Virtual Control Room Workload Management • GridWayMetascheduler, DIANE New Resources • GRelC, Instrument Element Infrastructure Services • StoRM NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  28. User Forum UF3 (Clermont-Ferrand) UF2-OGF20 (Manchester) UF4-OGF25 (Catania) UF1 (CERN) http://technical.eu-egee.org/index.php?id=148 NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  29. Issues Followed upin project Largelyresolved Emphasis inYear 2 Technical Issues • Fragility of applications with upgrades • Ease of use (availability of Java APIs) • SAM Nagios transition for VO-specific tests • Firewall configurations and data transfers • MPI support Administrative Issues • Late recruiting • Unresponsive partners Systemic Problems • Visibility of the NA4 support services. • Underutilization of those support services. NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  30. Deviations from Work Plan Lower spending than planned. Slow start up. Low visibility and under-utilization of services. Higher spending than planned. Expect rate to continue. Becomes additional unfunded contributions. Significant fraction of expended effort. NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  31. Plans for Year 2 Support Activities: • Improve visibility and use of all support services • Publicize the seed resources for new users and new VOs • Work on transition to EGI support structures Strategic Discipline Clusters • Continue current scientific activities • Work on transition to the EGI SSC models Management • Continue coordination activities • Make more use of community building funds • Enhance cooperation with NA2 to increase dissemination NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  32. Specialized Support Centers No major structural changes: • NA4 Steering Committee  User Forum Steering Committee • Strategic Discipline Clusters  Specialized Support Centers Each SSC: • Must be much more autonomous than now • Must find and attract financial and political support • Must be the center of gravity for grid use within their communities It will be a hard challenge to have fully functional SSCs in time for the start of EGI. NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

  33. Summary Three principal tasks of NA4 have worked well. User Community • 13000 users, 220 applications, 112 registered VOs • Majority of use from 23 core VOs • Overall CPU use increased by factor of 2 Scientific impact • Shown results could only be achieved with the grid. • User Forum 4 program and Book of Abstracts • Detailed achievements provided in DNA4.4.1 Future Work • Improve visibility and utilization of support services • Guide formation of SSCs for the EGI transition NA4 - C. Loomis - EGEE-III First Review 24-25 June 2009

More Related