1 / 38

NA4 and EGEE User Forum IV

NA4 and EGEE User Forum IV. C. Vuerli (1,2) (presentation by F. Pasian (1,2) ) (1) INAF – Astronomical Observatory of Trieste (2) INAF – Information Systems Unit Friday 27 March 2009. EGEE-III NA4. Main Goal: expand and support the User Community

erno
Download Presentation

NA4 and EGEE User Forum IV

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NA4 and EGEE User Forum IV C. Vuerli (1,2) (presentation by F. Pasian (1,2)) (1) INAF – Astronomical Observatory of Trieste(2) INAF – Information Systems Unit Friday 27 March 2009

  2. EGEE-III NA4 • Main Goal: expand and support the User Community • Collaborative tools have been set up to improve internal communication and efficiency • EDMS to manage documents • Internal mailing lists for each of the twelve teams • Indico to manage meetings and calendars • The NA4 activity backlog to manage current and ongoing tasks • NA4 scratch pad for various working and draft documents • A set of Policies and procedures has been set up to clarify how the activity provides services to the user community • NA4 maintains a list of web pages for public consumption Friday 27 March 2009 - IGI - Task Force Progettazione

  3. NA4 Support Teams • VO Support • Helps Virtual Organizations and aids user communities in successfully interacting with the project • Application Porting Support • Helps users transfer applications to the EGEE Grid • Direct User Support • Consolidates and coordinates all the user-centric, application oriented support activities in EGEE Friday 27 March 2009 - IGI - Task Force Progettazione

  4. NA4 Support Teams • Regional Support • Ten NA4 Regional Support provides support to Teams Users and scientific communities wishing to get first level support regarding access to EGEE Grid infrastructure Friday 27 March 2009 - IGI - Task Force Progettazione

  5. Clusters and Coordination • Strategic Discipline Clusters • Astronomy & Astrophysics • Computational Chemistry • Earth Science • Fusion • Grid Observatory • High-Energy Physics • Life Sciences • Activity Coordination • Most decisions are taken after discussions within the NA4 SC (Steering Committee), consisting of the team leaders of the 11 NA4 teams and the above managers • NA4 SC is led by Charles Loomis, Vangelis Floros, and Frank Harris Friday 27 March 2009 - IGI - Task Force Progettazione

  6. User Forums • NA4 has in charge the organization of yearly EGEE User Forums • UF I (CERN, 2006) • UF II (Manchester, 2007) • UF III (Clermont-Ferrand, 2008) • UF IV (Catania, 2009) Friday 27 March 2009 - IGI - Task Force Progettazione

  7. User Forum IV • UF IV Contributions • 161 Contributions, 18 Demos, 100 Oral Presentations, 43 Posters • UF IV Sessions • 5 Plenary • 148 Parallel • UF IV Topics • Scientific results obtained using grid technology • Planned or on-going scientific work using the grid • Experiences from application porting and deployment • Grid Services exploiting and extending gLite middleware • Programming environments • End-user environments and portal technologies • Emerging Technologies within the EGEE infrastructure Friday 27 March 2009 - IGI - Task Force Progettazione

  8. UF IV / Fusion • Six oral presentations • The usage of Grid infrastructures for fusion research has provided interesting results that open themselves new lines of research. • More Applications run in the Grid with different schemes and structures (e.g. sequential applications, PIC codes and optimization procedures based upon genetic algorithms). • Use of Grid for data management is totally new in fusion • Report about an experience of a complex workflow between an application that runs in a shared memory of the computer and in the Grid. Friday 27 March 2009 - IGI - Task Force Progettazione

  9. UF IV / Computational Chemistry • Six oral presentations • Computational chemistry is now the third EGEE Grid consumer, just after High Energy Physics and Life Science. • Applications cover a wide area ranging from quantum dynamics computations for small molecular systems up to molecular dynamics studies of huge molecular systems of biological and industrial importance. • Oral talks along with accompanying posters detail not only development and adoption of high-level tools and services easing execution of numerical experiments in chemistry and their planning, but also scientific results obtained with help of grid computing. • The User Forum as a perfect opportunity to get an overview about current research activities among chemical community on EGEE Grid. Friday 27 March 2009 - IGI - Task Force Progettazione

  10. UF IV / Earth Sciences • Eight oral presentations • Earth Science (ES) covers a large and diverse user community, academy, European and international organization as well as industry. • Phenomena under study are dependent on geographical coordinates, altitude and time. Active collaborations set up on a worldwide basis  distributed work. • Applications need for access to intensive computing facilities and to large and heterogeneous sets of data. • Development of application specific grid- and web services. • Gridified Applications reflect the variety of scientific ES disciplines: atmospheric chemistry, climate, hydrology, meteorology pollution, solid earth physics. • Grid provides more computing resources to the user and is also a way of sharing data, algorithm and of developing common tools. • Currently the interface between ES environment and Grid middleware is not simple for many applications and developments Friday 27 March 2009 - IGI - Task Force Progettazione

  11. UF IV / High Energy Physics • Seven oral presentations • HEP is one of the pilot application domains in EGEE, and is the largest user of its grid infrastructure. • HEP applications are very demanding; they serve as a powerful role in understanding and improving EGEE delivered services. • HEP experiments also produce high-level middleware components than often become valuable prototypes for the overall grid community. • The expertise developed by HEP users is open to the other EGEE grid users • User Forums are a big opportunity to share the experiences between Grid users and also to trigger important collaboration between different clusters • Session 1/2 devoted to the largest LHC experiments • Session 2/2 dedicated to the data analysis framework of the CMS experiment Friday 27 March 2009 - IGI - Task Force Progettazione

  12. UF IV / Bio-info and Bio-medicine • Eight oral presentations • Life Sciences is a very active field of research using the EGEE infrastructure. • Several EGEE related projects are also exploring the use of gLite middleware for bioinformatics and healthcare. • For several years now, the Biomed Virtual Organization keeps standing as the first non LHC Virtual Organization consuming about 5% of the infrastructure resources. . • User Forum is a wonderful opportunity to get an overview of the present adoption of grids in the Life Sciences communities through the dedicated oral sessions but also through the posters and demos exhibition as well as the OGF Life Sciences and e-Health workshop. • Complementary to the “medical imaging” session, the “bioinformatics and biomedicine” session program reflects the variety of topics currently addressed on EGEE and its related projects. Friday 27 March 2009 - IGI - Task Force Progettazione

  13. UF IV / Astronomy-Astrophysics • Seven oral presentations • Astronomy & Astrophysics in EGEE since 2004 with two pilot applications. • Now five funded and eight unfunded research groups contribute to the A&A cluster with applications and use-cases. • A&A applications are extremely challenging in terms of resource demands, both conventional and non conventional. • Three main A&A VOs currently in place • New A&A research groups recently asked to join the A&A VOs to exploit EGEE Grid facilities. • The 4th User Forum was an excellent opportunity to get an overview of the progress in adopting EGEE and its Grid infrastructure within the A&A community – A&A contributions equally addressed scientific and technological aspects related to the usage of the Grid technology as well as the adoption of tools and services to make easier or more efficient the porting of applications. • First discussion on necessary steps to migrate the A&A community to EGI. Friday 27 March 2009 - IGI - Task Force Progettazione

  14. UF IV A&A / Seven Presentations • Outcome of the EuroVO-DCA WP5 ActivitiesC. Vuerli • Online Visualization for Grid-based Astronomical Parametric Studies, E. Pajorova • Using DIANE for Astrophysical Applications, V. Tran • Detection of compact objects in maps of the CMB Radiation with Planck, M. Lopez-Caniego • ESIP – Grid based satellite Data Processing Platform, D. Gorgan • Migration of the Magic Datacenter and Monte Carlo simulation to a Grid infrastructure, R. Firpo Curcoll • TheoSSA on AstroGrid-D, I. Nickelt, H. Enke Friday 27 March 2009 - IGI - Task Force Progettazione

  15. UF IV A&A / Topics • Presentations concerned (I): • Reports about Grid-related activities carried out by working groups and results achieved • Development of tools to make the porting of astrophysical applications in Grid smoother and simpler for application developers (visualization tools, parametric studies, etc.) • Porting of simulation code to Grid • Planck: detection of compact objects, extragalactic point sources, SZ clusters (using the Planck VO on EGEE) • recent progress in the adoption of the Grid infrastructure in Magic (using the Magic VO on EGEE) Friday 27 March 2009 - IGI - Task Force Progettazione

  16. UF IV A&A / Topics • Presentations concerned (II): • How to implement Virtual Observatory standards (defined within world-wide bodies – IVOA) on top of Grid middleware • Combination of a Virtual Observatory service for stellar model spectra with a grid compute service. Astronomers can request model spectra according to certain specifications. If no pre-calculated results are available, the model calculation is executed in a Grid, which provides the necessary scaling and greater reliability. • Grid-based software platform for satellite image processing and development of environmental applications Friday 27 March 2009 - IGI - Task Force Progettazione

  17. NA4 and Transition to EGI • NA4 currently plays a key role in the process of migrating the EGEE Community to EGI. • All members of the NA4 SC are committing themselves to create SSCs for their respective cluster communities • A first meeting dedicated to the transition was organized at LAL, Orsay, in December 2008 • NA4 provided important contributions to the EGI_DS Blueprint in terms of requirements coming from the various communities of users • At UF IV the most part of disciplinary clusters started a discussion finalized to the transition to EGI • A key event will take place in mid May 2009 (in Athens) aimed at migrating the various clusters Friday 27 March 2009 - IGI - Task Force Progettazione

  18. Outcome of the LAL meeting • SSCs is generally viewed as a feasible model • SSCs must be owned by the user community served by the SSC and be driven by that user community • Further details about the outcome of the meeting can be found athttp://indico.cern.ch/materialDisplay.py?materialId=minutes&confId=45736 • SSC rights • EGI will provide resources to the SSCs • SSCs will have access to resources • SSC will have representation in the EGI council • Access to training and documentation • Have forums for SSC user communities and SSC managers Friday 27 March 2009 - IGI - Task Force Progettazione

  19. Outcome of the LAL meeting • SSC responsibilities • Have a cohesive community • Have a European-level existence • Have a fairly long lifetime (minimum 2-3 years) • Report facts and figures about the use of EGI • Follow the defined EGI policies, both security and operational • Operate transparently allowing a clear view of the SSC's activities • SSC interfaces with • the User Forum Steering Committee • the EGI technical support • the middleware coordination group • Other SSCs • Operations • NGIs Friday 27 March 2009 - IGI - Task Force Progettazione

  20. SSCs in the Blueprint • The EGI_DS Blueprint has been approved in Prague on January 20th 2009. • The sections of the document referring SSC are: • 2.3.4.1 (EGI User Forum, page 9/41) • 3.3.2 (Specialized Support Centres, page 20/41) • 3.3.3 (Effort estimates for SSC Interfacing Layer and NGIs, page 21/41) Friday 27 March 2009 - IGI - Task Force Progettazione

  21. Slide by D. Cresti shown at UF IV EGI Middleware Coordination Board - MCB MW OPS UCS EGI User Forum User Forum Steering Committee - UFSC SSC1 SSC2 SSC3 SSCn SSC NGI UCS NGI UCS NGI UCS SSC SSC User Technical Support (e.g. USAG) Friday 27 March 2009 - IGI - Task Force Progettazione

  22. From EGEE Clusters to EGI SSCs • January 2009: questionnaire distributed by NA4 concerning the migration of disciplinary clusters to SSCs. Questions were: • Will the cluster move to a SSC? • The SSC name • The scope • List of persons/institutes and amount of human resources • List if institutes able to provide computing resources • Organization of the SSC • Required funding and plan to secure it • Evolution plan (timescales, etc.) • At present the most mature proposals are probably those coming from Life Sciences and A&A Friday 27 March 2009 - IGI - Task Force Progettazione

  23. A&A SSC Proposal • Scope • Consolidate and make permanent the presence of the European Astronomical Community within EGI, the European Grid Initiative • Activities • Consolidation and maintenance of A&A VOs currently in place • If necessary creation of new A&A VOs • Interactions with other VOs in EGI • Porting of new applications on the EGI Grid infrastructure • Development of tools and services to make the Grid as user friendly as possible • Members • Initially members are those that currently compose the cluster but the expansion process is already in progress Friday 27 March 2009 - IGI - Task Force Progettazione

  24. A&A SSC Structure ……. NGI-3 NGI-k NGI-1 NGI-2 A&A SSC SC (Steering Committee) SSC SC Chair EGI UFSC GPB MW Devel/Maint Group UCS Group UTS Operations Group Direct User Support Group SSC Front Desk EGI Users Community GPB Coordinator UTS Coordinator EGI MCB EGI UTSG Friday 27 March 2009 - IGI - Task Force Progettazione

  25. A&A SSC Structure • SC (Steering Committee) • Governs the whole SSC • Members are Local Coordinators of the SSC Groups • Local Coordinators interact with their respective NGIs • A&A SSC SC Chair • Is chosen within the SSC SC • Should be member of the EGI UFSC (User Forum SC) • GPB (Grid Planning Board) • Mainly deals with development/evolution of the Grid MW • MW development/maintenance and UCS (User Community Tools/Services) are two Groups active in GPB • A member is nominated coordinator • The coordinator should be member of the EGI MCB Friday 27 March 2009 - IGI - Task Force Progettazione

  26. A&A SSC Structure • UTS (User Technical Support) • Deals mainly with “Operations” and “Direct User Support” • A member is nominated Coordinator • The Coordinator interacts with EGI UTSC (User Technical Support Group) • SSC Front Desk • Gives support to various categories of users: members of the EGI main bodies, members of other SSCs, users who ask information about the A&A SSC, etc. Friday 27 March 2009 - IGI - Task Force Progettazione

  27. A&A SSC Man-Power • Porting of scientific applications to the EGI Grid infrastructure • Experts of the Grid Middleware and its related tools/services. This man-power will mainly operate within the GPB. • Man-power dedicated to the UTS and dealing with • procurement of necessary resources • user training/dissemination activities • Other man-power still TBD Friday 27 March 2009 - IGI - Task Force Progettazione

  28. A&A SSC Timing Friday 27 March 2009 - IGI - Task Force Progettazione

  29. A&A SSC / Open Questions • Is the proposed organization adequate for our community? • What areas to cover? (MW, Tools/Services, Application Porting, Training/Dissemination, Help Desk, etc.) • How many people? • Which interfaces with EGI.org? (in particular with UFSC and with NGIs) • Is the suggested timing reasonable? • Which funding mechanism (From EGI.org? From NGIs? Through EC calls?) • How to attract new A&A research groups? • What about interoperability between EGI A&A SSC and VObs? Joint EGEE-III A&A Cluster – EuroVO meeting? Friday 27 March 2009 - IGI - Task Force Progettazione

  30. A&A SSC / Discussion at UF IV • First preliminary discussion about the migration of the astrophysical cluster to EGI • Diana Cresti attended the session and provided important information during the discussion • The most important points discussed: • What a SSC is exactly in EGI • Funding mechanisms for SSCs in EGI • Possible dimensions (in terms of man-power) for A&A SSC Friday 27 March 2009 - IGI - Task Force Progettazione

  31. A&A SSC / Discussion at UF IV • First outcome: • People did not express comments about the suggested organization of the SSC. They need some more time to examine this preliminary draft • The community must stay prepared to answer to a possible EC call in the next months to acquire the necessary funds to set up and operate the SSC • The future SSC should be extended and go beyond the current cluster, by involving new research groups not in EGEE • The major tasks for the next months therefore should aim at: • Consolidating the collaboration and the cohesion of the cluster • Disseminating the results got within EGEE trying to involve in this way new research groups • Special attention must be paid to relationships with EuroVO Friday 27 March 2009 - IGI - Task Force Progettazione

  32. SSC for Life Sciences http://knowledge.eu-egi.eu/knowledge/index.php/D32#Appendix_C Friday 27 March 2009 - IGI - Task Force Progettazione

  33. SSC for Life Sciences • The Life Sciences community is very large and heterogeneous • In the beginning of EGI there might be just one SSC for this entire community • High likelihood that, as grid usage in the field evolves, more specialized SSCs may spin off from the original one • Several Thematic clusters: • Bioinformatics • Medical Imaging • Drug discovery and Health Friday 27 March 2009 - IGI - Task Force Progettazione

  34. SSC for Life Sciences / VOs • In terms of usage typology, the community can be divided into three categories: • Research Teams coming together for a limited time to constitute a project which pursues some research objectives. These users are keen to have their own VO for the duration of their project and therefore their own scientific gateway as well. • Research teams having a continuous scientific activity that requires regular access to limited resources (1-2 CPU years) and from time to time, access to important resources (10-20 CPU years). These people are the customers of the biomed VO that they join once for all. The biomed VO should have its own scientific gateway. • The ESFRI infrastructures will structure communities with well defined long tern scientific topics and therefore specific requirements. Each ESFRI should have its own international VO, its own scientific gateways and be managed from the ESFRI collaboration board. Friday 27 March 2009 - IGI - Task Force Progettazione

  35. SSC for Life Sciences / UCS Layer • LS SSC proposal to structure the UCS layer services: • One User Forum Representative and member of the UFSC • One Grid Planning Officer to participate in the EGI MCB meetings. • 2 FTE for User Technical Support and other support tasks. • A team of 2 FTE to design and build the Life Sciences Gateway. • One web content manager to work with the Gateway team and other information-related tasks. • 1 FTE for coordination of Documentation and Training specific to the LS community. • A team of 2 FTE for Consulting with new communities, Application Porting Support, and to maintain and monitor the Application Database for the Life Sciences. • The global UCS layer for the LS SSC is thus initially estimated at between 9 and 14 FTE. Friday 27 March 2009 - IGI - Task Force Progettazione

  36. Conclusions (I) • All disciplinary clusters now in EGEE-III continue to grow in terms of users, applications and shared resources • Clusters started the migration process toward EGI and proposals to set up SSCs are already in place • At present the maturity level of the various SSC proposals are highly inhomogeneous • A coordination activity to drive the formation of SSCs is extremely important • Some fundamental questions related to EGI SSCs are still open (mainly relationships with EGI.org bodies and NGIs and funding mechanisms) • The necessary coordination will be assured by NA4 Friday 27 March 2009 - IGI - Task Force Progettazione

  37. Conclusions (II) • Next important step towards the constitution of SSCs in EGI (to be confirmed) • 5-6 May 2009 • Athens • Goals of the meeting • Discuss the common characteristics of an SSC. • Determine the different types of SSCs (scientific, support, business, ...). • Discuss sources of funding for each type of SSC. • Determine how each SSC will deal with the probable September FP7 Infrastructure Call. • Discuss timelines for preparing proposals or contributing to the EGI proposal. Friday 27 March 2009 - IGI - Task Force Progettazione

  38. A&A Requirements Computational • Grid nodes with HPC capabilities  (e.g. COMETA) • Fine grained parallelism (MPI)  (in CREAM?) Data access • “Native” relational database support (huge DBs) EGEE-III WG • “Native” image access  gridftp driver for cfitsio Virtual Observatory: implementation of IVOA standards on top of Grid middleware  post EuroVO-DCA? • Single Sign-On • Registries (yellow pages) • VOSpace • Workflows • Data Access (see above ... but VObs-compliant) Friday 27 March 2009 - IGI - Task Force Progettazione

More Related