1 / 31

GridLab 2003/4 „Steady leadership in changing times!”

GridLab 2003/4 „Steady leadership in changing times!” Jarek Nabrzyski Project Coordinator Poznan Supercomputing and Networking Center GridLab Project Funded by the EU (5+ M € ), January 2002 – December 2004 Application and Testbed oriented

paul
Download Presentation

GridLab 2003/4 „Steady leadership in changing times!”

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GridLab 2003/4„Steady leadership in changing times!” Jarek Nabrzyski Project Coordinator Poznan Supercomputing and Networking Center Meeting with Sun Microsystems at PSNC: Exploitation, 13 May 2004

  2. GridLab Project • Funded by the EU (5+ M€), January 2002 – December 2004 • Application and Testbed oriented • Cactus Code, Triana Workflow, all the other applications that want to be Grid-enabled • Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools: • Resource management (GRMS), • Data management (GDMS), • Monitoring (Mercury) and information services, • Adaptive components (Pythia), • Mobile user support and remote visualization, • Security services (GAS), • Portals (GridSphere), • ... and test them on a real testbed with real applications

  3. GridLab Members • PSNC (Poznan) - coordination • AEI (Potsdam) • ZIB (Berlin) • Univ. of Lecce • Cardiff University • Vrije Univ. (Amsterdam) • SZTAKI (Budapest) • Masaryk Univ. (Brno) • NTUA (Athens) • Sun Microsystems GmbH • HP France • ANL (Chicago, I. Foster) • ISI (LA, C.Kesselman) • UoWisconsin (M. Livny) • collaborating with: • Users! • EU Astrophysics Network, • DFN TiKSL/GriKSL • NSF ASC Project • other Grid projects • Globus, Condor, • GrADS, • PROGRESS, Clusterix • GriPhyn/iVDGL, • European Grid Projects (GRIDSTART) • KISTI, Canadian Grid, LSU…

  4. Review Issues and Concerns • It is GridLab’s first direct contact with the new PO and 2 out of 3 reviewers are new to the project, • Difficult to review a project of this size and complexity in several hours, • Decision to present the project from theintegration angle, • Problems with synchronization of deliverables with the last-minute amendment (details in the WP presentations) • Presentations: Short time per workpackage, short time for the demo, • Innovative GAT, the portal and services presented in a practical demo during SC2003 in Phoenix • The more features we add (to every single WP out of 12) the more complex the integration gets

  5. GridLab Aims • Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community, ...), • Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations • Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.

  6. What GridLab isn’t • We are not developing low level Grid infrastructure - we do more actually, we want to be infrastructure independent, • We do not want to repeat work which has already been done (want to incorporate and assimilate it …) • Globus APIs, • OGSA/OGSI/…, • ASC Portal (GridSphere/Orbiter), • GPDK, • GridPort, • DataGrid, • GriPhyn, • ...

  7. Project Goals for 2003 • Integration of all the GridLab software, • “Zakopane Scenario” driving all the developments and integration, • First release of GridLab software scheduled for December 2003, • Make the testbed more stable and ready for any demo action and/or development, • SC2003 was chosen to demo the GridLab software, • Strong dissemination of the project in order to get attention of interesting Grid applications, • Start the GAT and other GridLab standardization work under GGF, • Strengthen management of the project, with clear responsibilities to ensure better control of resources, fast identification of the risks, ensure progress, • GridLab Open Source License, • Prepare the plan for exploitation of the results, • Adjust the Technical Description of Work to the current needs of the project

  8. What have we done during last 8 months? • Response to the reviewers report sent to all • Open Source License • GT2 vs. GT3 issues (finally it was wise decision) • Strategy for dissemination in 2004 • Deliverables, some delays here (see details in the WP presentations) • Contract amendment is on the way • GridLab meeting in Eger (April) and Olomouc (October) • More developments and integration • More and more WPs brought to the scenario • Today most of the WPs were integrated with each other • SC2003 in Phoenix (2 big „demos”, many „small” demos) • Painful demo, but successful: GRMS + portal + GAT application + Testbed + WP5 information service + Adaptive with some last minute problems with other services • „The devil is in the details” - the most true sentence.

  9. Major achievements • Integration efforts and devotion of project staff, • Meeting in Eger in March/April, meeting in Olomouc in October, • Many small integration meetings (Poznan-Berlin-Potsdam, Potsdam-Budapest, Potsdam-Prague, Cardiff, GGF meetings and at other confs.), • Getting a major attention towards the GAT of many application communities, • GRIDSTART, GGF, application communities and 6FPP (EGEE, DEISA, HPC-Europa, …) • Expectations are high! • Many successful implementations of services • Single GridLab services being used by other projects, • Most of the services are fully integrated with each other, but can be configured as stand alone • GAT can connect to most of the services. All will be available in a month.

  10. Major achievement (cont.) • GridLab middleware being exploited by other projects and production testbeds • GridSphere: tens of projects (Grid Infrastructure Project, GEON, HPC-Europa, Physics Portal Development Project, …) • GRMS (Clusterix, SGI Grid, Progress, Canadian Grid) • Mercury Monitor (DataGrid, P-GRADE) • WP8 Visualization - GriKSL • Open Source License • GridSphere has it’s own Open Source license

  11. Major achievementsStandards, collaboration • Global Grid Forum • Positions in GGF • Alexander Reinefeld - GGF Advisory Board • Jarek Nabrzyski – step down from the SC and AD for Apps and now take care of the GSA RG), • Ed Seidel, Thilo Kielman chairing the Apps RG • Thilo Kileman chairing Grid CPR WG • Andre Merzky chairing the Persistent Archive WG and secretary of Data Area and Persistent Environments • GAT architecture put at the Apps and APM groups • Many WG talks about GridLab • Putting the GAT to the BOF in October 2003 • Very successful • Participation in Gridstart! • GridLab chairs at least half of the TWGs • Other • Projects: GriPhyN, GRIKSL, CrossGrid, Reality Grid (and some others) have a certain level of collaboration with GridLab • Companies: Grid Systems, Platform Computing, IBM, HP, Sun

  12. Major achievementsDissemination • Improved the www.gridlab.org • Several new papers accepted for books, journals and conferences • Many talks have been given at major conferences and events • Events organized • GridLab Conference and Workshop in Eger, March-April 2003 • Grid Portals Conference, August 2003, Edinburgh • Grid Application Workshop in Czestochowa, September 2003 • GridLab Conference and Workshop in Olomouc, October 2003 • Grid Resource Management Workshop, Poznan, October 2003 • Workshops at international conferences and symposia • Tutorials on major conferences, also via AccessGrid • 2 GridLab brochures • In 2004 we will produce 14 different GridLab brochures featuring all the services, tools and advanced scenarios (see first three issues)

  13. Book on Grid Resource Management • 4 GridLab chapters: • Application requirements for resource brokering • Multicriteria resource management (GRMS) • Triana peer-to-peer RM • Workflow management • ISI/Argonne/Wicsonsin contributions • Published in November by Kluwer Academic Publishers

  14. Technical Progress • GAT API specification and GAT adaptor developer’s guide • GAT tested by Cactus and a GAT subset implementation for Triana (GAP) • Strenghten the collaboration with ISI, ANL, Wisconsin • New Portal Framework (GridSphere) - 1st release in 2003 (2nd in 2004) - being used by tens of projects! • Participation in the Grid Technology Repository (Jarek Nabrzyski serves as the Int. Advisory Board member) • Strong participation in the GRIDSTART project (GridLab was one of the main contributors to the Technical Roadmap) • Services ready and interoperable in spite of changing requirements • GAT ready for standardization process, already under the community comment process

  15. Technical Progress (cont.) • Extending the GridLab scenario • Successful meeting in Eger (new application users involved, new developments) and in Olomouc (integration) • Problems/Issues • GAT development delayed - underestimated efforts needed for the GAT development • Should be fully ready in a month • Will be revisited and refactored! User’s input is needed. Good docs is needed. • Hiring a new person, experienced with Cactus development to speed up the CGAT integration. • GAT documentation is behind! We are shifting efforts a bit to speed up the process. • Some requirements and scenarios have changed dramatically: new ones appeared (WP7) • Some unexpected problems with hackers on one of the test machines

  16. Issues • GAT development delayed - underestimated efforts needed for the GAT development • Should be fully ready in a month • Will be revisited and refactored! User’s input is needed. Good docs is needed. • Hiring a new person, experienced with Cactus development to speed up the CGAT integration. • GAT documentation is behind! We are shifting efforts a bit to speed up the process. • Some deliverables are delayed • New Annex1 caused some problems • GAT development delayed (CGAT, GAT documentation) • Some requirements and scenarios have changed dramatically: new ones appeared (WP7) • Some unexpected problems with hackers on one of the test machines

  17. Plans for 2004 • Continue the development of GAT and services • more info in the following presentations • More complicated scenarios • collaborative environments, • submitting, controling and steering jobs from mobile devices, • more dynamic behavior of applications • Two more GridLab Workshop Meetings (Lecce: 16-22 May, Zakopane: early December) • Organizing the GridLab/GT3.2/WSRF integration meeting withUS partners (to ensure GAT compatibility) • Preparing for the Supercomputing demos • GGF BOF (GAT) and finally GGF WG • GGF Scheduling Architecture Working Group now started (GRMS) • Exploitation of the project results • Close work with GridLab’s commercial partners • Global Grid Application Alliance with GridLab’s leadership • GGF activities (long term) • GridSuit (based on GridLab plus Gridstart Open Source, PSNC plus partners) • Open Source + commercial support • European Grid Support Centre (PSNC)

  18. Summary • GAT is important • We will put yet more effort into the GAT • GAT will be revisited after finalizing the 1st release (in one month) • Refactoring • Lecce Meeting to bring application people, TWG8, tutorial, exploitation • Documentation! • GGF/Gridstart • There was a great interest and we are talking to many groups • The project works better than ever! • Project is unique in this sense (this might be an impression of ours only) • Integration was much more difficult than expected • We had to deal with many (too many) unexpected problems in 3rd party software.

  19. Collaboration with ISI

  20. Collaboration with Argonne • OGSA C bindings • GridLab/GT3.2 Inegration Workshop (4 Argonne GT3 developers/experts are coming to help us with the integration) • Help with developing the GT3.2 adaptors • GAT review

  21. Collaboration with Condor Group

  22. Condor cont.

  23. Exploitation Plan Jarek Nabrzyski Meeting with Sun Microsystems at PSNC: Exploitation, 13 May 2004

  24. Pillars of exploitation • Close work with GridLab’s commercial partners • Global Grid Application Alliance with GridLab’s leadership • GGF activities (long term) • GridSuite (based on GridLab plus Gridstart Open Source, PSNC plus partners) • European Grid Support Centre (PSNC, HP) • European Grid Technology Broker

  25. Exploitation of the results through commercial partners • HP • Grid Resource Toolkit. HP investigates which GL software can be adopted. • Distribution through server/software distribution • Sun • This meeting! • Actions to be taken: • GridLab will work closely with the companies to ensure that the companies evaluate the software and make right decisions on which software is supposed to be part of their distributions • Responsible: WP managers will provide software plus documentation to a contact person at the vendor side.

  26. Global Grid Application Alliance • Global • Europe - GridLab • US - Ed Seidel • Asia Pacific - both Satoshis • Korea - KISTI • Tasks • Bring the GAT and services to the community • Train people to make then use the GAT • People will write adaptors for their favorite services • Work on the new scenarios that the Alliance will bring • Work on portable applications on a global scale using the GridLab technologies • Actions: • The initial meeting of the founders took already place • SC will be responsible for the actions: meetings soon to be scheduled

  27. GGF, GRIDSTART activities • SAGA • GSA • and more will be continued beyond the timeframe of the project • Gridstart TWGs will be used to work with other Grid projects in order to deploy the GAT and GridLab tools in these projects • TWG chairs (from GridLab) are responsible to make it happen (GridLab WP managers, SC members)

  28. GridSuite at PSNC • GridSuite (based on GridLab plus Gridstart open source, PSNC plus partners) - Grid out-of-the-box complete solution • Open Source + commercial support • PSNC has declared to hire a team of 6-8 persons to build the GridSuite this year • PSNC will work with GridLab partners to provide an expertise to build the GridSuite and to design a business plan for the effort. • Release roadmap is under way. • This might be a background for the commercial company selling the Grid services for European industry • Actions to be taken: • PSNC will discuss this with the GridLab Steering Committee soon, • This will be more open to other GridLab partners (not limited to PSNC) • Strategy is being developed by PSNC now • Initial money from the European infrastructure funds (under way) and PSNC’s own funds.

  29. European Grid Support Centre • European Grid Support Centre (HP, PSNC plus UK eScience, CERN, PDC) • Institutions’ own funds • The task is to support different user groups (currently DataGrid, EGEE, GridLab, NorduGrid) • The tools and the procedures are now being established. • The ground for GridLab’s support for its users and infrastructure builders, • Currently PSNC and HP from GridLab only. More will be involved with increase of the number of users. • Actions: • Build the procedures and assure that GridLab user’s know about the Center and use its services, • To be discussed by the SC in more details in 6 weeks time, • Extension of the group with a time and load (currently one person half-time at PSNC, similarly at HP, 3-4 persons at PSNC soon).

  30. Exploitation by partners - general strategy • Every GL partner will work with the users that the SC agrees to work with closely, • In such a case particular institution will make sure that appropriate resources are assigned to work with the user in order to deploy GridLab software in the user’s environment, • The appropriate resolutions will be worked out in details and decided by the Steering Committee by the existence of the GAT release, no later than by the end of April. • At the beginning we will focus on strategic users to ensure the added value to the project. • We will work closely with the GridLab application communities first of all (Cactus and Triana) on the GAT usage.

  31. Beyond the project • Seek for local funding by every partner • Survive the gap after the GridLab • Continue work, bring more users, get new requirements, analyze scenarios, work with the GGAA (Global Grid Application Alliance)

More Related