A framework for Grid users, operators and developers Harald Kornmayer

1. A framework for Grid users, operators and developersHarald Kornmayer (Forschungszentrum Karlsruhe) on behalf of the g-Eclipse consortium 28th of march 2007 ISGC 2007, Taipei, Taiwan

2. Outline • Grid infrastructures • The Grid and their users • Grid application life cycle • The g-Eclipse project • Idea • project • First results • Roadmap • Sustainability • Conclusion and Outlook

3. e-Science infrastructure • Grid infrastructure are emerging • National level • D-Grid • AustrianGrid • …. • European level • EGEE • Deisa • … • worldwide • EGEE • LCG computing Grid • … i.e EGEE infrastructure in Europe

4. The reality for the user • Many application domains start using Grid infrastructures • But… • Grid technology is complex • Different systems are used • Middleware (Unicore, gLite, Globus) • Installation (rpm, tar, Quattor, …) • Monitoring • Different programming paradigms • Batch type systems vs. service oriented systems • Many programming languages  The threshold is to high for the “standard” user! 

5. Can we learn from others? • Are there other distributed systems supported by different middleware systems? J2EE • (enterprise beans, JMS, web services, …) • Based on specifications • implemented by different Vendors • Websphere, BEA, Oracle, JBoss, Geronimo, … • Customers don’t want to be vendor dependent • Use a generic middleware independent development tool  Web Tools project (WTP) at Eclipse.org • www.eclipse.org/webtools • Wizards, Editors, etc to simplify the development of Web applications • Independent on the underlying infrastructure

6. Grid application life cycle • In most cases, e-Scienists have their application(s) • Legacy code written in different languages (FORTRAN, C, C++, …) • e-Scientist wants to collaborate • A Virtual Organisation is build around a Virtual Computing Center on existing (and new) infrastructure • e-Scientist creates a Grid project • A project, not funding!!  • e-Scientist interacts with the Grid without knowing all details!!(development, deployment, testing, management, …) •  Tooling is necessary!! • Wizards, Editors, … • Hide the complexity!!

7. Grid project Replica Manager Resource Broker Programming languages Computing Storage Applications Infrastructure Grid project Domain services Batch apps Infrastructure Providers Policies Interactive apps Virtual Organisations Service Providers Workflows Members Web services Roles

8. Grid user roles & use cases • 3 different roles • user • operator • developer • in general... • job management • resource management • files • Applications • hardware • ... • “user centric approach” needed

9. g-Eclipse – the idea • Grid infrastructure are complex systems • Provide tools to hide the complexity • Users want easy access to the system • Users act within different roles • Grid applications users • Grid resources providers and operators • Grid application developers • Provide a general UI framework/eco system for the different Grid actors based on a reliable platform • (re-)use Eclipse and contribute! • gain OS independence (by using JAVA!)

10. Eclipse • Eclipse is an eco system • Build for extension • more than a JAVA IDE • Java IDE  JAVA plattform  platform  eco system • Support for C/C++, Fortran, Parallel programs, SOA, J2EE, … • Eclipse is a agile community • Vendor independent • Strong support form business • (SAP, Oracle, Google, IBM, …) • 10 projects (including > 50 subprojects) • The biggest “coordinated” open source project • Synchronization of > 10 project in one release per year! • Eclipse 3.3 Europa release

11. The g-Eclipse project • Funded by European Union • 24 months, approx. 2 M€ funding • STREP • 6 Partners • Coordinator: Forschungszentrum Karlsruhe • More Information: http://www.geclipse.eu Objectives • Integration of existing Grid tools • Exemplary support for the widely deployed gLite middleware during the first project year • selection of a second middleware in year 2 • Provision of the g-Eclipse framework for other EC Grid projects • Integration support for third party developments • Fostering an open source project within the Eclipse community University of Cyprus

12. g-Eclipse Architecture g-Eclipse VOMS UI VOMS Authent. Impl. gLite Middleware UI gLite Middleware Resource Management Implementation Authentication UI Resource Management UI Grid Authentication Core Grid Resource Management Core Job Management Res. Management Eclipse Platform

13. User Interface contributions Resource Editor Authentication Token View Job Editor Site Configuration Editor Site View Job View Grid Workflow Editor Resource View Web View User Perspective Operator Perspective Terminal View Developer Perspective Authentication Preference Page Project Wizard Job Wizard VO Preference Page Authentication Token Wizard Project Preference Page

14. Screenshots I Grid project view Cheat Sheets Editor Auth Token View Glue Info view VOMS Web view

15. Screenshots II Grid project view Terminal view Project with Legacy code Grid project Remote Grid File system Job List view User Support Web view Open the file with a click Grid Jobs File Resource of the VO

16. Screenshots III Grid debugging • Remote debugging don’t look different from local debugging

17. Eyecatcher • Visualisation on the Grid with GVid

18. Outlook - Roadmap 2007-01 Milestone 0.1-M1 authentication, job, file, resource management exemplary support for gLite 2007-06 Version 0.5 core features improvements support for gLite first review 2007-12 Milestone 1.0-M1 core features improvements support for second middleware 2008-06 Version 1.0 final release with core features support for second middleware second review • for each of the releases separate milestones will be defined on 4-6 weekly periods

19. Obstacles so far gLite • Implementations run only on the EGEE standard platform • Missing JAVA APIs or Web service implementations • VOMS • LFC (Logical file catalog) • Workload management system (WMS) • Sometimes strange error messages (“ … shit … “ ) • No defined process to contribute!

20. Sustainability • g-Eclipse is accepted as a project of the Eclipse foundation • Following the Eclipse procedures and rules • to ensure quality, openness and transparency • based on meritocracy • the more one contributes to Eclipse, the more responsibility one earns • Meritocracy gives companies/partners who bring more developers to a project more say in the direction of that project. • Including licensing issues • Who checks code infections in Apache? • g-Eclipse is gathering community • In the Grid and Eclipse community • Contribute! • By using g-Eclipse • Reading open mailing lists (geclipse-dev@eclipse.org) • submitting bugs • By sending patches • By getting a g-Eclipse committer

21. Summary • g-Eclipse project will build an Eclipse based user framework for Grid infrastructure • Support for the “Grid application life cycle” • First g-Eclipse milestone releases are available! • See the webpages for more details • www.geclipse.eu or www.eclipse.org/geclipse • Still work in progress • g-Eclipse is using EGEE infrastructure • g-Eclipse will support a second middleware in 2008 • g-Eclipse is open for contributions • Integration of existing tools • new middleware implementations • g-Eclipse will benefit from other Eclipse projects