Components of Grid Development Middleware Resource Integration Connectivity

1. Overview • Components of Grid Development • Middleware • Resource Integration • Connectivity • Acceptance of Grid in the Community • Sustainability

2. Components of Grid Development • Middleware • Complementary development and extention of Middleware stacks from DGI and Communities • Starting point is (mostly) Globus 4.x • Open Source based and oriented • Integrative towards community specific tailored middleware (based on globus) (HEP) und specialized middleware (Unicore)

3. Components of Grid Development • Resource integration • 1 Level: National integration • Unified Certification Policies • Building of an efficient certification chain • for users • for services • Service Certification is still not sufficient • for hardware (Hosts) it is sufficient • for Services the Certification (Webservices, Compute-Services etc) no unified system • different approaches not integrated (Shibboleth) • VO Management still not available on national level

4. Components of Grid Development • Resource Integration • Level 1: National Integration Obstacles: • Resource-providers in scientific community face legal ond procedural obstacles which hamper the grid way provding resources with Vos, especially the SCC • Community-Resources are more flexible in that aspect In the long run we need a change of these procedures and substitute them with grid-based usage policies (establishing new comittees for eg. allocating resources to a science project)

5. Components of Grid Development • Resource Integration • Level 2: Grid-Middleware Accounting and Billing procedures are still not available Reasons: • different community procedures and requirements • software for these procedures still in concept phase • resource integration into a functional grid only now comes into being Job monitoring and quality assessments for QoS agreements for resource usage is one of the crucial steps in the next year, as a base for even raising the question of resource allocation procedures toward grid ways • BMBF sponsored considerable hardware resources, which provide a great opportunity for progress, since there are no legal (and local) restrictions imposed

6. Components of Grid Development • Connectivity The Grid is hungry for bandwidth • Useability of grid integrated resources is very limited without availability of sufficient network connectivity • Requirement for scientific communities is: • Centers of Grid development are the nodes of DGI amd Communities • HEP-Community already has a network concept for the LHC-Experiment • at least connect all BMBF hardware nodes with 10GBit/s • (Example BaWü: for science institutions a countrywide Gbit network without fees)

7. Components of Grid Development • Connectivity • For QoS of grid services with respect to network connectivity no middleware stack has any component to offer • Without incorporating QoS for network connectivity accounting/billing is only partially possible

8. Components of Grid Development • Improving of acceptance of the Grid is one of the most important tastks of the Community grids • Getting scientific communites to use the grid infrastructure for their day to day work is one of the most important contibutions towards the sustainability • Grid integration of huge projects is a pilot towards this goal • Example: LOFAR • Building Institute resource grids and VOs, establishing resource sharing on that level is one of the roads • Getting more associated partners into Community projects by offering support for middleware installation to get them started • Collaboration on international level with other projects within the communities • Example : IVOA / GAVO • Example : GAIA • Example : Ukraine Astrophysics Grid

9. Resource Integration: Next steps • Get the resources in a useable state: • Deploy our VOMRS on all resources, have all local RA use the tool • Ensure a regular update of the gridmaps on all grid resources • Run a daily health-checker on each resource that’s in MDS to report the running services, the state of the local gridmaps and if possible, the gridusers and their jobs • Start an issue tracking module for the resource providers to report on problems with the grid machines, establish a procedure for resolving conflicts with gridjobs and gridusers • Draft a set of policies for grid resource usage • Basic policy: all AstroGrid-D grid resources are available to all our VO members • Exceptions are ok, but only for specialized resources (eg don’t try to run a taskfarming job on a cluster, or a program without gravitational interaction on a cluster with GRAPE boards) • Local users run with a higher priority than grid users

10. Resource Integration: Next steps Special for clusters: • Incorporate more resources from AstroGrid-D, especially the pending clusters • Start to define default libraries and environments for the clusters (mpirun etc), document standards for the different local queue-managers and policies • Provide above information via commandline facilities as well as hardware info etc, incorporate queue status information Special for workstations: • Provide owners (main user) with a means to suspend grid jobs for some time and bar starting new ones if this state is signaled • Get this suspended status communicated to the information service • Add additional security related monitoring of globus ports wrt security and attacks

11. Resource Integration: Next steps BMBF Hardware for our Nodes: For ZAH and Potsdam already agreed: Installation of same OS (Scientific Linux 4.3 or 4.4) Source installation of Globus 4.03 Start an admin group for these resources across institute borders Ensure the participation of the community already in the planning stages of the integration of all middleware stacks to ensure a forward solution