Datagrid Coordination Meetings 4 March 2001, Amsterdam 23 June 2001, Rome 7 October 2001, Rome

1. Datagrid Coordination Meetings4 March 2001, Amsterdam23 June 2001, Rome7 October 2001, Rome Summary for HEP-CCC Meeting 16 November 2001, CERN L. Price, ANL L. E. Price, ANL, p. 1

2. Goals of DG Coordination • Define jointly the architecture of the data grid to avoid incompatible architectures from different projects. • Ensure that tools developed by all projects will interoperate to make as much as possible a seamless data grid system for LHC and our other scientific participants. • Partition major tasks among the projects in order to use scarce resources to best advantage and avoid having each project invent only the top priority 1/3 of what we ultimately need. L. E. Price, ANL, p. 2

3. Participants L. E. Price, ANL, p. 3

4. From Paul Avery Benefits and Costs of Coordination • Benefits • Effective reach of projects • Common infrastructure • Larger effort improves visibility, attracts funding • Costs • More interactions • More meetings • More coordination • More reporting • More B.S. L. E. Price, ANL, p. 4

5. From Paul Avery Many Flavors of Joint Projects • GriPhyN, PPDG and EU DataGrid are already joint projects • Interconnected management structures • Each serving needs of multiple constituencies • GriPhyN: CS, HEP/LHC, gravity waves, Digital astronomy • PPDG: CS, HEP (LHC + current expts), Nuc. Phys., networking • DataGrid: CS, HEP, earth sensing, biology, networking L. E. Price, ANL, p. 5

6. From Richard Mount Joint Projects: The Reality L. E. Price, ANL, p. 6

7. Discussion Points Common testbeds are necessary as a means of strongly encouraging compatibility, as well as the usual practical benefits of testbeds, namely that any problem of the system in the real world will be revealed. Support of shared code base. In addition to the overlapping goals of the datagrid projects, it was noted that the projects all rely on tools developed outside of the projects, including Globus, Condor, and others. Many practical and legal issues are raised by the need to make program available to and usable by other parts of the overall datagrid community. Open source and other sharing concepts were discussed in some detail. Desirability of working toward a common or at least compatible architecture. This is so partly because the LHC experiments, at least, are connected with all of the projects and will need to use parts of each project. It also permits separately developed modules to interoperate with less effort than otherwise. L. E. Price, ANL, p. 7

8. From Ian Foster Common Architecture • A unifying structure & approach • Standard protocols, APIs, policies • Reflects pragmatic choices corresponding to best available technology • Documented (and mostly implemented) • Certainly a work in progress • Requires • Some structure (“architect”? “arch board”?) • Procedures for collecting input from the user community and guiding development L. E. Price, ANL, p. 8

9. From Fabrizio Gagliardi Main coordination issues • Compatibility of DataGrid architecture with PPDG and GriPhyN • Synchronisation of deliverables among the different projects • Common test beds across Atlantic • Globus support and development in US and EU L. E. Price, ANL, p. 9

10. Other Points of Discussion Mechanisms for coordination. Ian Foster and Carl Kesselman are already members of the EU DataGrid Architecture Task Force Coordinating council concept, consisting of management representatives from the projects. Technical steering group consisting of technical representatives of the projects. Synchronization of deliverables is also an issue, since some experiments are connected to multiple grid projects. Dedicated facilities. Dedicated bandwidth for tests, expecially across oceans. Grid Operating Center will probably be needed. L. E. Price, ANL, p. 10

11. From Satoshi Sekiguchi Ninf – ETL/TIT Is a frontier developing “Network enabled servers” Has been collaborating with NetSolve, UTK Grid RPC – APM WG Metacomputing – TACC/JAERI MPI for vectors, STAMPI ApGrid A regional testbed across the Pacific Rim Globus promotion  Firewall compliance extension Others ITBL – MEXT (ex-STA part) Tsukuba-One (DWDM ring) Super SINET IPv6 promotion North America (STARTAP) Japan Europe TransPAC (123 Mbps)  Europe South Korea server China Latin America Hawaii Hong Kong Taiwan  Thailand Philippines Malaysia client arguments Singapore Indonesia results Exchange Point Access Point Current Status Planned Numerical Libraries Applications Australia Ninf_call(FUNC, arg1, ...) Grid activities in Japan L. E. Price, ANL, p. 11

12. 2nd Meeting: Rome 23 June 2001 • Focus on organization • Relationship to CERN “LHC Grid Project” • Licensing and code sharing proposal • Ian Foster: Consortium for Open Grid Software • International Data Grid Coordination Organization L. E. Price, ANL, p. 12

13. Consortium for Open Grid Software From Ian Foster L. E. Price, ANL, p. 13

14. From Ian Foster Shared Development of Grid Software:A Proposal • Long-term success of Grid concept requires establishment of a framework for shared development of Grid software, that • Addresses legal/social issues associated with many contributors and users • Addresses governance issues • Is connected with “standards” processes • Builds credibility for open architecture, open source Grid approach • COGS: Consortium for Open Grid Software L. E. Price, ANL, p. 14

15. International Data Grid Coordination Organization • Extended discussion, resulting in following organization plan: • InterGrid Management Board • Start from the ad hoc group that has met twice • Consider small number of additions to be sure all players are represented • Board represents managements of other projects; does not manage the projects itself! • Meet 3 times per year, usually face to face • Joint Technical Board • Membership: 6 from North America; 6 from Europe; 2 Asia • Meet monthly by telephone or video conference • Face to face when needed L. E. Price, ANL, p. 15

16. 3rd Meeting: Rome 7 Oct 2001 • Update on LHC Grid Computing from Hoffmann (in person) • Agreement that LHC project will be part of HENP Intergrid Coordination group (new name) • Initial presentation of potential joint testbeds • Transatlantic testbeds or even better worldwide testbeds are considered to be the heart of the grid coordination process. Interoperability will be required for any success. • After initial coordination of grid architectures, testbeds will generally be done by groups from international physics experiments associated with the grid projects, so that practical benefit will be obtained. • Refinement of COGS plan • License draft generally agreed to with some comments • Governance and organization less clear—but considered not to be required L. E. Price, ANL, p. 16

17. Getting going on joint testbeds • Tasks: • (a)identify and agree on the initial sites to be included in the pilot testbed, with 3 or 4 sites from each side of the Atlantic; • (b)agree with the collaborations on the target applications and an initial data challenge programme; • (c)define the resources, environment, software and services to be provided at each of the component sites; • (d)specify the Grid protocols, interfaces and standards necessary to ensure inter-working and a satisfactory application environment; • (e)prepare a plan for building and operating the testbed; • (f)prepare a plan for initial usage of the testbed by selected experiments. L. E. Price, ANL, p. 17

18. Participating Data Grid Projects • PPDG • GriPhyN • DataGrid • National Projects: Italy, France, UK, Holland,…. • DataTAG • iVDGL • GridPP • LHC Grid Computing Project • Good liaison with experiments: • BaBaR, D0, ATLAS, CMS, ALICE especially L. E. Price, ANL, p. 18

19. From Ian Foster Licensing: Goals • Allow anybody to use Grid software for any purpose • No constraints on intended purpose • Can include things you don’t like! • Allow anybody to contribute to Grid sofware • Get credit for their work • As few bureaucratic barriers as possible • Two goals are not necessarily compatible! L. E. Price, ANL, p. 19

20. From Ian Foster Globus Toolkit as a Vehicle • We offer the Globus Toolkit as a vehicle for those who want to use it • Believe that there are advantages: • Strong “brand” with considerable acceptance • Considerable code base already, of core code and contributed modules • Make commitment to community to support this process as best we can L. E. Price, ANL, p. 20

21. First Meeting of Joint Technical Board: 8 Oct 2001 • Of membership of 14, 5 (+LEP) were together at Frascati, 6 were on a conference phone call, and 2 were not appointed yet. • General discussion of practical organization and goals of testbed projects. Beyond general agreement to “Keep it Simple”, there was not time to conclude the discussion. • Presentations of status, architecture plans, and near term milestones for each data grid project • Agreement to meet first Monday of each month at 7 am in California/4 pm in middle European time by phone or possibly video. L. E. Price, ANL, p. 21

22. Summary • Datagrid projects have individual identities and responsibilities to funding agencies and cannot be managed as a single project • Neverthless close coordination is important for success • The adopted compromise is to organize so the technical leaders of the different projects talk frequently (JTB) • The managements also need to work together, but meetings are less frequent (IGMB) • A legal basis for joint software development is important and being worked on. • Common testbeds are very important. L. E. Price, ANL, p. 22