Global Scientific Collaborations and Grids: A View from Fermilab

1. Global Scientific Collaborations and Grids: A View from Fermilab

2. The 3 Top issues for Particle Physics today (1) Collaboration (2) Collaboration (3) Collaboration • Global within the field - needed for every stage of an experiment • to build the detector and the software, operate and monitor the experiment, produce Monte Carlo data, process data, analyze data, participate in the physics • With other sciences – for Funding need recognition and partnerships outside our field. Also to train the next generation of scientists. • With Educators and Communicators – For public education and to make a difference in society • This is needed worldwide – on both sides of the digital divide Vicky White

3. The Fermilab Experimental Program Statistics “Active” experiments from the 2003 Research Program Workbook • Of 213 Institutions involved 114 of them are non-US • Of 1916 physicists, 753 are non-US • Of 699 students, 234 are non-US Vicky White

4. Significant Contributions to the Fermilab Program from the “other side” of the digital divide • Brasil • Argentina • Colombia • Mexico • Czech Republic • Taiwan • Russia • China • Korea • India • …. and others Vicky White

5. Highlighting Brasil (1) History of truly significant contributions to Fermilab Experiments since 1983 • Series of charm and heavy flavor expts • E791 was ahead of its time in volume of data • Analyzed about 10% of E791 data in Rio - one of only 4 institutions   - Ohio, Mississipi, Fermilab, CBPF • Significant contribution to the physics - data led to the 1st publication for E791. • Focus • Selex CBPF/LAFEX + Fermilab hosted CHEP95 – pushing networking and videoconferencing technology Vicky White

6. Highlighting Brasil (2) • Most recently big contributions to the D0 experiment • Designed and built the POTS for the Forward Proton Detector giving the D0 expt a new physics reach • Physics Leadership – far beyond the size of the effort • Leon Lederman, Fermilab Director Emeritus, has throughout provided support and encouragement for Physics in South America Vicky White

7. IT Technology and Physics • Network connectivity and IT infrastructure must be available at some minimal level in order to participate in a Physics Experiment. • Being on the wrong side of the digital divide is painful • We need to help ensure that governments understand • Just how important this contribution is to the Science & • THAT HIGH ENERGY PHYSICS IS IN THE FOREFRONT OF SHOWING HOW GLOBAL E-SCIENCE CAN BE CARRIED OUT AND OF DEMONSTRATING THE USE OF TECHNOLOGIES AND BEHAVIORS THAT WILL PROFOUNDLY AFFECT EDUCATION AND COMMERCE AND SOCIETY. Vicky White

8. Fermilab Ongoing Scientific Program, 2004 • Run II (CDF, D0, Accelerator. Upgrades) *** • MINOS * • MiniBoone • Sloan Digital Sky Survey ** • CMS ***** • Lattice QCD Theory collaboration ** • Pierre Auger and CDMS • Test Beams • BTeV in pre-project stage, preparing for review **** • SNAP, Off-axis neutrinos and a few other efforts in R&D, LOI stages *represents Large Global Collaboration/Petabyte Datasets/Massive distributed computational power needed And R&D for the Linear Collider – a worldwide effort we assume Vicky White

9. Inclusive Worldwide Collaboration is essential for Particle Physics What are we doing at Fermilab to help this ? • Networks and Network research • Grids -- lots on this – important strategically • Guest Scientist program • Education and Outreach program • Experiment sociology and leadership – changes • Videoconferencing • Virtual Control Rooms • Physics Analysis Center for CMS • Public Relations Vicky White

10. Networks • Starting 1 year ago Fermilab moved aggressively to make a plan for ~3 GigE connectivity in the (fairly near) future • Dark Fiber to Starlight (contract just about in place finally) • Worked on Office of Science Strategic Networking plan • Expect Metropolitan Area Network (MAN) in Chicago area linking ANL, Fermilab and Starlight in a ring • Public Planning processes and reports are important – e.g. TAN report, ICFA SCIC • We try to participate whenever we are invited to Vicky White

11. ESnet in 2003: OC192 and OC48 Links Coming Into Service;Consider Links to US HENP Labs; Evolution Not Sufficient Vicky White

12. ESNet proposal for Chicago area MAN (Metropolitan Area Network) Vicky White

13.   Vicky White

14. Network Research • Also in the past year Fermilab chose to become a beginning player in the Network research arena • As Harvey Newman says often raw bandwidth is not enough – we need to learn how to use it, monitor the network, look for research partners and opportunities • Joining with several partners on proposal for UltraNet (DOE research network) exploitation and with Caltech et. al. on Ultralight program of work/proposal • Some international opportunities for broader collaborations • UK, CA, Czech, Netherlands, DataTag, ? • Research agenda pushes provisioning of network Vicky White

15. Network Monitoring • Modest effort to help with Les Cottrell’s IEPM and Pinger projects • Modest efforts by Fermilab network group to help track down network bottlenecks and problems Vicky White

16. Grids for Science • There is no doubt that computing resources for Particle Physics are going to be distributed globally and shared between “Virtual Organizations” • Happening already with Run II, Babar, RHIC and LHC experiments Vicky White

17. LHC: Key Driver for Grids • Complexity: Millions of individual detector channels • Scale: PetaOps (CPU), 100s of Petabytes (Data) • Distribution: Global distribution of people & resources 2000+ Physicists 159 Institutes 36 Countries CMS Collaboration Vicky White

18. Grid Projects and Coordinating Bodies Fermilab is involved in numerous Grid projects and coordination bodies • PPDG, GriPhyN and iVDGL - the Trillium US Grid projects • LCG - GDB, SC2, (PEB until recently), bodies and working groups on Security, Operations Center, Rollout, etc. • SRM – storage systems standards • Joint Technical Board and Interoperability work • Global Grid Forum – Physics Research Area • Global Grid Forum – Security Research Area • Open Science Grid – meta-level collaboration and coordination • And probably some I forgot….. Vicky White

19. Fermilab Grid Strategy • Strategy (from a Fermi-centric view) • Common approaches and technologies across our entire portfolio of experiments and projects • Strategy (from a global HEP view) • Work towards common standards and interoperability • At Fermilab we are working aggressively to put all of our Computational and Storage Fabric on “the Grid” and to contribute constructively to interoperability of Grid middleware Vicky White

20. Successes – D0 and CDF • SAM-GRID – a functional Grid system developed for D0 and now in use by CDF for Data Storage with meta-data, Data Delivery and caching, Job Execution (JIM) and Monitoring • SAM “stations” worldwide • Lots of real operational experience and solutions for robustness and scalability • Evolving to use Globus and Condor based “standards” • Supports GridFTP + other storage access/file Xfr Vicky White

21. Regional Center Analysis site SAM-GRID System at D0 Integrated Files Consumed vs Month (DØ) Summary of Resources (DØ) 4.0 M Files Consumed Integrated GB Consumed vs Month (DØ) 1.2 PB Consumed 3/03 3/02 Vicky White

22. SAM at a Glance – D0 and CDF • http://d0db.fnal.gov/sam_local/SamAtAGlance/ • Includes Stations in U.S., Czech Republic, India, France, UK, Netherlands, Germany, Canada • Monitored • Drill down to display jobs running and files being delivered • http://cdfdb.fnal.gov/sam_local/SamAtAGlance/ Vicky White

23. Successes - US-CMS Vicky White

24. Grid2003: An Operational Grid - Huge Success ! • 28 sites (2100-2800 CPUs) • 400-1100 concurrent jobs • 10 applications • Running since October 2003 Korea Vicky White http://www.ivdgl.org/grid2003

25. Grid2003: A Collaborative Effort • Trillium Grid projects • PPDG + GriPhyN + iVDGL • (US-ATLAS, US-CMS, BTEV, LIGO, SDSS, Computer Science) • US-ATLAS and US-CMS projects • Fermilab, LBL, Argonne • U. New Mexico, U. Texas Arlington • Korean site • Kyungpook National University (CMS) • New sites • University of Buffalo (CCR) Vicky White

26. Grid2003: 3 Months Usage Vicky White

27. Open Science Grid • Goals (http://www.opensciencegrid.org/) • Support US-LHC research program, other scientific efforts • Federate with LCG and peer with EGEE in Europe • Involve laboratories (DOE) and universities (NSF) • Getting there: Grid2004 (OSG-1), Grid2005 (OSG-2), … • Series of releases  increasing functionality & scale • Persistent Production quality Grid + Grid Laboratory • Jan. 12 meeting in Chicago • >60 participants from labs and universities, Computer Science and Physics and Biology, and Grid Projects Vicky White

28. Federating Grids As the US representative to the LHC Computing Grid Grid Deployment Board (LCG GDB) I have repeatedly pushed the case for the LHC Computing Grid to be a “Grid of Grids” – not a distributed Computing center • Resources must be able to belong to >1 Grid • E.g. NorduGrid, UK Grid, Open Science Grid, TeraGrid, Region XYZ Grid, University ABC Grid and Fermilab Grid! • Grids must learn to interoperate • Governance, policies, security, resource usage and service level agreements must be considered at many levels including at the level of a Federation of Grids Vicky White

29. Inclusive Worldwide Collaboration is essential for Particle Physics What are we doing at Fermilab and in Fermilab experiments to help this ? • Networks and Network research • Grids -- lots on this – important strategically • Guest Scientist program • Education and Outreach program • Experiment sociology and leadership – changes • Videoconferencing • Virtual Control Rooms • Physics Analysis Center for CMS • Public Relations Vicky White

30. Support for Guest Scientists and Engineers • Although budgets are tight Fermilab (and especially the Computing Division) wants to encourage and support a guest scientist and guest engineer/software engineer program • Tremendously beneficial to the expts • Tremendously important for ongoing collaboration and communication Vicky White

31. Education • Quarknet meets “Grid” • Traditional K-12 Education Program at Fermilab is working with Grid Projects and Computing Division on the educational opportunities afforded by the Grid • Open Science Grid – will have a significant Educational component. Vicky White

32. Sociology and Leadership changes • Increasing awareness at Fermilab of the need to include those outside Fermilab • Meeting times to take account of time zones • Videoconferencing • Posting materials and documenting • Leaders not resident at Fermilab Vicky White

33. Videoconferencing • We do more and more at Fermilab each year – 5 new fully equipped conference rooms last year • Total of 19 rooms - all Polycom based • Fermilab is instituting better “official” support for VRVS and H323-based conferencing. • New server v. soon • Expect to see growth in desktop video in next 2 years • User Requirements meeting soon – to plan next steps Vicky White

34. Virtual Control Room • MINOS has one at Fermilab for a detector at the Sudan Mine in Minnesota • Control Room Logbook • US CMS is going to build one at Fermilab • Now we in the US are the ones who will be far from the apparatus and the action! • We hope our work in the next few years will be broad reaching and benefit all who are “far from the experiment”. Vicky White

35. CMS Physics Analysis Center(s) in the U.S. • Serious thought going into this now. What is a PAC? • We can’t all go to CERN, all the time, to do physics • Must empower physicists to do analysis away from CERN • Technology is important – networks, videoconferencing, access to data • But there is more • What do we need at Fermilab to have a vibrant center for physics where people come to work together both physically and virtually? • The effort we put into this may have far reaching effects on the “democratization of science”. Vicky White

36. Public Relations • Judy Jackson, head of Public Affairs at Fermilab, works closely with many other public affairs groups – at CERN, SLAC and elsewhere • Interactions.org website • Forming a Strategic Grid Communications group – presented a plan for this at a recent Open Science Grid meeting Vicky White

37. Role of Science in the Information Society Fermilab/SLAC exhibit from SC2003 on Networking/Grids/Storage/Simulations went to the RSIS exhibition in Geneva recently Vicky White

38. Inclusive Worldwide Collaboration is essential for Particle Physics What are we doing at Fermilab to help this ? • Networks and Network research • Grids -- lots on this – important strategically • Guest Scientist program • Education and Outreach program • Experiment sociology and leadership – changes • Videoconferencing • Virtual Control Rooms • Physics Analysis Center for CMS • Public Relations Vicky White

39. IT Technology and Physics • Network connectivity and IT infrastructure must be available at some minimal level in order to participate in a Physics Experiment. • Being on the wrong side of the digital divide is painful • We need to help ensure that governments everyone understands • Just how important this contribution is to the Science & • THAT HIGH ENERGY PHYSICS IS IN THE FOREFRONT OF SHOWING HOW GLOBAL E-SCIENCE CAN BE CARRIED OUT AND OF DEMONSTRATING THE USE OF TECHNOLOGIES AND BEHAVIORS THAT WILL PROFOUNDLY AFFECT EDUCATION AND COMMERCE AND SOCIETY. Vicky White

40. Conclusions • We must keep working on all fronts to collaborate, educate, make the case for Science and for the Information Technology and Networking improvements that are essential for success – everywhere • in particular to ensure that the Digital Divide does not exclude physicists. Vicky White