1 / 21

DØ Grid PP Plans – SAM, Grid, Ceiling Wax and Things

DØ Grid PP Plans – SAM, Grid, Ceiling Wax and Things. Iain Bertram Lancaster University Monday 5 November 2001. SAM, DØ, and the Grid. DØ Basics What is SAM? History Current Deployment Collaborators SAM, Grid and Future Developments Overview of Plans UK Plans CDF and SAM.

vivian
Download Presentation

DØ Grid PP Plans – SAM, Grid, Ceiling Wax and Things

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DØ Grid PP Plans –SAM, Grid, Ceiling Wax and Things Iain Bertram Lancaster University Monday 5 November 2001

  2. SAM, DØ, and the Grid • DØ Basics • What is SAM? • History • Current Deployment • Collaborators • SAM, Grid and Future Developments • Overview of Plans • UK Plans • CDF and SAM Iain A Bertram - Lancaster

  3. The DØ Experiment • Detector Data • 1,000,000 Channels • Event size 250KB • Event rate ~50 Hz • On-line Data Rate 12 MBps • Est. 2 year totals (incl Processing and analysis): • 1 x 109 events • ~0.5 PB • Monte Carlo Data • 5 remote processing centers • Estimate ~300 TB in 2 years. Iain A Bertram - Lancaster

  4. ~500 Physicists 72 institutions 18 Countries Collaboration Iain A Bertram - Lancaster

  5. Iain A Bertram - Lancaster

  6. SAM (DØ & FNAL Project) • SAM is Sequential Access to data via Meta-data • Project started in 1997 to handle DØ ’s needs for Run II data system. • SAM is a data-grid • No fully functional Grid currently exists • SAM does have many GRID functionalities • Stations – logical collection of computers, networks, storage • Transparent access and transport of data between stations • Data Cataloguing – Replica Management • Fabric management • Job Submission on Local Station Only http://d0db.fnal.gov/sam Iain A Bertram - Lancaster

  7. Deployment Iain A Bertram - Lancaster

  8. Deployment II WAN LAN Datalogger MSS MSS Reco-farm Central Analysis ClueD0 (Others) • Current active stations • FNAL (several) • Lyon FR (IN2P3), • Amsterdam NL (NIKHEF) • Lancaster UK • Imperial College UK • Others in US Stations at FNAL MSS Interconnected network of primary cache stations Communicating and replicating data where it is needed. Iain A Bertram - Lancaster

  9. Statistics 25TB • Number of registered users: 360 • Data in the system: 25 TB • 160k files • Acessing > 3 TBa day • Goal 13 TB/day • Fully integrated into DØ Analysis Framewrok Iain A Bertram - Lancaster

  10. SAM Collaborators • PPDG - Particle Physics Data Grid • DØ Participation • Condor • Globus • Fermilab CMS Computing Group • iVDGL International Virtual Data-Grid Laboratory. • IGMB – InterGrid Management Board Iain A Bertram - Lancaster

  11. Future Plans SAM is an operational GRID • Ideal platform for demonstrating Grid technologies on the time scale of 2 years • Modular design allows integration of modern Grid Tools • Ideal Testing Ground for LHC scale experiments • Full Scale Test of Grid Middleware Iain A Bertram - Lancaster

  12. DØ Goals • DØ is fully committed to making SAM a fully functional GRID on the timescale of 2 years. • DØ is committed to using standard GRID tools wherever possible. • DØ has/is committing significant resources to the GRID. Iain A Bertram - Lancaster

  13. General Goals • Use of standard middleware to promote interoperability • Globus Security infrastructure, Interoperability with Fermilab Kerberos security infrastructure • GridFTP as one of the supported file transfer protocols • Globus job submission. • Condor and extensions job submission • Publish availability and status of SAM station resources • Publish catalog of data files and their replicas using standard or standards emerging from PPDG and DataGrid Iain A Bertram - Lancaster

  14. General Goals II • Additional Grid functionality for Job specification, submission and tracking. • Use of full Condor services for migration and checkpointing of jobs – as much as is possible with DØ software and the DØ software framework. This may require work on both Condor software to achieve full functionality • Building incrementally enhanced Job specification language and job submission services that ensure co-location of job execution and data files and reliably execute a chain of job processing, with dependencies between job steps. The first step in this is expected to be work in conjunction with the Condor team to provide for specification and execution of a Directed Acyclic Graph of jobs using an extended version of the DAGMAN product that CMS is testing for their MC job execution. Iain A Bertram - Lancaster

  15. General Goals III • Enhancing Monitoring and Diagnostic capabilities • Extensions to existing system of logging all activities in the system to both local and central log files - as demanded by robustness and increased use of system. • Incorporation of emerging Grid Monitoring Architecture and monitoring tools. Little exists on this at this point and this work will involve working with other Grid projects and participating in Global Grid Forum working groups Iain A Bertram - Lancaster

  16. Proposed Applications • Monte Carlo Production System • To upgrade the distributed Monte Carlo production system as a short term use case for demonstrating, in an incremental fashion, essential components of the Grid. In particular this involves demonstrating transparent job submission to a number of DØ processing centres (SAM stations), reliable execution of jobs, transparent data access and data storage and an interface for users to understand and monitor the state of their MC requests and the resultant MC jobs and data that satisfy their requests Iain A Bertram - Lancaster

  17. Proposed Applications • General User Applications • To demonstrate analysis of data products (both MC and Detector Data) on desktop systems in the UK, using an enhanced version of the DØ SAM Grid system that incrementally incorporates Grid middleware components. This will not only demonstrate active use of a Grid for analysis but will also eventually demonstrate interoperability between the DØ Grid and other emerging Grid testbeds. Iain A Bertram - Lancaster

  18. Milestones Iain A Bertram - Lancaster

  19. CDF and SAM • Rick St Denis Talk • CDF to use SAM for data access • UK setting up test facilities • Combined DØ and CDF Proposal Iain A Bertram - Lancaster

  20. Proposal • Request 6 FTEs • Four of the additional FTEs will work on integrating CORE Grid functionality into SAM (this talk) • One FTE for DØ applications (MC and Analysis) • One FTE integrate CDF software with SAM Iain A Bertram - Lancaster

  21. Conclusions • SAM is an operational GRID • Offers great opportunity for testing GRID Middleware • Goal: Fully functional test-bed in two years. Iain A Bertram - Lancaster

More Related