1 / 19

Storage and Data Movement at FNAL

Storage and Data Movement at FNAL. D. Petravick CHEP 2003. FNAL Overall Goals. Provide competent one copy permanent store to all the Lab’s experiments. Provide scalable and performant data flows Tape and Disk Local area and Wide area.

lysa
Download Presentation

Storage and Data Movement at FNAL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Storage and Data Movement at FNAL D. Petravick CHEP 2003

  2. FNAL Overall Goals • Provide competent one copy permanent store to all the Lab’s experiments. • Provide scalable and performant data flows • Tape and Disk • Local area and Wide area. • Provide standard interfaces allowing for interoperation with other sites. • Collaborate! (DESY > 5 yrs) GLOBUS, LBL, JLAB, CERN DLP -- CHEP 2003

  3. DLP -- CHEP 2003

  4. Overview • Central Storage Systems capacious and scalable enough to be a hub of a data intensive system. • Linux as a hardware platform. • Permanent and temporary semantics. • Competent local and GRID interfaces. ENCP, dCap FTP(s) and SRM (Storage Resource Manager) investigating GLUE schema and monitoriing DLP -- CHEP 2003

  5. DLP -- CHEP 2003

  6. DLP -- CHEP 2003

  7. DLP -- CHEP 2003

  8. DLP -- CHEP 2003

  9. RunII Work Plan • SAM • Grid FTP integration with SAM. • DCAP integration • SRM integration with SAM. • SRM integration with legacy CDF AC++ • Restore rates for CDF experiment. • Provide hardware for D0 Wide area transfers. DLP -- CHEP 2003

  10. US CMS Work Plan • Use dCache as a vehicle for Grid File access locally. • Evaluate Storage elements for Tier II centers (dCache, DRM, NeST EDG wp 5 , Dfarm). • Follow, help formulate LCG requirements. • Detailed work to meet CMS Data Challenge on CMS deadlines. • Monitor and Improve Network • Pinger IEPM (co-project) • FAST and other h/p TCP stacks. (co-project) DLP -- CHEP 2003

  11. DLP -- CHEP 2003

  12. Job-Aware, Replica-Aware Mdw Replica Stores Permanent Store Direct File Access Interface SRM Inter- face FTP Inter- face Monitoring Interface Framework + job DLP -- CHEP 2003

  13. Lattice Gauge Work Plan • Support the Fermilab Lattice Computation Facility. • Use GRID FTP, SRM to integrate with • JLAB (cluster) , BNL (QCDOC) facilities. • Understand relationship of storage systems to QIO (community I/O package) • Investigate utility of caching data at FNAL. • Investigate symmetries with earth science type system. DLP -- CHEP 2003

  14. Other Work Elements • Work on Grid FTP V1.1 specification. • Include work on a few scaling issues for “seas of linux box” type systems. • Work on SRM protocol, futures. • V2.1 (co – pi @ FNAL) • Look at “Object Based Storage” in conjunction with US CMS. • Proposed investigation is root integration. • Understand the requirements of analysis systems. DLP -- CHEP 2003

  15. Other work elements… • Support Grid protocols for dFarm. • Make it a “storage element” • Another innovative package. • Experimenting with Grid Authentication. • Prepare for special routing on the WAN • E.g. lambda networking. • Support the experimental community at FNAL. • Auger in GridFTP production. • Minos in Kerberized, Weak FTP. DLP -- CHEP 2003

  16. DLP -- CHEP 2003

  17. Advanced Monitoring • Animations were Crucial for debugging The RUN II LAN based data systems. • Looking for analogous tools for grid based works • Follow items such as GLUE. DLP -- CHEP 2003

  18. Advanced Network Integration • FNAL proposes to have two paths off site. • Path to Starlight will allow R&D on advanced network concepts (e.g. lambda network) • Central data movement systems will be an early user. DLP -- CHEP 2003

  19. Summary • Very substantial and successful support of FNAL Program. • Very proactive work on fabric side. • Data in Movement are interesting. • Goal :Routine • performant flows • on LAN and WAN • with interoperation DLP -- CHEP 2003

More Related