1 / 14

Xrootd Storage @ SLAC a brief summary for users Wei Yang 2008-07-23 SLAC ATLAS forum

Xrootd Storage @ SLAC a brief summary for users Wei Yang 2008-07-23 SLAC ATLAS forum. How Xrootd works Glue file servers together by a redirector User only need to know XROOT path: root://redirector:port//path/file Simple, low overhead No complex features such as locking

pryce
Download Presentation

Xrootd Storage @ SLAC a brief summary for users Wei Yang 2008-07-23 SLAC ATLAS forum

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Xrootd Storage @ SLAC a brief summary for users Wei Yang 2008-07-23 SLAC ATLAS forum

  2. How Xrootd works Glue file servers together by a redirector • User only need to know XROOT path: root://redirector:port//path/file Simple, low overhead • No complex features such as locking • Good for reading dominated environment, e.g. HEP data analysis /xrd/file1 /xrd/file2 /xrd/file3 redirector I want /xrd/file2 Time out… Who has /xrd/file2 Waiting for a reply from data nodes I have /xrd/file2 /xrd/fileN Please talk to data node 2 data node 1,2,3,…,N Nobody has /xrd/file2

  3. User interface to Xrootd • TXNetFile class (C++) • Fault tolerance • High performance thought intelligent logics in TXNetFile and server • Command line tools • xrdcp simple, native, light weight, high performance • Posix preload library • export LD_PRELOAD=/…/libXrdPosixPreload.so • ls/cat/cp root://redirector:port//path/file • A subset of UNIX I/O command will work with Posix preload library • on files, not on directories • Some overhead, I/O performance isn’t as good as xrdcp

  4. User interface to Xrootd, cont’d • XrootdFS • File system interface for Xrootd • Mount the Xrootd cluster on client host’s local file system tree • Provide standard Posix I/O interface to the Xrootd cluster • open(), close(), read(), write(), lseek(), unlink(), rename() • opendir(), closedir(), readdir(), mkdir() • Work with most UNIX commands/tools • cd, ls, cp, rm, mkdir, cat, grep, find • ssh/sftp server, gridftp server, SRM, xrootd server • scp/sftp, gridftp clients, SRM clients, dq2 clients • Be aware: no file locking, no ownership/protection • file creation delay • some UNIX command are not scalable, e.g. find, ls • cp is slow (due to small I/O block size) • More overhead on I/O performance

  5. GRID interface for Xrootd • Grid Storage Element (SE) is not clearly defined by Globus • Currently LCG and OSG define SE interface • GridFTP • FTP client/server based on GSI authentication • A GridFTP plug-in module for Xrootd was developed at SLAC • Storage Resource Manager (SRM) • Many functions. Mainly used as a load balancer for GridFTPs • Berkeley Storage Manager (BeStMan) • A trim down, high performance SRM, BeStMan-xrootd, is in • production at 3 US ATLAS Tier 2 sites. • Backend storage used by ATLAS: Castor, dCache, DPM, Xrootd, others • dq2 clients use GridFTP and SRM client tools to access remote data

  6. Bestman-xrootd implementation Implemented or partially implemented • srmLs() srmRm() srmMkdir() srmRmdir() • srmPrepareToPut() srmPrepareToGet() • Support WLCG static space token • srmGetSpaceTokens() Bestman-xrootd itself does NOT transfer data • Rely on PtP() and PtG() and GridFTP servers to transfer data No request queuing • No srmStatusOfPut() and srmStatusOfGet()

  7. How srmPreparToPut and srmPrepareToGet work SURL: srm://srm.a.com/srm/v2/server?SFN=/path/file O O ~ Client SRM TURL: gsiftp://gftp03.a.com/path/file TURL SURL : Storage URL TURL : Transfer URL SURL SRM GridFTP GridFTP GridFTP GridFTP GridFTP

  8. WT2 Architecture 10 Gbit Cisco Cisco SLAC Firewall Grid Interface/Proxy DQ2/LRC Cond DB dq2 client GridFTP SRM Internet Free Zone LSF Master Xrootd redirector 1Gbit NFS servers grid user home WN Cisco switch Xrootd server WN 1Gbit WN WN 10 Gbit SUN X4500 4 Gbit WN Cisco switch WN Xrootd server RHEL 4/64 ~ 600 cores, 270 TB usable WN

  9. ATLAS Distributed Data Management system US cloud New dataset Lyon T1 NE T2 MW T2 RAL T1 BNL storage File Transfer Service CERN storage DQ2/LRC dataset A Central catalog DB DQ2/Catalog WT2 disk Dataset replica info Dataset info subscript WT2 to dataset A Other DS File File DQ2 tools dq2_* (old, out-of-date) dq2–* (new, not yet deployed at OSG) File Site subscription info File Don Quijote II

  10. Panda result job DDM central catalog Panda BNL Panda Mover input task 1 task 2 Catalog output task 3 Storage Grid task 4 DQ2 … task n • Panda is a US ATLAS production system • Uses DDM for output, Panda Mover for input • Supports distributed analysis via the GRID

More Related