1 / 10

Features of the new CASTOR

Features of the new CASTOR. Alice offline week, 16/9/2004 Olof Bärring, CERN. Outline. The new stager architecture hybrid solution for the ALICE online MDC status and deployment Features for the ALICE online MDC Features for ALICE physics MDC. Application. RFIO/stage API.

tao
Download Presentation

Features of the new CASTOR

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Features of the new CASTOR Alice offline week, 16/9/2004 Olof Bärring, CERN

  2. Outline • The new stager • architecture • hybrid solution for the ALICE online MDC • status and deployment • Features for the ALICE online MDC • Features for ALICE physics MDC

  3. Application RFIO/stage API 3rd party Policy Engine Garbage Collector Request Handler Recaller Request repository and file catalogue (Oracle or MySQL) Migrator Tape mover (RTCOPY) client daemon Resource Management Interface rfiod (disk mover) rfiod (disk mover) rfiod (disk mover) CASTOR tape archive components (VDQM, VMGR, RTCOPY) LSF Maui Disk cache New CASTOR Stager Architecture Authentication mvr cntl file system load monitoring

  4. Application RFIO/stage API 3rd party Policy Engine Garbage Collector Request Handler Recaller Request repository and file catalogue (Oracle or MySQL) Migrator Resource Management Interface Tape mover (RTCOPY) client daemon rfiod (disk mover) rfiod (disk mover) rfiod (disk mover) CASTOR tape archive components (VDQM, VMGR, RTCOPY) LSF Maui Disk cache Not ready for ALICE MDCs Authentication mvr cntl file system load monitoring

  5. Hybrid solution (online MDC) • Three components not yet implemented • Garbage collector • Migrator • Recaller • Hybrid solution • A slimmed version of today’s stgdaemon is used as the framework • Interfaced with new request repository (but not the file catalogue) • Old recaller/migrator have been interfaced with expert system for file system selection policies and request repository for submitting the tape requests to the RTCOPY client daemon

  6. Application RFIO/stage API 3rd party Policy Engine Today’s GC script stager_castor stgdaemon Request Handler Recaller Request repository (Oracle or MySQL) Migrator Tape mover (RTCOPY) client daemon Resource Management Interface rfiod (disk mover) rfiod (disk mover) rfiod (disk mover) CASTOR tape archive components (VDQM, VMGR, RTCOPY) LSF Maui Disk cache ALICE MDC stager hybrid mvr cntl file system load monitoring

  7. Status and deployment • The hybrid stager is ready for performance tests • minor problem with retry of failing tape requests remains to be solved (fixed last night?) • For this years ALICE MDC there are two parallel setups forseen • development setup for testing out new features and fixing bugs on the fly • production setup where the MDC runs • The parallel setups use independent oracle servers for their request catalogues

  8. lxs5010 lxs5011 lxs5012 • rhserver • rtcpclientd • dlfserver (logging) • stgdaemon • rmmaster • expertd • moab (Maui) lxshare031d lxshare032d lxshare034d lxshare030d lxshare027d lxshare179d lxshare028d lxshare033d • Disk server • rmnode • rfiod • rootd • Disk server • rmnode • rfiod • rootd • Disk server • rmnode • rfiod • rootd • Disk server • rmnode • rfiod • rootd • Disk server • rmnode • rfiod • rootd • Disk server • rmnode • rfiod • rootd • Disk server • rmnode • rfiod • rootd • ORACLE Test setup

  9. Features for ALICE MDC • Dynamic migration streams • New migration candidates can be added to an already running stream • In principle a stream could run forever... • Migration candidate is decided just-in-time • The next file to migrate is selected when the tape mover is ready to receive the data • Throttling • Using LSF or Maui allows for throttling when the system is completely loaded • rootd (and later xrootd) is the disk mover serving the client

  10. Features for offline MDCs • Throttling • Using LSF or Maui allows for throttling when the system is completely loaded • File catalogue in Oracle will remove the old stgdaemon limit on ~200-500k entries • rootd (and later xrootd) is the disk mover serving the client • Disk-to-disk replication of “hot” files • Full-fledged policy system for GC, recalls and migrations • Remaining issue: what to do with tape performance for small files?

More Related