140 likes | 263 Views
StoRM Some basics and a comparison with DPM. Wahid Bhimji University of Edinburgh GridPP Storage Workshop. This talk:. Considering moving to StoRM ??? Comparison with DPM Benefits / Problems Status . Whats StoRM ?. Lightweight SRM server Works on top of POSIX filesystems
E N D
StoRMSome basics and a comparison with DPM Wahid Bhimji University of Edinburgh GridPP Storage Workshop Wahid Bhimji – StoRM
This talk: Considering moving to StoRM ??? • Comparison with DPM • Benefits / Problems • Status Wahid Bhimji – StoRM
WhatsStoRM? • Lightweight SRM server • Works on top of POSIX filesystems • Relies on that filesystem to replicate data etc. • Used widely in Italy and a few place outside Wahid Bhimji – StoRM
DPM - reminder Worker Node rfio Pool Node DISK Pool Node DISK gridftp DPM Head Node Pool Node DISK … User … • “Pool” node (dpm-disk) • Directly mounted disk • RFIO– local transfers • GridFTP– WAN transfers • Head Node (dpm-mysql) • SRMV2 Interface • DPM server: Request tracking (MySQL) • DPM Nameserver (DPNS) (MySQL) Wahid Bhimji – StoRM
Storm Structure Worker Node Disk Servers file:// Disk Servers Disk Servers Disk Servers Disk Servers StoRM: Can be one machine. GridFTPserver(s) GridFTPserver(s) User Storm Frontend Storm Backend Backend: Namespace (by file path) Filesytem Drivers Frontend: SRM Interface Request Tracking (MySQL) Wahid Bhimji – StoRM
Differences to DPM • Namespace derived from filename • Local transfers through file:// POSIX access • Uses ACLs for permissions • Lot less (no) user tools • So no equivalent of dpm-* dpns-* tools • But filesystem operations are just du * rm * ls * • For spacetoken allocations etc. edit file: • /opt/storm/backend/etc/namespace.xml Wahid Bhimji – StoRM
Namespace srm://se2.glite.ecdf.ed.ac.uk:8444/srm/managerv2?=/atlas/atlasdatadisk/bill/bob DPM this would be DPNS path and to get to physical file you would need to use DB • On Storm there is no DB. The physical file is at: SPACE_ROOT/bill/bob SPACE_ROOT is defined for each VO/ spacetoken: Might be just /lustre/atlas/atlasdatadisk Wahid Bhimji – StoRM
Versions • Storm 1.5 released recently • 1.4 no longer “supported” • Currently for SL4 (32bit – prob. works on 64) • SL5 – “after easter” • Installations at QMUL and Edinburgh • both have issues at the moment • probably resolvable. • INFN-T1 are using 1.5 most other sites 1.4 Wahid Bhimji – StoRM
Instalation • Obtain repos http://grid-it.cnaf.infn.it/mrepo/repos/ glite-generic.repo ; lcg-ca.repo; ig.repo; jpackage.repo • “Standard” yum install / YAIM configure : yum install lcg-CA ig_SE_storm_backendig_SE_storm_frontend /opt/glite/yaim/bin/ig_yaim -c -ssiteinfo/site-info.def –nig_SE_storm_backend -nig_SE_storm_frontend • Setup filesystem (Lustre/GPFS etc.; acls needed) • You also need users etc (as for DPM + other nodes) Wahid Bhimji – StoRM
Weighing it up Wahid Bhimji – StoRM
Benefits Easiest current way to make use of Lustre / GPFS or other cluster filesystem: • file:// POSIX access – better local access performance that rfio (afternoon discussion) • Scalability / resilience (INFN-T1 uses Storm) • Simplicity: Navigable directories/ familiar posix commands (but you also have to maintain Lustre / GPFS if you use those) Wahid Bhimji – StoRM
Key reasons not to move • Limited UK experience, smaller user community than DPM /dCache – mostly Italian sites • Some implementation of SRM features lacking (in 1.4 – not clear if fixed in storm 1.5) • Access permissions • Information provider • Space reservation? ;-) : http://www.youtube.com/watch?v=A7uvttu8ct0 Wahid Bhimji – StoRM
StoRM space reservations? ;-) Wahid Bhimji – StoRM
Conclusions • StoRM is not (so) scary. • It has some quirks/limits but so does DPM • You can use the same hardware • SRM performance looks to be similar to DPM • Growing maturity and user community • It is currently the easiest way to use cluster filesystemsto give scalability and the performance of local file:// access • Discuss (after Chris’s talk…) Wahid Bhimji – StoRM