SCOTGRID: A PROTOTYPE TIER-2 CENTRE

1. SCOTGRID: A PROTOTYPE TIER-2 CENTRE Steve Thorn Authors: A. Earl, P. Clark, S. ThornUniversity of Edinburgh ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

2. Overview • What is ScotGrid? • ScotGrid resources • LCG2 experiences (at Edinburgh) • System administrator • Particle physics user • Future plans • Conclusions ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

3. What is ScotGrid? • One of four Tier-2 sites in the UK • Durham, Edinburgh & Glasgow • Member of GridPP collaboration • Over 100 users • Supports: ATLAS, BaBar, Bioinformatics, CDF, Computing Science, Engineering, LHCb, Medipix, MICE, PhenoGrid, UKQCD, ZEUS • First UK Tier-2 • Over 200 CPUs • ~40 TB disk • Initially funded by SHEFC (Scottish Higher Education Funding Council) • www.scotgrid.ac.uk ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

4. Hardware overview ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

5. Edinburgh University Front Ends: CE & SE:IBM x205: P-IV 1.8 GHz, 256 MB RAM LCFG:IBM x340: dual P-III 1 GHz, 2 GB RAM Storage (backend): 1 × IBM x440: 8 × Xeon 1.9 GHz, 32 GB RAM 2 × IBM FAStT900 providing 24 TB RAID 10 TB of 155 TB SAN Worker Nodes:4 × dual Xeon 2.8 GHz, 2 GB RAM, 200 GB disk Housed at the multi-million pound Advanced Computing Facility. 20 Gbit/s link to campus. ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

6. Benchmarking IBM x440: 8 × Intel Xeon 1.9 GHz, 32 GB RAM High Performance Linpack benchmark using ATLAS libraries. Matrix calculations to test processor and memory performance. Achieved ~75% of theoretical max performance with un-optimised system Full results at www.ph.ed.ac.uk/~aearl/hpl/ ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

7. Glasgow University Front Ends (CE,SE,LCFG,UI): 4 × Transtec 1001L: P-IV 2.8 GHz, 512 MB RAM Storage: 1 × IBM x370: P-III Xeon 700 MHz, 16 GB RAM 1 × IBM FAStT500 providing 5 TB RAID Worker Nodes: 59 × IBM x330: dual P-III, 1 GHz, 2 GB RAM 34 × IBM blades: dual Xeon 2.4 GHz, 1.5 GB RAM 5 × IBM x340: dual P-III 1 GHz, 2 GB RAM ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

8. Usage statistics ATLAS Glasgow CPU farm 1498570 CPU hours 4471 kilo SPECint2000 weeks (23 September 2004) LHCb BaBar CDF Grid Data Management Bioinformatics ZEUS UKQCD Medipix Information Retrieval Device Modelling MICE ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

9. Durham University New ScotGrid member since September 2004 Worker Nodes: 40 × dual P-IV 2.2 GHz, 2 GB RAM, 30 GB disk Based at the Institute for Particle Physics Phenomenology Using Sun Grid Engine ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

10. LCG2 experience: sys admin • Storage emphasis – too few Front End machines • Installed with LCFGng • Storage limited to 1 TB until SRM based deployment • Hardware moved to Advanced Computing Facility – successful • Worker Node 15 miles (24 km) from Front Ends • Participating in LCG2 Test Zone: ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

11. Installation problems • Multiple network interfaces • CE(2) and SE(3) • eth0 (external) is automatically configured by LCFGng • Others require manual config. of driver, network parameters (ip, netmask etc.) and routing • SE eth0 not used so eth1 & eth2 manually defined • SCSI and RAID controllers • SE only • Custom boot kernel • Full configuration of /etc/modules.conf required /* Second network interface eth1 */ EXTRA(hardware.modlist) eth1 +hardware.mod_eth1 alias eth1 tg3 EXTRA(network.interfaces) eth1 network.ipaddr_eth1 192.168.1.6 network.netmask_eth1 255.255.255.0 network.hostname_eth1 glenlivet network.network_eth1 192.168.1.0 network.broadcast_eth1 192.168.1.255 REPLACE(install.installmethods,network,network route) install.imethod_route route install/root ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

12. LCG2 Experience: user • Aim: to get a physicist with no grid experience to use LCG2 • Applied for Grid certificate • Well documented but time consuming • Joined LHCb VO • Install UI software on user’s desktop • Not possible due to OS mismatch and firewall • Used Glasgow’s UI to successfully run jobs ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

13. Future plans • Edinburgh • Deploy new Worker Nodes: 4 × dual processor from GridPP • Storage software development – SRM to SRB interface • Increase LCG storage to full capacity including SAN • Glasgow • Upgrade older Worker Nodes • Migrate to Scientific Linux for Front Ends • Durham • Deploy new Front End systems: 4 × dual processor from GridPP • Join LCG2 Test Zone • Seeking funding • to increase CPU at Edinburgh and storage at Glasgow • Research and support staff • Other institutes? ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

14. Conclusions • Successfully proven Tier-2 concept • Reached million CPU hour landmark • Over 100 users within 13 research groups/collaborations • Gained valuable LCG2 knowledge (installation and maintenance) • Edinburgh & Glasgow participating in LCG2 Test Zone • Next steps: prototype → production • Encourage more particle physics analysis • Continue with data challenges • More resources and funding ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University

15. Questions? ScotGrid: a Prototype Tier-2 Centre – Steve Thorn, Edinburgh University