110 likes | 117 Views
This report provides an overview of the current hardware setup at the Liverpool HEP site, including desktops, laptops, batch farm, Matrix, HEP servers, and MAP2 cluster. It also discusses proposed network upgrades, security measures, and monitoring systems in place.
E N D
Liverpool HEP – Site Report May 2007 John Bland, Robert Fay
Staff Status • Two members of staff left in the past year: • Michael George August 2006, Andy Washbrook December 2006 • Replaced by two full time HEP system administrators • John Bland, Robert Fay December 2006 • One full time Grid administrator • Paul Trepka • One part time hardware technician • Dave Muskett
Current Hardware • Desktops • ~100 Desktops: Upgraded to Scientific Linux 4.3, Windows XP • Minimum spec of 2GHz x86, 1GB RAM + TFT Monitor • 48 new machines, rest upgraded to equivalent spec • Laptops • ~30 Laptops: Mixed architectures, specs and OSes. • Batch Farm • Scientific Linux 3.0.4, + software repository (0.7TB), storage (1.3TB) • 40 dual 800MHz P3s with 1GB RAM • Split 30 batch / 10 interactive • Using Torque/PBS • Used for general analysis jobs
Current hardware – continued • Matrix • 10 node dual 2.40GHz Xeon, 1GB RAM • 6TB RAID array • Used for CDF batch analysis and data storage • HEP Servers • User file store + bulk storage via NFS (Samba front end for Windows) • Web (Apache), email (Sendmail) and database (MySQL) • User authentication via NIS (+Samba for Windows) • Quad Xeon 2.40GHz shell server and ssh server • Core servers have a failover spare
Current Hardware – continued • MAP2 Cluster • 960 node (Dell PowerEdge 650) cluster • 280 nodes shared with other departments • Each node has 3GHz P4, 1GB RAM, 120GB local storage • 12 racks (480 nodes) dedicated to LCG jobs • 5 racks (200 nodes) used for local batch processing • Front end machines for ATLAS, T2K, Cockcroft • 13 dedicated GRID servers for CE, SE, UI etc • Each rack has two 24 port gigabit switches • All racks connected into VLANs via Force10 managed switch
Storage • RAID • All file stores are using at least RAID5. New servers starting to use RAID6. • All RAID arrays using 3ware 7xxx/9xxx controllers on Scientific Linux 4.3. • Arrays monitored with 3ware 3DM2 software. • File stores • New User and critical software store, RAID6+HS, 2.25TB • ~3.5TB general purpose ‘hepstores’ for bulk storage • 1.4TB + 0.7TB batchstore+batchsoft for the Batch farm cluster • 1.4TB hepdata for backups and scratch space • 2.8TB RAID5 for LCG storage element • New 10TB RAID5 for LCG SE (2.6 kernel) with 16x750GB SATAII.
Network • Topology MAP2 WAN Force10 Gigabit Switch Switch Switch Switch NAT LCG servers Offices Servers 1GB link
Proposed Network Upgrades • Topology - Future MAP2 3GB WAN 3GB Force10 Gigabit Switch firewall LCG servers Offices Servers 1GB link VLAN
Network Upgrades • Recently upgraded core Force10 E600 managed switch to increase throughput and capacity. • Now have 450 gigabit ports (240 at line rate) • Use as central departmental switch, using VLANs • Increasing bandwidth to WAN using link aggregation to 2-3GBit/s • Possible increase to departmental backbone to 2GBit/s • Adding departmental firewall/gateway • Network intrusion monitoring with snort • Most office PCs and laptops are on internal private network • Wireless • Wireless is currently provided by Computer Services Department • HEP wireless in planning
Security & Monitoring • Security • Logwatch (looking to develop filters to reduce ‘noise’) • University firewall + local firewall + network monitoring (snort) • Secure server room with swipe card access • Monitoring • Core network traffic usage monitored with ntop/MRTG (all traffic to be monitored after network upgrade) • Use sysstat on core servers for recording system statistics • Rolling out system monitoring on all servers and worker nodes, using SNMP, MRTG (simple graphing) and Nagios • Hardware temperature monitors on water cooled racks, to be supplemented by software monitoring on nodes via SNMP.
Printing • We have three group printers • Monochrome laser in shared area • Colour led • Colour ink/phaser • Accessible from Linux using CUPS and automatic queue browsing • Accessible from Windows using Samba/CUPS, and auto driver installs • Large format posters printed through university print queues.