1 / 13

The 2001 Tier-1 prototype for LHCb - Ital y

The 2001 Tier-1 prototype for LHCb - Ital y. Vincenzo Vagnoni. Genève , November 2000. Outline. 2001 LHCb Italian Tier-1 prototype architecture Linux diskless nodes howto Network attached storage Tests made in Bologna Conclusions. 2001 LHCb Italian Tier-1 protoype.

dusan
Download Presentation

The 2001 Tier-1 prototype for LHCb - Ital y

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni Genève, November 2000

  2. Outline • 2001 LHCb Italian Tier-1 prototype architecture • Linux diskless nodes howto • Network attached storage • Tests made in Bologna • Conclusions The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  3. 2001 LHCb Italian Tier-1 protoype • 15 CPU 2001 (700 SI95) Tier-1 computer farm (at the beginning of the year, with further increase if we can show that more CPU power is needed for the italian groups) • 15 single-processor motherboards, 256 MB RAM each or more, rack-mounted, diskless, with redundant power supply and cooling. • 1 TB IDE disk array in RAID-5 configuration, hosted in a NAS (Network Attached Storage) unit (also to be increased if needed). • 100 Mbps Ethernet Switch. The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  4. 2001 Tier-1 Computer Farm Prototype The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  5. Components (Motherboards, Racks, NAS, Switches) The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  6. Advantage of diskless nodes • Easy Operating System installation and maintainance • Adding a machine to the farm requires to run a simple script on the disk server • The Operating Systems are centralized and accessible through the file system of the disk server • Enhanced disk fault tolerance • The disk server can have large hard disk arrays with redundant information by using a RAID 5 • No need of UPS battery for the client nodes • Only the disk server needs UPS, no damage if a client is powered off • System compactness • The nodes are basically simple motherboards with integrated ethernet cards • They can be arranged in racks, ~25 nodes in only 1 m2. The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  7. Working concept (using for example a PXE BootROM) • At boot time the diskless client sends its ethernet card MAC address by making a bootp (or dhcp) broadcast request over the LAN • The diskless client ethernet controller must be equipped with a BootROM (for example a PXE, Pre-eXecution Environment, compliant one) • The bootp (or dhcp) server replies assigning the IP address to the client and passing the file name of the boot code on the server • Using a PXE BootROM the boot code is a small loader (pxelinux.bin), provided with the syslinux package, also able to read a configuration file on the server with some additional information (i.e. the arguments to pass to the linux kernel) • The client makes a TFTP connection to the server asking to download the PXE loader and executes it • The loader tftp-downloads the linux kernel, and executes it The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  8. Further details… • The linux kernel needs to be compiled with root-over-NFS support to be able to mount the root filesystem through NFS • The Ethernet Card driver must be compiled resident into the kernel (in principle could be also provided by an initial ramdisk) • The clients can share some directories of the linux filesystem (/usr, /opt, …), but need some others to be private (at least /etc, /dev, /var, /tmp) • Optionally some non-shareable directories (for example /dev, /var, /tmp) can be mounted on ramdisks The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  9. Network Attached StorageOpenNAS RS15 RaidZone • File Server supports NFS (also Windows SMB, but fortunately we don’t need this guy) • Built-in Web Server and DNS Server • R3 RAID5 (Redundancy with RapidRecovery) • Java-based remote interface for all OpenNAS functions, including diskarray configuration • 15 hot-swappable drives bays, with 15 Ultra ATA/100drives (80GB each) + 1 hot-swappable spare • Dual PIII 800MHz CPUs and 256 MBECC RAM • Dual 100BaseT network connections(Gigabit optional) • Dual redundant 300W hot-swappable power supplies • Possibility of two RAIDZONE disk array expansions (up to 3 TB) • Cost: $20850 (RS15) + optional $19595 (1 TB expansion) The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  10. Tests made in Bologna • Hardware setup • 4 PC HP Vectra, single processor PIII 500 MHz, 256 MB RAM, one acting as root filesystem disk server and three as diskless clients • 1 PC HP Kayak, dual processor PIII 733 MHz, 256 MB RAM, as an additional client with different architecture • Software setup • Linux RedHat 6.2, upgraded with kernel 2.2.16, or optionally (to be choosen at boot time) Linux SuSE 6.4 • Tests made to evaluate: • Complexity of installation and administration • Performance of the system (especially interesting due to the absence of the system swap area, even if in principle a network based one could be arranged) The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  11. Installation tests results • Installation trivial • Once the server system is configured, adding a new client takes 1 minute • All the installations are by default identical • No additional network activity observed (monitored directly into the switch) • The OSes rarely need to access non cached information on the server (server disk almost completely inactive) • Robust system • Simulated network failures for few seconds do not interfere with the OS work • Powering off the client machines (without a shutdown!) doesn’t create any problem at all (no need to check the disk data integrity at reboot) The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  12. Performance test results • Performance tested by running two HERA-B MonteCarlo simulation (ARTE) jobs on a dual processor 733 MHz, 256 MB RAM, HP Kayak with diskless installation (running Linux SuSE 6.4 in this case) • Performance compared with an identical machine with non-diskless installation and local disk swap area • Identical performances! The MC jobs were tuned to allocate ~100 MB RAM each • Even with 256 MB RAM, two heavy jobs can run together without using a local disk swap area at all • Conclusion: no problem for a single-processor motherboard equipped with 256 MB RAM running one single job The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

  13. Conclusions • The 2001 LHCb Tier-1 prototype architecture is presented • Based on diskless nodes configuration with a Network Attached Storage disk server and a 100 Mbps switched ethernet • System proof of principle tested • This kind of system can be an interesting contribution to the LHC computing community • It saves time, room and money! The 2001 Tier-1 prototype for LHCb-Italy Vincenzo Vagnoni

More Related