1 / 16

PC Farms & Central Data Recording

PC Farms & Central Data Recording. CERN - European Laboratory for Particle Physics. ATLAS Trigger/DAQ Workshop Chamonix, October 20, 1998 Frédéric HEMMER - CERN/IT. Http://wwwinfo.cern.ch/pdp/pc/EF/ATLAS-TDAQ-1098. Overview. NA48 Data Recording NA45 Data Recording in Objectivity

dexter-bonner
Download Presentation

PC Farms & Central Data Recording

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PC Farms & Central Data Recording CERN - European Laboratory for Particle Physics ATLAS Trigger/DAQ Workshop Chamonix, October 20, 1998 Frédéric HEMMER - CERN/IT Http://wwwinfo.cern.ch/pdp/pc/EF/ATLAS-TDAQ-1098

  2. Overview • NA48 Data Recording • NA45 Data Recording in Objectivity • NA57 Data Recording in HPSS • Summary Frédéric Hemmer CERN/IT

  3. 3Com 3900 Cisco 5505 NA48 Central Data Recording Sub detector VME crates Event Builder Online PC Farm FDDI Fast Ethernet SUN E450 500 GB Disk space XLNT Gbit Fast Ethernet 7 KM Gigabit Ethernet 3Com 9300 GigaRouter HiPPI HiPPI FDDI Offline PC Farm CS/2 2.5 TB Disk space Frédéric Hemmer CERN/IT

  4. NA 48 Data Recording in 98 • May  September 1998 • Raw Data on Tape • 68 TB (1450 tapes, mainly 50 GB tapes) • 12.5 TB Selected Reconstructed Data • Total with 97 data : 96 TB • Average Data Rate : 18 MB/s (peaks @ 23 MB/s) • CDR system can do 40-50 MB/s; limitation is CPU Time available • Data recorded as files (4 million) Frédéric Hemmer CERN/IT

  5. NA48 On Line Farm • 11 Subdetector PC’s (dual PII-266, 128 MB) • 8 Event Building PC’s (dual PII-266, 128 MB, 18 GB SCSI) • 4 CDR routing PC’s (dual PII-266, 64 MB, FDDI) • All running Linux • Software event building in the interburst gap • Optional Software Filter (tags data) • Send data to computer center (local disk buffers : 144 GB , 2 hours) • On CS/2 : L3 Filtering and tape writing Frédéric Hemmer CERN/IT

  6. Cisco 5505 Fast Ethernet 3Com 3900 NA48 Plans for 1999 Sub detector VME crates Event Builder 4 * SUN E450 4.5 TB Disk space 7 KM Gigabit Ethernet Fast Ethernet 3Com 9300 HiPPI HiPPI Gigabit Ethernet On/Offline PC Farm Frédéric Hemmer CERN/IT

  7. NA45 Data Recording Sub detector VME crates NA48 SCI Event Builder On Line PC Farm Fast Ethernet 3Com 3900 PCSF 7 KM Gigabit Ethernet 2 * SUN E450 500 GB Disk space Fast Ethernet 3Com 9300 HiPPI HiPPI 3Com 3900 Gigabit Ethernet Frédéric Hemmer CERN/IT

  8. NA45 Raw Data recording in Objectivity • October 98 ; November 98 • Estimated bandwidth : 15 MB/s • Processes translate Raw Data format to Objectivity • Database files (1.5 GB) are closed, then written on tape • Steering done using a set of perl scripts on the disk servers • On line filtering/reconstruction/calibration possible • Farm is running Windows NT • Reconstruction can use PCSF Frédéric Hemmer CERN/IT

  9. PCSF Configuration (1) • Server running NT 4.0 Server SP3 • 1 dual capable Ppro @ 200 MHz, 96 MB, with 9 GB data disk (with mirroring). LSF central queues. • Server running NT Terminal Server Beta 2 • 1 dual Ppro @ 200 MHz, 128 MB, with 4 GB data disk. Runs IIS 3.0 and is accessible from outside CERN. It also host the asp’s for Web access • Servers running NT 4.0 Workstation SP3 • 9 dual Ppro’s @ 200 MHz, 64 MB, 2*4GB • 25 dual PII’s @ 300 MHz, 128 MB, 2*4GB All equipped with boot proms Frédéric Hemmer CERN/IT

  10. PCSF Configuration (2) • Machines interconnected with 4 3com 3000 100BaseT switch • Display/Keyboard/Mouse connected to a Raritan multiplexor • PC Duo for remote admin access  There were problems with other products • All running LSF 3.0.  LSF 3.2 does not work, support weak • Completely integrated with NICE Frédéric Hemmer CERN/IT

  11. Racking evolution 1998 1997 Frédéric Hemmer CERN/IT

  12. HPSS test system Frédéric Hemmer CERN/IT

  13. HPSS Tests results (NA57) • Disk to disk transfers without tape migration: • sustained: 5 - 6MB/s (1GB files) 4-5MB/s (200MB files) • average: ~5.5MB/s • peak: 6.5MB/s • With disk to tape migration: • sustained: 2 - 3MB/s (200MB files) • average: 3 - 5MB/s • peak: 5.5 - 6.5MB/s • Disk @ DAQ limited to 7 MB/s read ! Frédéric Hemmer CERN/IT

  14. STK Powderhorn silos Redwood IBM 3494 robot 2 * DEC Alpha 4100 4 CPU, 512Mb 3590 Gbit Ethernet FDDI HiPPI 100BaseT 3900 switch Gbit Ethernet NA57 100BaseT MPPC2604 Computer center ~120 GB mirrored Barracuda-9, 18GB RS6000/F50, 2CPU, 512MB Production system ~120 GB mirrored RS6000/F50, 2CPU, 256MB Frédéric Hemmer CERN/IT

  15. Current & Future Data rates Frédéric Hemmer CERN/IT

  16. Summary • On line PC farms are being used to record data at sensible rates (Linux) • Off line PC farms are being used for reconstruction/filtering/analysis (Linux/NT) • New paradigms of recording data are explored (Objectivity/HPSS) • Still a lot to do on scalable farm management, global steering, CDR monitoring, etc.. Frédéric Hemmer CERN/IT

More Related