1 / 42

ATLAS HLT/DAQ Stato e prospettive

CSN1 Settembre 2005. ATLAS HLT/DAQ Stato e prospettive. Valerio Vercesi. Outline. Pre-series Status in USA15/SDX1 Commissioning and exploitation Large Scale Test Activities, experiences Lessons learnt Activities Monitoring, ROD Crate DAQ Algorithms development and deployment Finance

deirdre-lane
Download Presentation

ATLAS HLT/DAQ Stato e prospettive

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSN1 Settembre 2005 ATLAS HLT/DAQ Stato e prospettive Valerio Vercesi

  2. Outline • Pre-series • Status in USA15/SDX1 • Commissioning and exploitation • Large Scale Test • Activities, experiences • Lessons learnt • Activities • Monitoring, ROD Crate DAQ • Algorithms development and deployment • Finance • Accounting • 2006 requests V. Vercesi - INFN Pavia

  3. S. Falciano (Roma1) Coordinatore Commissioning HLT • A. Negri (Pavia) Coordinatore Event Filter Dataflow • A. Nisati (Roma1) TDAQ Institute Board chair e Coordinatore Muon Slice PESA • F. Parodi (Genova) Coordinatore b-tagging PESA • V. Vercesi (Pavia) Deputy HLT leader e Coordinatore PESA (Physics and Event Selection Architecture) • Attività italiane • Trigger di Livello-1 muoni barrel (Napoli, Roma1, Roma2) • Trigger di Livello-2 muoni (Pisa, Roma1) • Trigger di Livello-2 pixel (Genova) • Event Filter Dataflow (LNF, Pavia) • Selection software steering (Genova) • Event Filter Muoni (Lecce, Napoli, Pavia, Roma1) • DAQ (LNF, Pavia, Roma1) • Monitoring (Cosenza, Napoli, Pavia, Pisa) • Pre-series commissioning (LNF, Pavia, Roma1) V. Vercesi - INFN Pavia

  4. ATLAS TDAQ system TDAQ 1 selected event every million ≅ Latency Rates Muon Calo Inner 40 MHz Pipeline Memories LEVEL-1 TRIGGER • Hardware-Based • Coarse granularity from calorimeter & muon systems LVL1 ~2 ms Readout Drivers ~75 kHz ROD ROD ROD RoI ROB ROB ROB High-Level Trigger LEVEL-2 TRIGGER • Regions-of-Interest “seeds” • Full granularity for all subdetector systems • Fast Rejection “steering” Readout Buffers ~1600 LVL2 farm LVL2 ~10 ms Event builder network ~2 kHz EVENT FILTER • Possibly “seeded” by Level 2 • Full event access • Algorithms inherited by offline EF farm ~1000 CPUs EF ~1 s ~200 Hz Storage: ~ 300 MB/s ( ) V. Vercesi - INFN Pavia

  5. TDAQ • Trigger e Data Acquisition hanno da sempre in fase di commissioning un doppio ruolo • Come “server” per il commissioning dei rivelatori • Come “client” per utilizzare le informazioni realistiche dell’esperimento per i propri studi di funzionalità e performance • La situazione si è già presentata durante il Combined Testbeam 2004 • Il TDAQ di ATLAS è un progetto in piena evoluzione in cui development/commissioning/exploitation sono ancora fasi molto miscelate • Presentazione di risultati e indicazione delle prospettive • Maggiore enfasi alle componenti con forte partecipazione italiana • Descrizione del piano di commissioning generale V. Vercesi - INFN Pavia

  6. Pre-series design “Module-0” of final system 8 racks (~10% of final dataflow) V. Vercesi - INFN Pavia

  7. Pre-series reality ROS rack LVL2 rack 6 racks SDX1 EF rack Switch rack Online rack V. Vercesi - INFN Pavia

  8. Commissioning and exploitation • Fully functional, small scale, version of the complete HLT/DAQ • Equivalent to a detector’s ‘module 0’ • Purpose and scope of the pre-series system • Pre-commissioning phase • To validate the complete, integrated, HLT/DAQ functionality • To validate the infrastructure, needed by HLT/DAQ, at point-1 • Commissioning phase • To validate a component (e.g. a ROS) or a deliverable (e.g. a Level-2 rack) prior to its installation and commissioning • TDAQ post-commissioning development system • Validate new components (e.g. their functionality when integrated into a fully functional system) • Validate new software elements or software releases before moving them to the experiment V. Vercesi - INFN Pavia

  9. Pre-Series Commissioning V. Vercesi - INFN Pavia

  10. Commissioning LVL2+ROS Preliminary First measurements with full LVL2 rack feeded by ROS data Using separate Control and Data networks V. Vercesi - INFN Pavia

  11. Commissioning EF V. Vercesi - INFN Pavia

  12. Pre-series • Status in USA15/SDX1 • Commissioning and exploitation • Large Scale Test • Activities, experiences • Lessons learnt • Activities • Monitoring, ROD Crate DAQ • Algorithms development and deployment • Finance • Accounting • 2006 requests V. Vercesi - INFN Pavia

  13. Large Scale Tests • Pre-serie work will help understanding the TDAQ system in terms of functionality • Forms the basis for future deployments/exploitations • Complexity of ATLAS TDAQ system arises also from the size of bulk components involved • Topology of communications, size of LVL2/EF farms, software, … • Test scalability of HLT system using presently available large installations • Understand issues like configuration, startup time, communication, control, error reporting, … • UCB/TRIUMF WestGrid Cluster (http://www.westgrid.ca) • 60 racks x 14 nodes = 840 Dual-CPU nodes(3 GHz CPUs / 2-4 GB RAM) • CERN LXSHARE Cluster (http://batch.web.cern.ch/batch) • Up to ~700 nodes (various flavours) • Reference page for all tests • http://atlas-tdaq-large-scale-tests.web.cern.ch V. Vercesi - INFN Pavia

  14. State transitions USR_RUNNING_TIME (default is 30 s) RUNNING Luke Warm Start Luke Warm Stop CONFIGURED Configure Unconfigure INITIAL Cold Stop Shutdown Boot Cold Start ABSENT Setup Close configure: load configure start: prepareForRun startTrigger stop: stopTrigger stopFrontEnd stopDataCollection stopEventFilter stopRecording unconfigure: unconfigure unload V. Vercesi - INFN Pavia

  15. LVL2 transition times • State transistion timing quite acceptable • No significant differences between 2 and 3 tier Run Control V. Vercesi - INFN Pavia

  16. EF results @ LST • Effect of realistic algorithm:TrigMoore vs HelloWorld • EF standalone 1EFD+2PTs / node • up to 200 nodes • MySQL as geometry DB used • significant slow down due to access/reading geom. DB • MySQL vs Oracle DB in TrigMoore • EF standalone, 1EFD+2PTs / node • Oracle DB – up to 160 nodes • MySQL DB – up to 200 hosts • MySQL works faster at “small scales”, while Oracle looks better at higher scales - to be investigated more • not able to on higher than 200 nodes with any of both partitions – to be investigate further (do we need to replicated DBs ?) V. Vercesi - INFN Pavia

  17. Pre-series • Status in USA15/SDX1 • Commissioning and exploitation • Large Scale Test • Activities, experiences • Lessons learnt • Activities • Monitoring, ROD Crate DAQ • Algorithms development and deployment • Finance • Accounting • 2006 requests V. Vercesi - INFN Pavia

  18. Cosmics Tile setup MobiDAQ (Mobile DAQ): read out of 8 drawers in the pit with temporary RODemu but real TDAQ (tdaq-01-02-00), tests of electronics, cosmic muons runs V. Vercesi - INFN Pavia

  19. GNAM Monitoring chain • Framework per monitoring on-line a basso livello • Core: trasporto di eventi, istogrammi e comandi • Plugin dinamici: decodifica e istogrammazione • Possibilita’ di correlazione fra diversi rivelatori • Comandi asincroni (reset, rebin, update) • Status • Incluso in TDAQ da aprile • Utilizza i servizi disponibili per il monitoring • Validato al CTB04; in uso in alcuni siti di commissioning

  20. GNAM al commissioning • Acquisizione dati nell’ambito del software TDAQ • Commissioning di MDT usa GNAM per monitoring online ed analisi dati • Sampling completo a livello di ROS [~2 KHz per noise test, ~200 Hz per pulser] • Integrazione di librerie: completato per MDT, in via di sviluppo per RPC • Stato del monitoring MDT: • Richiesta minimale di informazioni allo shifter (nomi camere) • Output: file di istogrammi e file di testo con risultati di analisi dati per ciascuna camera e per ciascun run • In sviluppo: • ottimizzazione dell’analisi dati on-line • installazione del presenter per la visualizzazione degli istogrammi • on-line event display V. Vercesi - INFN Pavia

  21. PMP Presenter • Visualizzazione asincrona on-line di istogrammi • Interattivo (reset, rebin, zoom, fitting, ecc...) • Operazioni grafiche su istogrammi (ROOT canvas) • Grafica configurabile • Status • Incluso in TDAQ da settembre • Utilizza i servizi disponibili per il monitoring • Pienamente funzionale al CTB04 • Riprogettato per nuove funzionalità e maggiore scalabilità V. Vercesi - INFN Pavia

  22. GNAM&PMP: sviluppi futuri • GNAM • Completare l’integrazione nel framework del TDAQ • Configurazione di Core e librerie dal database del TDAQ • Messaggistica e gestione degli errori software • Supporto per la generazione di allarmi automatici • Livelli di severità, routing • Verifica delle prestazioni e delle risorse necessarie • CPU, Memoria, Banda • Studio della scalabilità • PMP • Completare la nuova versione • Minimizzazione del traffico di rete • Adattamento della IGUI alla nuova struttura • Generazione allarmi • Plugin di analisi degli istogrammi V. Vercesi - INFN Pavia

  23. ROD Crate DAQ R C C M E M O R Y C O R B O Config & Control Data readout • RCD is used as interface with the RODs for • Control • Configuration • Monitoring • Statistics • Event sampling • Data readout (through VME) • User guide for detectors developers available • Validation system in Bld. 40 • DAQ Commissioning – Phase 1: • The ROD Emulator system will be used in order to validate all common RCD software and infrastructure • After adding and validating the detector sw and hw specific items, multi crate event building will be used in the absence of the full DAQ chain R O S ROD Emulator VMEbus memory + CORBO = Memory + Registers + Interrupt capability R E B Event Fragments V. Vercesi - INFN Pavia

  24. RCD exploitation • Experience from Combined Testbeam extremely useful • Recall almost all detector used it in the CTB • Successful workshop to put forward new requirements • As a consequence, several improvements during last months • Configurable interrupt handling • Simplified user interface to access ordered event fragments • Data driven event building for multicrate acquisition in the commissioning phase • Simplified ROD emulation • Hardware trigger distribution • All sub-detector commissioning (but LAr…) sites use RCD • MDT and RPC on the forefront • BB5 integration, Point 1 with MROD, Lab testing with ROD emulators V. Vercesi - INFN Pavia

  25. Algorithms: Muon slice • LVL1 simulation is the fundamental input for the measurement of the full muon vertical slice performance • LVL2 and EF Muon algorithms have been extensively tested on data simulated in ATLAS • Rome Physics Workshop: June 2005 • LVL2: Fast • Confirm the LVL1 trigger with a more precise PT estimation within a Region of Interest (RoI) • Global pattern recognition, track fit, fast PT estimate via Look Up Table (LUT) with no use of time consuming fit methods • Event Filter: TrigMoore • Based on offline reconstruction algorithm Moore • Can run seeded (reconstruction starting from RoI of previous levels) • Precise PT determination • General goal is now to achieve more realistic estimate of trigger selections and corresponding rates • Real geometry, configuration and conditions database usage, … V. Vercesi - INFN Pavia

  26. Athena release 10.0.4 Low pT 6 GeV Threshold LVL1 Coincidence Windows Efficiency curve Low-pT Inefficiency map V. Vercesi - INFN Pavia

  27. Early stages of project Endcap differs from Barrel M and O station are outside B field Inhomogeneous B field – bending is local Algorithm Pattern recognition and fit in TGC → position and slope in EM Extrapolate segment into MDT EM/EO → Roads in EM/EO, find hits, fit Next pattern recognition and fit in MDT as in mFast – not done yet Extrapolation into EI and LUT MuFast endcap extension V. Vercesi - INFN Pavia

  28. MuFast: MDT miscalibration • Commissioning the algorithms: realistic approach to data handling • The plot shows the muFast resolution for two different scenarios: • the correct MDT r-t function is used, red points • a systematic shift of + 0.2 mm is added to the radius returned by the correct r-t function, blue points +10% degradation @ 50 GeV, to be compared with a + 5% expected by a naive calculation V. Vercesi - INFN Pavia

  29. TrigMoore: MDT miscalibration • Single muons (with pT = 6, 10, 20, 40, 100 GeV/c, produced for the Rome Initial Layout) have been reconstructed in two different scenarios • Using the correct MDT r-t relation function (red squares in pictures) • Applying a systematic +0.2 mm shift on the radius obtained with the correct MDT r-t function (blue circles in pictures) The relative degradation in (pT) is +5% for muons with a 6 GeV/c transverse momentum, increases to +13% around 50 GeV/c This MDT miscalibration leaves almost unaffected  resolutions. pT resolution (Moore) V. Vercesi - INFN Pavia

  30. LVL2 tracking: SiTrack • Preliminary results obtained on DC1 b-jet samples at initial luminosity V. Vercesi - INFN Pavia

  31. B-tagging @ LVL2 • Results obtained with the “standard” SiTrack algorithm on DC1 data • b-tagging: likelihood ratio using transverse and longitudinal impact parameters • Upgraded version to be tested soon: should improve both efficiency and track parameters resolution and hence significantly improve the b-tagging performance V. Vercesi - INFN Pavia

  32. Algorithms steering g60Hy iSg60 gIsol60 l2g60i Cluster60 eg60Hy eg60 EMtrackHard TrackHardEM T2Calo EM60Hy EM60 e60Hy iSe60 eIsol60 l2e60i g25Hy iSg25 gIsol25 l2g25i EM25Hy EM25 Cluster25 eg25Hy eg25 TrackSoft25EM T2Calo EMtrackSoft e25Hy iSe25 eIsol25 l2e25i EMROI g20Hy iSg20 gIsol20 l2g20i T2Calo Cluster20 eg20Hy eg20 EMtrackSoft TrackSoft02EM EM20Hy EM20 e20Hy iSe20 eIsol20 l2e20i g15Hy iSg15 eIsol15 l2e15i Cluster15 EM15 eg15 EM15Hy eg15Hy TrackSoft15EM T2Calo EMtrackSoft e15Hy iSe15 eIsol15 l2e15i V. Vercesi - INFN Pavia

  33. ATLAS Commissioning Phases • Commissioning means bringing ATLAS systems from “just installed” to “operational”. It is broken in 4 phases • Subsystem standalone commissioning • Integrate subsystems into full detector • Cosmic rays, recording data, analyze/understand, distribute to remote sites • Single beam, first collisions, increasing rates, etc… • A consistent part of commissioning activities will be done during the installation itself • Phases will overlap since different systems may be at different development levels • For the barrel calorimeter commissioning will start soon • Tile calorimeter is already taking data V. Vercesi - INFN Pavia

  34. HLT Commissioning • Commissioning is a set of activities which spans the time interval from the installation of the HLT racks and nodes … • A rack is the elementary unit for commissioning • OS, Dataflow and Online software are installed • ... to the phase when the HLT is filtering physics data and recording them • HLT selection algorithms are installed and running stably • The complete trigger menu (at least for early physics) is configured • The trigger selection efficiencies and background rejection rates are understood and can serve as input for physics measurements • Phase-1 Commissioning definition is the most urgent • Heavily use the Pre-series to exercise the procedures for installation and commissioning • Important steps will cover the integration of detectors into full system • Involve operations that have a very strong coupling with the offline commissioning activities • Development of specific algorithms looking at simple data decoding (cabling,…) • Final commissioning phases extend far beyond the data-taking startup (interface with run coordinator team) V. Vercesi - INFN Pavia

  35. Cosmic muons in ATLAS Concrete Surface building PX14/16 shielding (2.5 g/cm3) Air PX16 (12.6 m Inner Ø ) PX14 (18.0 m Inner Ø) ATLAS Geant Simulation Initial detector Rock ~ Silicon 600m x 600m x 200m deep (2.33 g/cm3) V. Vercesi - INFN Pavia

  36. Overall plan V. Vercesi - INFN Pavia

  37. Outlook • A lot of work during this year, system entering phase of complete deployment • Purchase plan proceeding as scheduled, with some minor delays • (Wo)manpower situation not always healthy • More help and support welcome • Three big tasks awaiting us in the next months • Commissioning the pre-series and extract a coherent and complete set of system performance measurements • Based on previous experiences and on already established partial results • On-line trigger selections evaluation (rates, efficiencies, physics coverage,…) evolving towards more realistic approach • Calibration, geometry “as installed”, mis-alignment, error handling, complete trigger menus, physics analysis based on trigger objects • Prepare for cosmic run next year • “Cosmic” slices and trigger menu (Tile, LVL1, “digital” LVL2) V. Vercesi - INFN Pavia

  38. Pre-series • Status in USA15/SDX1 • Commissioning and exploitation • Large Scale Test • Activities, experiences • Lessons learnt • Activities • Monitoring, ROD Crate DAQ • Algorithms development and deployment • Finance • Accounting • 2006 requests V. Vercesi - INFN Pavia

  39. Accounting • Contributo INFN alla Pre-serie • Read-Out System: 51 kCHF (ROS Racks) • Online Computing System: 40 kCHF (Monitoring, Operations) • Online Network System: 44 kCHF (Switches, FileServer) • Inviati al CERN a Dicembre 2004 • VV riceve in copia tutte le fatturazioni dei singoli acquisti ed un sommario mensile dello stato finanziario • Contributo CORE 2005 • Online Computing System: 45 kCHF (Monitoring, Operations) • Inviati al CERN a Maggio 2005 • Già acquisiti due file server • Read-Out System: 275 kCHF (ROS Racks) • Questo acquisto si espleta secondo una gara e non con un semplice market survey o price inquiry come fino ad ora avvenuto • Richiesta alla Giunta l’autorizzazione per partecipare alla gara • Grazie a Speranza che si è prodigata per espletare le pratiche necessarie • Il CERN preferisce gestire la gara su un periodo di due anni • Omogeneità dei componenti vs miglioramento delle prestazioni • Si sommano i 275 kCHF del 2006 V. Vercesi - INFN Pavia

  40. Cost Profile (kCHF) V. Vercesi - INFN Pavia

  41. Missioni Estere • LNF • Commissioning HLT/DAQ Pre-serie e pit: 6 m.u. • Ferrer, Kordas (+ Miscetti, Giovannella) • Pavia • VV coordinatore PESA e duputy HLT: 1 m.u. • Negri A. responsabile Event Filter: 1 m.u. • Scannicchio D. commissioning HLT: 2 m.u. • Roma1 • Speranza responsabile commissioning HLT: 2 m.u. • Leandro chair IB, coordinatore slice mu: 1 m.u. • ROD crate DAQ e HLT/DAQ muoni : 4 m.u. • Pasqualucci, Di Mattia, … V. Vercesi - INFN Pavia

  42. Milestones • 30/06/2005 • TDAQ - Installazione, test e uso della "Pre-serie" (~ 10% TDAQ slice) • “ragionevolmente” raggiunta: ritardi accumulati soprattutto sugli acquisti delle componenti • 24/12/2005 • TDAQ - Installazione e test dei ROS di Pixel, LAr, Tile, Muon (interfacciamento al ROD Crate e integrazione nel DAQ) • Parte del piano di commissioning in esecuzione: piccola dipendenza dalla data di consegna dei ROS • 30/04/2006 • Completamento dei test sulla pre-serie e definizione delle funzionalità per il supporto al commissioning TDAQ • 31/08/2006 • Commissioning delle slice di ROS dei rivelatori utilizzando le funzionalità della pre-serie (modulo-0 del sistema finale) • 31/12/2006 • Presa dati integrata dei rivelatori nel pozzo con raggi cosmici V. Vercesi - INFN Pavia

More Related