1 / 37

ATLAS HLT/DAQ

CSN1 Trieste Settembre 2006. ATLAS HLT/DAQ. V. Vercesi for the ATLAS Italia HLT/DAQ Group. S. Falciano (Roma1) Coordinatore Commissioning HLT A. Negri (Irvine, Pavia) Coordinatore Event Filter Dataflow A. Nisati (Roma1) TDAQ Institute Board chair e Coordinatore PESA Muon Slice

rayya
Download Presentation

ATLAS HLT/DAQ

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSN1 Trieste Settembre 2006 ATLAS HLT/DAQ V. Vercesi for the ATLAS Italia HLT/DAQ Group

  2. S. Falciano (Roma1) Coordinatore Commissioning HLT • A. Negri (Irvine, Pavia) Coordinatore Event Filter Dataflow • A. Nisati (Roma1) TDAQ Institute Board chair e Coordinatore PESA Muon Slice • F. Parodi (Genova) Coordinatore b-tagging PESA • V. Vercesi (Pavia) Deputy HLT leader e Coordinatore PESA (Physics and Event Selection Architecture) • Attività italiane • Trigger di Livello-1 muoni barrel (Napoli, Roma1, Roma2) • Trigger di Livello-2 muoni (Pisa, Roma1) • Trigger di Livello-2 pixel (Genova) • Event Filter Dataflow (LNF, Pavia) • Selection software steering (Genova) • Event Filter Muoni (Lecce, Napoli, Pavia, Roma1) • DAQ (LNF, Pavia, Roma1) • DCS (Napoli, Roma1, Roma2) • Monitoring (Cosenza, Napoli, Pavia, Pisa) • HLT/DAQ system commissioning and exploitation (Everybody) V. Vercesi - INFN Pavia

  3. ATLAS TDAQ Dual(quad)-CPU nodes ~30 ~1600 ~100 ~ 500 Local Storage SubFarm Outputs (SFOs) Event Filter (EF) Event Builder SubFarm Inputs (SFIs) LVL2 farm Event rate ~ 200 Hz Second- level trigger Data storage SDX1 pROS DataFlow Manager Network switches stores LVL2 output Network switches LVL2 Super- visor Gigabit Ethernet Event data requests Delete commands Requested event data USA15 Event data pulled: partial events @ ≤ 100 kHz, full events @ ~ 3 kHz UX15 Regions Of Interest USA15 Data of events accepted by first-level trigger 1600 Read- Out Links ~150 PCs VME Dedicated links Read- Out Drivers (RODs) ATLAS detector Read-Out Subsystems (ROSs) RoI Builder First- level trigger UX15 Timing Trigger Control (TTC) Event data pushed @ ≤ 100 kHz, 1600 fragments of ~ 1 kByte each V. Vercesi - INFN Pavia

  4. Level-1 Muon Trigger System No RPC hit No Trigger Low-Pt Trigger DC3 data DC2 data Low-pT 6 GeV Threshold Efficienza dell’algoritmo di selezione è > 99% Tutte le inefficienze sono dovute a zone dello spettrometro non coperte da RPC. In particolare: Settori speciali (Feet), ascensore, crack =0 e supporti del magnete. Efficienza globale Level-1 nel Barrel: 83% Low-pT Trigger 79% High-pT Trigger V. Vercesi - INFN Pavia

  5. Pre-series system in ATLAS point-1 5.5 8 racks (10% of final dataflow, 2% of EF) Partial Superv’r rack-TDAQ rack- 3 HE PCs One Full L2 rack-TDAQ rack- 30 HLT PCs One Switch rack-TDAQ rack- 128-port GEth for L2+EB Partial EFIO rack-TDAQ rack- 10 HE PC(6 SFI - 2 SFO - 2 DFM) Partial EF rack-TDAQ rack- 12 HLT PCs Partial ONLINE rack-TDAQ rack- 4 HLT PC(monitoring) 2 LE PC(control) 2 Central FileServers One ROS rack-TC rack+ horiz. Cooling- 12 ROS 48 ROBINs RoIB rack-TC rack + horiz. cooling- 50% of RoIB surface: SDX1 underground : USA15 • ROS, L2, EFIO and EF racks:one Local File Server, one or more Local Switches • Machine Park: Dual Opteron and Xeon nodes, uniprocessor ROS nodes • Operating System: Net booted and diskless nodes, running SLC3 V. Vercesi - INFN Pavia

  6. Event Building TS = number of requests from SFI nodes for event fragments WT = number of events processed in parallel by L2 farm  = L2 accept ratio Parameterizing observed EB measurements, we better understand the conditions for stable & performant operations V. Vercesi - INFN Pavia

  7. Pre-series studies V. Vercesi - INFN Pavia

  8. ROD Crate DAQ • Last major upgrades • To accommodate detectors’ requests • New API for operational monitoring • To get “coherent” operation information • Simultaneous module readout in a crate • RCD creates a packet of info and sends it to Monitoring • A monitoring task analyses the packet • Further improvements in last TDAQ version • OKS schema for RCD and ROS simplified • Detector modules in a crate can be managed • Serially or in multi-threaded mode • Each state transition can be set as multi-threaded • Can dramatically improve initialization time • RCD plans • The RCD structure should be stable enough • All the main requests from detectors have been fulfilled • We continue supporting the implementations • Main task is now RCD commissioning • Goes on with ROD commissioning • Includes support to all detectors V. Vercesi - INFN Pavia

  9. Monitoring Framework V. Vercesi - INFN Pavia

  10. Monitoring • Reliable and stable GNAM CORE • Configurable from on-line database, with histogram history • Logging system (to migrate to ERS) • Access Manager to ATLAS conditions DB • Detector libraries • MDT, RPC, Tile, CSC, (TGC): in use for commissioning • Interface to Event Display available (for some detectors) • Main issue: Smart Monitoring • Automatically catch most low-level problems • Needed for reaching and maintaining stable running conditions • Common tools needed • Histogram comparison toolkit • Error message routing and filtering • Detector experts • Identify most common problem sources • Implement and test alarm strategies V. Vercesi - INFN Pavia

  11. HLT Selection algorithms • Lots of ongoing activities on algorithm development side • LVL1 and LVL2 algorithms run in 11.0.6 CSC production for all trigger slices • EF algorithms almost finished (target 12.0.3, next week) • Lots of ongoing work to improve/test/validate algorithms • PESA algorithms being reviewed (Project Milestone) • ID LVL2, ID EF, Calo LVL2 e/g reviews have already taken place • Muon LVL2+EF ~90% complete, Calo EF e/g in October • Calo jet/tau/etmiss simple extension of e/g • Building up comprehensive information about performance • Ongoing work for testing in athenaMT/athenaPT • Measurements of system (timing) and physics performance • LVL1/HLT AODs fully available in Rel 12 for trigger-aware analyses (Project Milestone) • Most data in place for ESD & AOD in release 11.0.5, Apr 06 • More hypothesis algorithms, slice configs and TriggerDecision in 12.0.1 V. Vercesi - INFN Pavia

  12. HLT Steering • Il cuore delle sequenze di selezione e decisione • Sviluppi basati sull’attuale implementazione (12.0.2) • Introduzione di selezioni topologiche elementari • Primi test attualmente in sviluppo nel settore della B-fisica (B→mm, J/Psi→mm, BS→DSp conDS→fp) • Introduzione di meccanismi per la sincronizzazione delle sequenze di algoritmi • Selezioni combinate di diversi oggetti (ad es. singolo e, singolo jet, singolo t) • Sincronizzazione delle selezioni topologiche • Iterazione del design di implementazione • Ridisegno dell’implementazione dello Steering • Nuove interfacce, più generali e flessibili, per gli algoritmi di feature extraction e hypothesis testing • Ristrutturazione della navigazione, disaccoppiata dallo Steering, per essere facilmente utilizzata nelle analisi Offline • Interfacciamento con l’introduzione del concetto di Trigger Chain • Fase di disegno quasi ultimata: inizio dell’attività di sviluppo e test V. Vercesi - INFN Pavia

  13. HighLighT e/gamma Electrons • Eff. w.r.t truth for ET=25GeV e± • Time per track in top events (~1.3 tracks/RoI with pTmin=0.5GeV) • Brem recovery to be optimized: in progress… • Optimizations ongoing also for eff/rej Photons • Use only calorimeter information at L2 and EF • Use CSC data, single photons ET = 20, 60 GeV, filtered QCD di-jets with ET(hard) > 17 GeV 98% 97% V. Vercesi - INFN Pavia

  14. Efficiencies for SM signals eff= # after L1+L2+EF # after kinematical cuts Z ee W en Z mm Z ee Use trigger optimisations giving 80% overall eff (outside crack) for 2e15i, e25i, e60. Efficiencies given after kinematical cuts: • 2e in ||<2.5 with ET>15 GeV for Zee • 1e in ||<2.5 with ET>25 GeV for We e25i • efficiency stays flat after turn-on. • Turn on curves similar for e60. Method for single-electron trigger efficiency determination from data in progress ( , ) V. Vercesi - INFN Pavia

  15. Trigger efficiency from Z → μ+μ- • Double Object (DO) method • Control sample: “Good” Z from 2 offline +- with loose selection cuts + 1± trigger signature satisfied • Trigger efficiency determined from counting in how many cases the second ± satisfies the trigger requirements • EF TrigMoore (MuonSpectrometer standalone) • EF TrigMoore (MuonSpectrometer + InnerDetector) For (µ20)>70% statistical uncertainty after 30min at L=1033cm-2s-1: ~1-2% Ongoing study using complementary method of orthogonal signature from ID (DOS method), almost finished V. Vercesi - INFN Pavia

  16. HighLighT muons Endcap: 1.05 <ll <2.4 6 GeV 20 GeV Large efficiency dishomogenity EndCap CSC data prelim. pT (GeV) Endcap stations outside magnetic field Innermost TGC wheels have reducedcoverage in small sectors • Barrel: • Final update of the LUT available: optimized with respect to the momentum resolution and to the efficiency • Hypothesis exists and the PT cut is optimized for the 6 GeV threshold • Endcap: • Bugs found in the TGC reconstruction code and in the analysis programs • Situation improved: harsh zones still remains, but results are understood now • Only TGC data has been used: try to measure muon PTand to parameterize the track path • An offline study of the momentum resolution obtained fom MDT data is also under way: preliminary results are in agreement with those we got using only TGC data. V. Vercesi - INFN Pavia

  17. HighLighT Jet/Tau/ETmiss • Very active trigger slices, just one year ago only small part of the selections were available (new groups joining TDAQ) • Jets • Algorithms for LVL1/LVL2/EF reco and hypothesis in place • Lots of work carried out to allow trigger-aware analysis • Ongoing work to understand physics performance • Tau • Algorithms for LVL1/LVL2/EF reco and hypothesis in place • Use tauRec at EF, support for other tau reco packages foreseen • Ongoing work to find best sequence at LVL2: first calo reconstruction or first tracking reconstruction • L2 selection ran successfully in athenaMT in 11.0.6 • ETmiss • Study performance of ETmiss at LVL1 and EF V. Vercesi - INFN Pavia

  18. Prescale Jets First place where to study and implement pre-scales Reconstructed Jet ET distribution for a combination of J1 to J8 samples, scaled according to their respective cross-section V. Vercesi - INFN Pavia

  19. Algorithms in Pre-Serie • Motivation • Get the trigger PESA algorithms working online • Make sure the latest offline developments get running online • Historical background • Tools allowing the emulation of online running were developed • athenaMT (LVL2) and athenaPT (EF) • A group of people was formed in order to provide full slice job options suitable for online running • Jets (Ignacio Aracena) • Taus (Pilar Casado, Richard Soluk) • Egamma (Xin Wu, Imma Riu) • Muons (Alessandro Di Mattia, Diana Scannicchio) • One single integrated job options built successfully able to run in the Pre-Serie L2 and EF farms from real data format (bytestream) • Try this integrated job options in an online partition with different input files • Single electrons, Top events, Muon events • Produce a unified input BS file to be tested online • Produce a BS output file from SFO and try extending the menu to other signatures (e.g. di-objects) using the same algorithms slices to see the trigger behaviour V. Vercesi - INFN Pavia

  20. Running L2 partition running ONLINE single electrons with the integrated job options in the nightlies of 11.0.6 V. Vercesi - INFN Pavia

  21. More running V. Vercesi - INFN Pavia

  22. Databases Online Detectors HLT farm COOL RelDB CORAL* OKS in-memory OODB Athena Server DCS Diagnostics Remote sites ATCN Network Calibration Updates including special Muons calibration path Tier-1 replica CERN Computer centre ATLAS Point1 Bypass (ATCN / CERN-IT) Tier-1 replica Offline master CondDB Online OracleDB Tier-0 recon replica Tier-0 farm Isolation / cut Data: in the DB, plus in (large) files referenced from the DB Diagnostics replica RDB systems used: Oracle, MySQL, sqlite *layer to decouple clients V. Vercesi - INFN Pavia

  23. Trigger Configuration • TriggerTool • GUI for DB population • easy and consistent menu changes for experts (LVL1 and HLT) • TriggerDB • stores all information to configure the trigger: LVL1 menu, HLT menu, HLT algorithm parameters (JO), HLT release information • stores all versions used, with a key • Configuration and Conditions DB • DB available at Point 1 and replicated to external sites • Retrieval of information for running • get information by key via two paths • extraction of data in XML/JO files • direct read-out • for both online + offline running Data Flow: offline user shift crew expert TriggerTool DB population scripts TriggerDB compilers Configuration System R/O interface online running offline running V. Vercesi - INFN Pavia

  24. Example database for TrigConf keys: stored in Conditions DB => Trigger conditions precisely known HLT menu HLT jobOptions HLT release LVL1 V. Vercesi - INFN Pavia

  25. HLT Large Scale Tests • Test complete HLT on many nodes • Scalability of DAQ, DB, HLT sw • Emphasis on DB, HLT (no LVL1) • DAQ: verify scalability & stability • DataBase: • Explore caching tools (DBProxy) • Detector oriented tests • 1000 to 1200 nodes • LST 2005: 600 (512 used for LVL2) • Will be provided by IT (agreed) with lxbatch etc. machines • Non-optimized network • Big enough for full size Level2, ¼ size Event Filter • Test Level-2 and Event Filter together (+EB and SFO) • As many trigger algorithms as possible • Large number of events, recycled • Collect monitoring info • Physics plots (produced by algorithms) • Detailed timing histograms, etc. • Need to have SHIFTS! V. Vercesi - INFN Pavia

  26. Tracking algorithms IdScan, modified to handle single tracks with large impact parameters SiTrack, version with special LUT tuned for muons not coming from IP Variables studied For T2Id, Offline and MC: Ntracks, d0, phi0, z0, eta0 Efficiency calculations Data: LVL2 wrt Offline Simulation: LVL2 and Offline wrt truth LVL2 ID in cosmics Cosmics at SR1: instrumented region, two SCT sectors V. Vercesi - INFN Pavia

  27. Analyses using trigger information as a “pre-processor” to correctly evaluate efficiencies, physics reach, etc. The reconstructed objects, used by the trigger are saved in the ESD/AOD file They can be used for comparison with truth/reconstructed information It is possible to re-play the trigger decision, by running the hypothesis algorithms on these objects Only the settings of the hypothesis algorithms can be changed in the analysis The effect of different threshold settings can be measured Trigger-aware analysis Data taking Production Analysis V. Vercesi - INFN Pavia

  28. Trigger & Physics Weeks • Follow up on TDAQ and Computing operation • e.g. where are the bottlenecks on (instantaneous and average) rates? • Updates on trigger slices and initial feedback on Trigger Aware Analyses in Release 12 • Work towards ultimate offline efficiencies and rejections, to be compared to corresponding figures for trigger selections • Especially relevant for Combined Performance e, t, m, b-tag • Follow up on minimum-bias event selection and analysis • Follow up on menu for L = 1 × 1031 cm-2s-1 (items and rates) • Small group formed to study the issue and propose a strategy • Ideas shown need to be developed into menus for commissioning and for early physics (optimisation for two purposes somewhat different) • Follow up on physics with b-jets • Efficiency to identify the jets for tagging in LVL2 in various physics scenarios - e.g. rates and RoI multiplicities • “Hot topics” • End-cap LVL2 muons • Forward-jet trigger • ETmiss rates (including beam backgrounds) • Next week end of October V. Vercesi - INFN Pavia

  29. Planning… Visibilità ragionevole fino a ~Marzo 2007 Schedula in revisione alla luce delle nuove informazioni su LHC: discussioni durante la TDAQ Week di Settembre: cioè adesso, in perfetta sovrapposizione con la nostra CSN1 (Ho il volo per Londra domani mattina) V. Vercesi - INFN Pavia

  30. ROS plan • Prima tranche dei ROS già pagata al CERN (275 KCHF) • Seconda tranche e resto delle network card ordinata e consegna ultimata di recente, pagamento previsto entro la fine dell’anno (275 KCHF) • Questo esaurisce il nostro contributo CORE al Read-Out System • Impegno importante previsto in ME per contributi all’installazione e commissioning dell’intero sistema • Sezioni coinvolte Pavia, Roma1, … V. Vercesi - INFN Pavia

  31. Data Collection plan • Other DataFlow PCs • Price enquiry finished for SFI, DFM, L2SV and pROS • Evaluations nearly complete • Choose which company(ies) to buy from • SFOs - specification to be finalised in ~1 month • 2007 CORE expected from Italy: 50 KCHF • Try and setup system capable of deliverying steady-state nominal SFO output from day 1 • Better position to exploit early running trigger and detector studies • No correlation with total resources available in HLT farms • e.g. send LVL1 output directly to off-line V. Vercesi - INFN Pavia

  32. Networks and Online plan • Switch Procurements (Chassis + Pizza Boxes) • Plan finalised, shopping list produced for 2006 • Sharing between FA’s agreed • File servers and Monitoring PC specs finalised • 135 KHCF su CORE 2006, inclusi monitoring PC e local file server: end of contributions V. Vercesi - INFN Pavia

  33. Infrastructure plan • SDX Infrastructure • Orders started - cable ladders, nuts & bolts, shelves for switches • About to decide on power distribution • Coolers for SDX • Order placed for 43 racks (all of upper level) • 2007 CORE expected from Italy: 80 KCHF V. Vercesi - INFN Pavia

  34. HLT farms plan TDAQ Resource Committee • HLT Processors • IT Market Survey completed • Specifications has been drafted: similar (but not identical) to ATLAS needs • Not possible to use a blanket contract for 2006: buy our own • Lively discussion about spending profile for this items • First iteration ended Saturday September 16th @ 3:44 (Big Ben time) • Propose today our best approach based on • Need to cover basic needs for commissioning, calibrations, first run period • Avoid buying anything that is not absolutely necessary • 95 KCHF back to CSN1 in 2006 • Other HLT/DAQ Items • First bills are appearing here - Rack coolers, inter-rack fibres, etc V. Vercesi - INFN Pavia

  35. Revised Cost Profile (KCHF) V. Vercesi - INFN Pavia

  36. Richieste ME (DAQ/HLT) • Per le responsabilità vedi la prima trasparenza • Installazione e commissioning ROS e HLT • 4 mesi uomo a Roma1 • 2 mesi uomo a Pavia • Commissioning DAQ muoni/pixel • 4 mesi uomo a Roma1 (incluso ROD Crate DAQ) • 2 mesi uomo a Pavia (incluso sviluppo Monitoring) • 9 mesi uomo a Genova (incluso HLT steering) • Event Building, SFI e SFO • 8 mesi uomo a LNF (inclusi Large Scale Tests) • Deployment degli algoritmi di selezione on-line • 2 mesi uomo a Pavia V. Vercesi - INFN Pavia

  37. Conclusioni • Stato attuale del progetto HLT/DAQ ben allineato con le scadenze future di ATLAS e di LHC previste nel 2007 • Il progetto è certamente complesso e anche le responsabilità e i finanziamenti italiani coprono diversi settori • Il sistema di HLT/DAQ ha legami profondi con tutte le altre aree di sviluppo: rivelatori, software online e offline, networking, performance di fisica, etc • Sono necessari maggiori contributi alla forza lavoro per la parte di installazione e test • Impegno fondamentale di consentire a tutti i rivelatori una fase efficiente di commissioning • Inserimento degli algoritmi nei test di cosmici • Realizzazione di catene complete di read-out e successivamente di Event Building • Le slice HLT hanno raggiunto uno stadio di maturità avazata, sono integrate nell’ambiente on-line e costituiscono gli ingredienti per la costruzione e lo studio dei Menu di Trigger • Il Progetto è in fase di revisione (management) • Necessità di adatttarsi alla nuova fase dell’esperimento • Coerenza con l’ATLAS Operational Model • Impegno perché gli italiani mantengano i ruoli di visibilità che si sono meritati V. Vercesi - INFN Pavia

More Related