1 / 47

Trigger Upgrade for LHC Beams

This project aims to upgrade the trigger system for the LHC beams, increasing trigger rates, reducing latency, and improving system flexibility. It includes a two-layer calorimeter trigger and an integrated muon trigger.

bmullins
Download Presentation

Trigger Upgrade for LHC Beams

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. P04 – Trigger Upgrade, 401.4 Wesley Smith, U. Wisconsin L2 Manager, WBS 401.4 October 14, 2015 DOE Review -- P04: Trigger Upgrade

  2. Requirements and Design Summary • LHC beams cross every 25 ns with ~ 23 interactions → 0.5 MB of data, but can only store ~ 1 kHz of 0.5 MB data → Trigger • Trigger rates are driven by the increase in luminosity, the center-of-mass energy, and by higher Pile-Up (PU) • Detector readout systems limit Level-1 trigger output rate to 100 kHz, latency to 4 μsec. • Mitigate by improving e/γ isolation, τ id, μ pT resolution, μ isolation, jets with PU subtraction and L1 menu sophistication • Increase system flexibility with high bandwidth optical links and large FPGAs, and standardize on μTCA telecom standard • Build and commission upgrade in parallel with current trigger system to safeguard physics, decouple from LHC schedule • Overall design: • Two-layer calorimeter trigger with tower-level precision and PU subtraction • Integrated muon trigger combining all CSC, DT and RPCs in track-finding DOE Review -- P04: Trigger Upgrade

  3. Final Overall Upgrade Scheme Ops: MPC MZ ↑HCAL Upgrade↑ US CMS Detector Upgrade Project ↓Trigger Upgrade↓ CTP7 MTF7 (Cluster Finding) (Cluster Finding) Trigger Upgrade MP7 (UK) MTF7(MS FW) (Object Finding) (Object Finding) Vienna MP7 (UK) Vienna MP7 (UK) DOE Review -- P04: Trigger Upgrade

  4. 401.04 Trigger Organization Chart • Muon Port Card & Sorter: • L4 Manager: Paul Padley • Muon Track-Finder:L4 Manager: Ivan Furic • Ivan Furic • CERN Integration:L4 Manager:Pam Klabbers • Stage-1 Calorimeter Trigger:L4 Manager:JeffBerryhill CD-23 Review -- Trigger Upgrade

  5. Muon Trigger WBS Detail → Calorimeter Trigger M&S on Ops. Prog. Institutions: University of Florida – Gainesville Northeastern University Rice University DOE Review -- P04: Trigger Upgrade Fermilab Director's Review -- US CMS Trigger Upgrade Wesley Smith, 16 July 2013

  6. Install fully paralleland higher bandwidthoptical path for CSC New CSC Muon Port Cards mezzaninecards installed in cavern during LS1 Prerequisite done byOperations Program Alleviates bottleneck & sends all segments from each CSC (robustness to PU and collimated signals) Build up new Track Finder in 2015 and start commissioning in parallel, ready during 2016 Phase 1 Muon Trigger Transition New system Old system Cavern Counting Room DOE Review -- P04: Trigger Upgrade

  7. Muon Port Card (Ops)(Rice - M&S on Operations Program) (60 needed + 15 spares + 5 test setups = 80) x 60 +20 spares • New mezzanine card with new FPGA and new link • 85 Mezzanine Cards manufactured • Installed on 75 MPC main boards • All 60 MPC main boards replaced, connected with new optical fibers DOE Review -- P04: Trigger Upgrade Use the existing MPC main board Backplane interface to TMB remains unchanged 3 original optical links still available

  8. Upgrade Muon Port Card • Summer 2015 upgrade link test • Tested with all 60 installed upgraded MPCs and SPs • Full paths tested: MPC (left) to patch panel (center) to SP (right) • Links with errors found, fixed • Firmware fully validated • Next: Continued support of legacy and upgrade MPC paths DOE Review -- P04: Trigger Upgrade

  9. EMUTF Hardware Counts • Require three μTCA chasses to process 12 sectors + sort output • MTF7 occupies 2 μTCA slots, • 12 units in system, each covers 60o sector • 1 Muon Sorter, also built on MTF7 • Use Boston U AMC13 for clocking and DAQ • Re-route and split optical signals using 12 passive optical patch panels↓ Chassis #1 Chassis #1 SP SP SP SP SP SP SP SP SP SP SP Chassis #2 Chassis #2 SP SP SP SP SP SP SP SP SP SP SP Chassis #3 Chassis #3 SP SP MS MS DOE Review -- P04: Trigger Upgrade

  10. Muon Track Finder card: MTF7 13 needed + 4 spares + 3 test setups = 20 Optimized for maximum input from muondetectors (84 input links, 28 output links) Dual card w/large capacity for RAM (~1GB)tobe used for pT assignment in track finding EMU Track-Finder(U. Florida) Back: Core FPGA card⇒ ⇐ Front:Optics card • Production nearing completion (next slides) DOE Review -- P04: Trigger Upgrade

  11. MTF7 Base Board Production • Have14 working production FPGA base boards, plus 2 prototypes. • 13 needed for system (12 processors + sorter) • New production was required for remaining 6 production modules for EMUTF (for spares and test stands) • Problem traced to poor PCB manufacture in first production (particularly vias). • Redesigned for newer I-Speed material with new PCB vendor, • Received all new PCBs • Submitted 4 new PCBs for assembly, tested: • Three tested fine out of box • Fourth had a solder bridge confirmed by x-ray under PT LUT mezzanine connector. • Reworked, and retested fine. • Submitted remainder for assembly Oct.1, 2 week lead time • Other issues: DC-DC converters • Part shorted by water/flux residue underneath • Easily fixed by compressed air DOE Review -- P04: Trigger Upgrade

  12. MTF7 Optics Card and Mezzanine Card Production • Optics card • All cards produced and working! • PT LUT Mezzanine • Four preproduction assembled cards were received from assembly, tested, and work fine • Have sent to CERN • Submitted remaining 11 modules for assembly 9/29 • 2 week lead time • Has not delayed commissioning, which has focused on data receiving and transmission DOE Review -- P04: Trigger Upgrade

  13. Full EMUTF System Installed @ P5 Input CSC patch panels 12 MTF7 processors DOE Review -- P04: Trigger Upgrade

  14. EMUTF System Status(U. Florida, Northeastern, Rice) • All boards and patch panels installed (12 sectors) • All of EMUTF system cabled from the Muon Detector • PCIe access to all boards verified • MPC fiber ID codes read out and verified • All upgrade links from MPCs are operational • Tests of full EMUTF system to new Global Muon Trigger • Successfully demonstrated transmission of collision muon candidates (recent LHC fill) from the detector through one MTF7 (60° sector) to Global Muon trigger. • Comparison done by counters, 50 kHz rate consistent with legacy system DOE Review -- P04: Trigger Upgrade

  15. EMUTF Firmware and Software • Some initial delays due to hiring FW and SW personnel • Personnel now on board and making up delays • MTF7 firmware completely ported to new Xilinx tools • Tested, and all features working • Developed FW for 10 Gbps async transmission from MTF7 to Global Muon Trigger • Facilitated the pattern and slice tests • Testing of the DAQ communication firmware module to AMC13 is complete • Low-level Software • PCI Express Interface SW is working with MTF7 • Interaction via PCI-express tested with loopback test • Top-level • A“cell” using new online framework has been made, with the integration of an “MTF7 Processor” class • Recently achieved communication with MTF7 hardware DOE Review -- P04: Trigger Upgrade

  16. uTCA Upgrade Muon Sorter - Rice • Goal: deliver all muon candidates from the EMUTF to the Global Muon Trigger, also: • Deliver local trigger requests • Apply algorithms (e.g. duplicate candidate elimination) • Retain local muon sorting function • Design: completely replace the VME Muon Sorter with uTCA electronics • Modular Track Finder (MTF7) base hardware (UF) • Mezzanine card for local triggering functionality • Status: 2014-2015 • Setup of uTCA test stand at Rice • Sorting algorithm testing and refinement • Local Trigger mezzanine card designed + fabricated • External optical loopback test performed at CERN, Rice • Currently working on muon transmission through RX Algorithm TX path and MTF7 DOE Review -- P04: Trigger Upgrade

  17. VME Muon Sorter Mezzanine Upgrade(Rice) • Goal: provide a possible fallback solution for transfers between “legacy” and “upgrade” data paths • Design: use existing VME MS baseboard, replace mezzanine with optical TX/RX. • Does not transmit all muon tracks • Status: Done • Two boards have been built, hardware has been tested at Rice DOE Review -- P04: Trigger Upgrade

  18. Calorimeter Trigger WBS Detail ← Muon Trigger Institutions: UI – Chicago Fermilab MIT Rice Wisconsin M&S paid byCMS-France DOE Review -- P04: Trigger Upgrade

  19. ECAL Energy(TCC) HCAL Energy(HTR) Regional Calo Trigger Global Calo Trigger HF Energy(μHTR) HCAL Energy(μHTR) oSLB Current L1 Trigger System Upgrade L1 Trigger System Layer 1 Calo Trigger oRM oRSC Entire Summary EM candidates Region energies Layer 2 Calo Trigger Calo. Trigger Upgrade in Parallel: Split inputs from ECAL and HCAL 2016: HCAL uHTR & assoc. Cal. Trigger Cards - STAGE 2 2015: ECAL, Legacy RCT & associatedCal. Trigger Cards* - STAGE 1 HCAL OpticalSplitters • ECAL: optical Serial Link Board (OSLB) and optical Receiver Mezzanines (oRM) connect to Present and Upgrade Calorimeter Trigger • All installed and being commissioned in CMS Global Runs • HCAL: optical splitters drive both HTRs and μHTRs • All uHTRs installed and driving Upgrade L1T (+ Current HF RCT via oRMs) US US UK UK DOE Review -- P04: Trigger Upgrade

  20. Stage-1 Upgrade in 2015(Operations and DOE-Nuclear) Layer-1 Existing RCT 18 crates RCT 0 … RCT 17 RCT 1 RCT 2 18 oRSCs(new) oRSC 0 oRSC 1 oRSC 2 … oRSC 17 USCMS oRSCs: DOE-Nuclear FY15 Eng: Ops 80%,Upgrade 20% - also Stage-2(e.g. CTP7 HW/SW/FW) Layer-2 MP7 (new) Algorithm Card(s) (1-3) Imperial HW + Core FW/SW USCMS Algorithm FW/SW CTP7 (new) Single Readout Card In Situ RCT Test / Monitor USCMS HW/FW/SW Global Trigger (existing) GT Inputs remain unchanged DAQ – AMC 13 (new) DOE Review -- P04: Trigger Upgrade

  21. oRSC Installation and Commissioning(Wisconsin subcontract from MIT DOE-Nucl.) • oRSC Purpose: Send all Legacy RCT output data to Upgrade Calorimeter Trigger processors • 18 cards (1/crate) each output: • 3 copies of 2 fibers at 10 Gbps  1-3 MP7s (process) • 1 copy of 2 fibers at 10 Gbps 1 CTP7 (RCT DAQ) • 1 copy at 2 Gbps legacy GCT (Legacy Parallel) • Installed and working for > 1 year • IBERT tests May 2014 • Production done Summer 2014 • Installation at CERN Sept. 2014 All 18 oRSCs mounted on RCT crates oRSC production board – U. Wisconsin DOE Review -- P04: Trigger Upgrade

  22. Cal Trig Stage-1 Status(U.I. Chicago, FNAL, MIT, Rice, U. Wisc) • Commissioning Completed July 2015 • RCT readout via CTP7 provides full +/- bunch crossing data • Detailed examination of calorimeter trigger signal timing • New Algorithms developed and commissioned • Pileup subtraction was implemented – first time in CMS L1T • Uses η-dependent count of regions with ET > 0 for each event • Tau clustering optimized • Heavy ion algorithms determined • Emulator/firmware comparisons validated • Parallel Running July – August 2015 • Validation of operation and performance (data vs. emulator) • Operating with Stage-1 Cal. Trigger as main calorimeter triggersince August 2015 • Reliable data-taking with good performance • Substantial improvements over pp Legacy trigger (plots follow) • Parallel running for HI testing now underway • Ready for HI running at end of the year. DOE Review -- P04: Trigger Upgrade

  23. Stage-1 Cal. Trig. Performance Example: Taus • Major improvement in Tau Trigger efficiency and rate • Relaxed Isolation Thresh.(GeV):rate (kHz) = 20:42.8, 28:14.5,30:3 (Relaxed Isolation) Stage-1 Improvement (28 GeV Threshold) DOE Review -- P04: Trigger Upgrade

  24. Stage-1 Cal. Trig. Performance Example: electrons • For the same plateau ~90% efficiency, the rate reduction for stage1 at ~25-35 GeV is 60%, while for legacy it is only 80% • 25 – 35 GeV is the range for the trigger thresholds used Rate Reduction Due to Isolation DOE Review -- P04: Trigger Upgrade

  25. Stage-2 Upgrade in 2016 Layer-1 18 CTP7s Each covering all h CTP 0 … CTP 17 CTP 1 CTP 2 USCMS 3 mTCA Crates DAQ via AMC 13s Layer-2 Full flexibility for traditional or time-multiplexed architecture Highest precision location of objects with dynamic clustering MP7 Algorithm Card(s) (9+1) Imperial HW + FW/SW USCMS SW mTCA Global Trigger DAQ via AMC13 DOE Review -- P04: Trigger Upgrade

  26. CTP7 Card – U. Wisconsin • CTP7 Goals for Cal Trig Layer 1 • Provide needed interconnect flexibility for all proposed architectures and link speeds • True Triple-Link-Rate, Multi-Clock-Domain Card • 6.4 and 4.8 Sync Rx, 10G AsyncTx in same MGT quad • Maximize the power available to the Virtex-7 690T FPGA for trigger data processing • Exploit latest-available technologies • e.g. ZYNQ Linux connected to V7 • First two units delivered Dec’13 • Excellent results in checkout • Full Production Complete Mar‘15 • Excellent checkout results continued • 6 Preprod., 50 Prod. CTP7s complete • 38 total CTP7s shipped to CERN DOE Review -- P04: Trigger Upgrade

  27. Production CTP7s CTP7 Without CXPs + heat-sinks ← Front Back→ CTP7s ← in Stage 2 Rack DOE Review -- P04: Trigger Upgrade

  28. Stage 2 Layer 1 Installation Complete! DOE Review -- P04: Trigger Upgrade

  29. Cal. Trig.Stage-2 Layer-1 Status • Hardware installation finished in April ‘15 • 1152 Layer-1 input fibers connected in June’15 • All 576 ECAL links tested and mapping validated • All 72 HF links tested and mapping validated • All 504 HBHE links tested and mapping validated • 864 Layer-1 output fibers connected June‘15 • All links tested and mapping validated • Firmware with production-ready DAQ functionality completed in August’15 • Full resolution input TPG DAQ readout • Layer-1 timed in, commissioned and stress-tested during TS2 in cDAQ • DAQ Status September ‘15: • All 3 Layer-1 FEDs timed in, commissioned and stress-tested in cDAQ during TS2 (1st Week of Sept.) • Operating in Global Data-taking since 9/25. DOE Review -- P04: Trigger Upgrade

  30. Stage-2 Layer-1 Software • Integration with the RCT Trigger Supervisor complete • 18x CTP7 and 3x AMC13 configuration tested in local mode • Link masking, alignment and alignment checks • DAQ configuration (AMC13 and CTP7 DAQ FW) • Running now in global data-taking • Achieved Milestone of Parallel Operation with Stage-1 • Data from Stage-2 / Layer-1 now always included in runs at P5 • Completing Layer-1 Data Quality Monitoring • Data unpacker + Offline DQM is running on recorded data • Online SW Longer Term Plan • Migrate Layer-1 from Trigger Supervisor to new online framework as soon as it becomes available for production use at P5 • Fully tested with latest release DOE Review -- B04-2: Stage 2 Calorimeter Trigger

  31. Cal. Trig. CIOx Status – U. Wisc. Essential custodial functions removed from individual CTP7s onto a single board connected to CTP7s via the backplane –savings in cost and circuitry • Design and layout complete • Parts for 5 boards are in hand • RFQ for PCBs sent • Making 3 prototypes for testing Final Production of 10 Boards DOE Review -- P04: Trigger Upgrade

  32. L1 Trigger Plans for Next Year • Commission upgrade EMU trigger system and operate in parallel during rest of 2015, thenswitch to operate as main trigger during 2016 run. • Commission full Calorimeter Trigger Stage-2 (Layer-1 + Layer-2) system and operate in parallel for rest of 2015, then operate as main trigger for 2016 Run • Install CIOx cards to provide additional diagnostics and programming access. • Improve performance for both using information from data-taking during 2016 run • Improvements in firmware, software, DQM DOE Review -- P04: Trigger Upgrade

  33. Trigger L2 Schedule/Milestones CD2/3 CD4 CD0 CD1 LHC Schedule LS 1: Splice rework Physics TS Physics ETS Physics TS Physics LS 2 L1T MPCM connection Muon Trigger Prototype Preprod Prod Calorimeter Connection Soft/Firmware Calorimeter Trigger Preprod Prod Prototype Soft/Firmware DOE Review -- P04: Trigger Upgrade

  34. Critical Path – Trigger • On time to finish at end of FY16 • Trigger hardware fabrication is largely complete • FY16 involves commissioning, FW, SW, and support Muon Trigger Calo Trigger Transition to operations DOE Review -- P04: Trigger Upgrade

  35. Budget (BAC) Status L1T • More than 75% (by scheduled budget) of project tasks are started • Hardware tasks are over 90% complete • Remaining tasks are mostly software and firmware development • Using data-taking in parallel operation to inform final stages of development • What is mostly to come is Labor DOE Review -- P04: Trigger Upgrade

  36. EVM L1T • Overall CPI, SPI, EV on track • Muon CPI, SPI lower due to delay in hiring EMUTF SW, FW personnel. • Now on board DOE Review -- P04: Trigger Upgrade

  37. EV Performance – L1T • Good performance overall • Actuals, Earned below budget due to delay in hiring EMUTF SW and FW personnel. • Now on board and contributing – schedule is catching up DOE Review -- P04: Trigger Upgrade

  38. Resource Profile • Labor dominated for rest of project DOE Review -- P04: Trigger Upgrade

  39. 401.04 Trigger – 3 Open Risk Threats Risk applies to 11 firmware activities, each of which may each be impacted by these amounts • Cannot impact CD-4 • Muon and Calorimeter Trigger fabrication is largely complete • Now being commissioned – transfer to operations at end of FY16 DOE Review -- P04: Trigger Upgrade

  40. Conclusions • Calorimeter and Muon Trigger Hardware production almost complete. • Full upgrade EMU Trigger System being commissioned for parallel operation while Legacy EMU Trigger System is the main trigger. • Calorimeter Trigger Stage-1 is the main Calorimeter trigger as of August. • Testing through parallel operation with Legacy Calorimeter Trigger System since June. Fully operational since July. • Immediate Physics Benefits to CMS • Calorimeter Trigger Stage-2 Layer-1 in full operation as of end of September • Running in parallel with Stage-1 • Fully expect that L1T will finish successfully on time and within budget DOE Review -- P04: Trigger Upgrade

  41. Additional Slides DOE Review -- P04: Trigger Upgrade

  42. BAC, EU on Not-complete Activities DOE NSF TOTAL DOE Review -- P04: Trigger Upgrade

  43. L1 Trigger BAC Status DOE Review -- P04: Trigger Upgrade

  44. Resource Profiles – now labor dominated (breakdown next) TOTAL Travel labeled as M&S DOE Review -- P04: Trigger Upgrade

  45. FTE Profiles L3 L1T DOE Review -- P04: Trigger Upgrade

  46. Calorimeter Trigger Components DOE Review -- P04: Trigger Upgrade

  47. Muon Trigger Components DOE Review -- P04: Trigger Upgrade

More Related