Very Forward Muon Trigger and Data Acquisition Electronics for CMS: Design and Radiation Testing

1. Very Forward Muon Trigger and Data Acquisition Electronics for CMS: Design and Radiation Testing Jason Gilmore Vadim Khotilovich Alexei Safonov Joe Haley 21 Sept 2012

2. CMS Endcap Muon System h = 0.8 Focus on the innermost Cathode Strip Chambers: ME1/1 CSCs h = 2.4 TWEPP 2012

3. Overview of the CSC System VME ME1/1 ME1/1, High-Eta TWEPP 2012

4. Overview of the CSC System VME ME1/1 ME1/1, High-Eta New: Increase to 7 CFEBs, all with fiber links TWEPP 2012

5. CSC: Frontend Trigger Problem • Out-of-time PU induces deadtime at higher luminosity  look at PU100 • Particular issue is the ME1/1 “TMB” building chamber track segments • Two aspects making ME1/1 special: • Very high occupancies • ME1/1 TMBs effectively serve two chambers (inner ME1/a, outer ME1/b) • Need better FPGA to maintain efficiency • The algorithm is ready (V. Khotilovich) • Design of prototype TMB completed • Improve muon trigger efficiency for |h|>2.1 • Rate increase compensated by requiring 3 station coincidence for |h|>2.1 • With new TMB can do w/o efficiency loss • Needs firmware modifications in CSCTF TWEPP 2012

6. CSC TMB Mezzanine 2012 Virtex 6 FPGA + XF128 PROM QPLL Finisar Transceiver, only on test boards Snap 12 Fiber Transmitter socket (used only on test boards) Snap 12 Fiber Receiver - fibers from 7 DCFEBs Signal-level translators 3.3 V to 2.5 V Dimensions: 7.5” wide by 5.9” high 11.1 mm clearance from TMB main board TWEPP 2012

7. TMB Mezzanine Location TWEPP 2012

8. Radiation Studies for New CSC Boards • Will the new components survive the expected exposure in the CMS Endcap at HL-LHC? • Not just CSC trigger boards, but also for the front-end boards • DCFEBs and ODMB, as well as new TMB mezzanine • Expected 1 MeV neutron fluence: 3 *1012 n/cm2 over 10-years • 9 krad dose, do tests up to ~30 krad level for 3-times safety factor • Will the Single Event Upset rates be unacceptably high? • FPGAs, fiber links, etc. used in front-end boards • Expected 20 MeV neutron fluence: 2.7 *1011 n/cm2over 10-years • Measure SEU cross sections for individual design elements • Initial radiation testing was done in 2011 • Digital components were tested with 55 MeV protons • Performed at the Texas A&M University Cyclotron facility • Other components tested with ~1 MeV neutrons • At Texas A&M University Nuclear Science Center reactor • A series of 5 exposures to test 40 different components • Results to be published soon, paper accepted by NIMA • Additional 2012 tests completed recently at UC Davis TWEPP 2012

9. Voltage Regulator Radiation Tests • Testing performed at the Texas A&M Nuclear Science Center • 1 megawatt reactor operating at 6 kW, provides 9.9 *108 n/cm2s • Multiple samples of several COTS regulators, two exposures • First exposure represents ~10 HL-LHC year dose (10 krad) • Second exposure adds ~20 HL-LHC years, total of 30 year dose (30 krad) • Regulator performance tested before and after each exposure • Regulators were unpowered during exposure • Some regulators showed no ill-effects • National Semi LP38501 and LP38853 • Micrel 49500 and 69502 • TI TPS74901 • Others did not fare so well… • Maxim 8557 • Sharp PQ035ZN1, PQ05VY053, PQ070XZ • TI TPS75601, TPS75901 • No improvement seen with additional cool-down time • More parts were tested later, all are summarized in following slides… TWEPP 2012

10. Summary of All Reactor Tests (1) TWEPP 2012

11. Summary of All Reactor Tests (2) TWEPP 2012

12. Summary of All Reactor Tests (3) TWEPP 2012

13. SEU Testing of COTS Components (1) • Testing performed at Texas A&M Cyclotron • 55 MeV protons with uniform flux, collimated to 1.5” diam • Maximum proton flux ~3 *107 cm-2s-1 • 45 to 90 minute runs on each target device, 5-10 kRad in these tests • Two samples tested for each COTS component • Reflex Photonics 3.5 Gbps Snap12 Receiver: model r12-c01001 • Random PRBG data patterns @3.2 Gbps on each of six links • FPGA drives data to Transmitter, fiber connects to Receiver and carries data back to FPGA • SEU cross section: s = (8.2 ± 0.3) *10-9 cm2 • Also tested to ~30 krad TID at TAMU reactor: no problems • Reflex Photonics Snap12 Transmitter: t12-c01001 • 3.5 Gbps, tested for use in ODMB upgrade • PRBG data patterns @3.2 Gbps on six links • s = (7.3 ± 2.4) *10-11 cm2 • Finisar Optical Transceiver: ftlf8524e2gnl • 4.25 Gbps, tested for use in DCFEB upgrade • Transmit randomized GbE data packets to PC • s = (1.0 ± 0.3) *10-10 cm2 TWEPP 2012

14. SEU Testing of COTS Components (2) • Xilinx Virtex-6 FPGA: xc6vlx195t-2ffg1156ces • No SEU mitigation in firmware for this study • Goal is to measure cross section of individual FPGA elements • Determine where mitigation is necessary • GTX Transceiver (55% used) • PRBG data transfers @3.2 Gbps • s = (7.6 ± 0.8) *10-10 cm2 • Block RAM (74% used) • 4 kB BRAM “ROMs” readout to PC • s = (5.7 ± 0.6) *10-8 cm2 • CLB (38% used): • 4 kB CLB “ROMs” readout to PC • s = (3.7 ± 0.5) *10-8 cm2 • TI Bus-Exchange Level-Shifter: sn74cb3t16212 • Randomized data patterns sent through all 24 signal paths • No SEU observed, s90% < 1.7 *10-11 cm2 TWEPP 2012

15. Impact of the 2011 SEU Measurements • How would these cross sections affect CSC operations in HL-LHC? • Snap12 Transmitter: < 1 SEU per year per link • Snap12 Receiver: ~1 SEU per week per link • These typically just affect a single data word • Finisar Optical Transceiver: ~7 SEU/day/link • Typically just affects a single data word • Low rate, less than one error in 3 *1013 bits • FPGA GTX Transceivers: ~3 SEU/year/link • FPGA Block RAMs: ~9 SEU/day/chip • These typically affect a single bit in a single cell • Need to investigate mitigation for FPGA BRAMs • FPGA CLBs: ~6 SEU/day/chip • Need to investigate mitigation for FPGA CLBs TWEPP 2012

16. Recent 2012 Radiation Studies • Testing at UC Davis Cyclotron • 64 MeV proton beam, flux up to ~1 *109 cm-2s-1 • Many of the same parts from previous SEU tests were retested using the same circuit boards • Snap12 parts are the only exceptions • New Emcore transmitters were tested in 2012 • All chips survived 30 kRad dose* • Monitored power for signs of latchup (none observed) • Some FPGA tests included mitigation this time • Enabled native ECC feature in Block RAMs • BRAM test used Read & Write under software control • Software designed to distinguish each failure mode • CLB tests based on triple-voting system • CLBs were implemented as a system of shift registers • Given common inputs and checked against each other • Error counts were recorded in registers and monitored by software TWEPP 2012

17. SEU Test Results 2012 (1) • Reflex Photonics 3.5 Gbps Snap12 Receiver: r12-c01001 • Random PRBG data patterns @3.2 Gbps on each of eight links • These SEUs only caused transient bit errors in the data • 2012 SEU cross section result: s = (6.4 ± 0.2) *10-9 cm2 • Combined 2011+2012: s = 9.5 *10-10 cm2 per link • Similar to 2011 result, about 40% smaller • Emcore 3.3 Gbps Snap12 Receiver: EMRS1216 • Same PRBG test as above • 2012 SEU cross section result: s = (9.8 ± 0.2) *10-9 cm2 • This gives s = 12 *10-10 cm2 per link • Similar to Reflex Photonics combined result, about 30% larger • Emcore 3.3 Gbps Snap12 Transmitter: EMTS1216 • Same PRBG test as above; tested two of these parts • These SEUs only caused transient bit errors in the data • 2012 SEU cross section: s = (1.7 ± 0.2) *10-10 cm2 • This gives s = 2.1 *10-11 cm2 per link • Nearly double the 2011 result for Reflex Photonics transmitter • Still very low rate of SEUs, so not a concern TWEPP 2012

18. SEU Test Results 2012 (2) • Finisar Optical Transceiver ftlf8524e2gnl: Transmit side • Gigabit Ethernet packet transmission tests to PCI card, 4 kB @ 500 Hz • Bad or missing packets received at the PC are “transmit” SEUs • These SEUs caused lost GbE packets and rare “powerdown” events • 2012 SEU cross section result: s = (4.3 ± 0.3) *10-10 cm2 • About 6 times the 2011 result; consistent with *6 increase in link duty cycle • Correcting for real CSC transmitter duty cycle: s = 8.2 *10-9 cm2 per link • We expect to see ~1 SEU per link per day during HL-LHC running • Finisar Optical Transceiver ftlf8524e2gnl: Receive side • New test in 2012, load the BRAMs with data and read them back • Errors read back twice the same way are “receive” SEUs • These SEUs only caused transient bit errors • 2012 SEU cross section: s = (7.5 ± 0.1) *10-9 cm2 per link • We expect to see ~1 SEUs per link per day • *Three Finisars tested: one died at 33 krad, another at 41 krad • The third chip survived with 30 krad • TI Bus-Exchange Level-Shifter: sn74cb3t16212 • Still no SEU observed,2011+2012 result: s90% < 4.0 *10-12 cm2 TWEPP 2012

19. FPGA SEU Results 2012 • GTX Transceiver (55% used in FPGA) • Random PRBG data patterns @3.2 Gbps on each of eight links • These SEUs only caused transient bit errors in the data • 2012 SEU cross section result: s = (10 ± 0.8) *10-10 cm2 • Similar to 2011 result, ~50% larger, consistent with additional active links • Block RAM (74% used in FPGA) • Built-in ECC feature was used to protect data integrity • Software controlled write and read for BRAM memory tests • No errors were detected in the BRAM contents: mitigation at work • 2012 SEU cross section: s90% < 8.2 *10-10 cm2 • CLB (43% used in FPGA) • Most of the logic is a shift register system with voting • Some of it was unvoted logic for control and monitoring • This masks the “mitigation” effect of voting somewhat • 2012 SEU cross section result: s = (6.0 ± 0.5) *10-9 cm2 • Much smaller than 2011 SEU result, factor of 6 better: mitigation at work • With this we expect ~1 CLB SEU per FPGA per day TWEPP 2012

20. Conclusion • TMB Mezzanine development coming to a close • We have a design and production plan for new CSC electronics • This will maintain a high level of efficiency for the foreseeable future • Prototypes have been built & tested Good results from radiation tests • We have found satisfactory COTS parts to meet all our design requirements • Development work still needed in SEU mitigation firmware • Final CSC ME1/1 Electronics production begins soon • Need over 500 DCFEBs, plus spares: starts next month • Need 72 each for new TMB and ODMB, plus some spares • Start producing these early in 2013 • Installation in CMS from June-August 2013 TWEPP 2012