Optical Data Links in CMS ECAL

1. Optical Data Links in CMS ECAL • CMS ECAL architecture and needs • Optical Data Links in HEP Experiments • ECAL Data Link system description • Data Link components • Data Link system • Opto-Hybrid assembly, Quality Assurance • Schedule

2. ECAL Silicon Tracker HCAL Barrel Preshower + Endcap Solenoid Return Yoke Muon Chambers The CMS Detector

3. Front-end board Trigger sums Data Supercrystal APD/VPT ADC Preamp CMS ECAL front-end architecture • ECAL has ~77,000 lead tungstate crystals arranged in supercrystals of 25. • Front-end electronics of each supercrystal send the data off-detector via the optical links. The architecture and the needed data rate require ≥600 Mb/s with modularity: • for Data: 1 link / supercrystal • for Trigger: 1 link / supercrystal (barrel) • 5 links/supercrystal (endcap) • Total data + trigger: ~9000 links

4. Optical Links and HEP Generic architecture of an optical link: data in fiber data out MUX Ser Tx Rx Deser DEMUX “Optical Layer” e/o, transport, o/e “First Overlay” encoding/decoding, usually performed in same chip as serialization/deserialization “Framing” Multiplexing multiple “conversations” in each link. Not usually relevant for HEP since systems are designed to run each link continuously at maximum data rate.

5. fiber Tx Rx “Optical Layer” e/o, transport, o/e Optical Layer considerations - 1 Tx: In HEP, usually either Edge-Emitting Laser Diode or VCSEL. Rx: Typically photodiode or photodiode array (typically of 12), followed by digital amplifying ASIC. Tx considerations: Edge-Emitting: linear response (relevant if you design an analog link) VCSEL: said to be more rad-resistant due to smaller active area, but practical experience is that fractional change of behavior per unit irradiation is typically similar. VCSEL is cheaper to produce and test on wafer. Was once thought that they would entirely replace Edge-Emitting. But benefit is diluted since packaging dominates the cost of both. In any case: device chosen must be consistent with the system speed and environmental specifications (often unique to HEP; more on this later). Edge-Emittingdiagram VCSEL diagram test on wafer Rx considerations: Speed, wavelength, saturation and sensitivity limits must be consistent with system specifications.

6. P. Denes Optical Layer considerations - 2 Optical fiber considerations: Two principle kinds: Single Mode (SM) and Multi Mode (MM). Differences in impurities in the silica and differences in core and cladding diameters give different spectral transmission characteristics. SM: lower attenuation, is the favored choice for telecom (longhaul) distances (10 km and greater). Uses smaller core to allow only one mode to propagate, making connector alignment more critical (and expensive). MM: favored for “datacom” (LAN) distances (~5 to 10 km). In principle cheaper than SM, but fiber itself may be more expensive since market is smaller (datacoms did not take off). HEP distances are short (~100m), so SM/MM choice is not critical. Usually free to select the cheapest solution satisfying environmental specifications. Qualification is essential: Early ECAL qualification of MM fiber of various suppliers: Good fiber loses <50% transmission after 1013 p/cm2. Bad fiber turns black.

7. fiber Ser Tx Rx Deser “First Overlay” encoding/decoding, usually performed in same chip as serialization/deserialization First Overlay considerations • Serializer/deserializer considerations: • Encoding: Two principle protocols are CIMT (“G-Link”, Agilent proprietary) and 8-bit/10-bit. Both share the following characteristics: • introduce an overhead of two bits per 8 bits of data • are usually employed continuously transmitting, either data or idle pattern • aid in maintaining synchronization of transmitter and receiver • allow some error checking • stabilize 0 and 1 levels by keeping number of bits of each the same • Principle protocol differences: G-Link is considered more robust in terms of maintaining and recovering synchronization. But it is proprietary. At the moment the fastest CIMT deserializer runs up to 1.4 Gb/s. In general there is no guarantee that a company employing proprietary technology will continue developing new products with it, nor any guarantee that it will not discontinue existing ones. • Speed and bits serialized: Typical scheme at LHC experiments is one (two) 16-bit words serialized at LHC clock rate of 40 MHz implying optical link speed of 800 (1600) Mb/s of which 80% is data and 20% is overhead.

8. General considerations HEP experiments typically have many specifications which are unusual in industry. The closer the transmitter is to the center of the detector, the more likely this is to be true. Every need has its consequence(s): The typically O($1M) procurement for a HEP experiment is relatively small for any company large enough to collaborate in the task. Development teams in laser transmitters are smaller than one might think, typically 10-30 people for a large company. => The experiment will incur significant development and qualification effort and costs. In designing an optical link for a HEP experiment: Typically start with a shopping list of needs. From that define specifications and select/develop technology. End up with a laundry list of “issues”.

9. Dense multi-ribbon cable Ruggedized ribbon Off Detector Pigtail fiber Front End GOL ser. ASIC: CERN MIC 12 96 12 Rx module GOH 1 1 12 12 Laser diode GOL Digital amp. ASIC PIN photo-diode array In-Line Patch Panel Distributed Patch Panel Back-end Patch Panel Digital Optical Receiver: NGK Edge-Emitting Laser Diode: ST Microelectronics CMS Optical Data Link System Diagram • GOL Opto-Hybrid (GOH): ECAL designs, prototypes, qualifies, defines manufacturing specifications, procures the manufacture, tests samples during production. • 12-channel NGK Rx: Off-the-shelf component: ECAL qualifies, procures, tests samples during production. • Fiber, connectors and adaptors: ECAL uses solutions already developed and procured for CMS Tracker.

10. GOH Components: Transmitter - 1 The transmitter of the data link is the GOH. Its principle components: • The Gigabit Optical Link (GOL) serializer and laser driver ASIC • Designed by CERN Microelectronics group. • Implemented in 0.25μm CMOS technology employing radiation-tolerant layout practices. • Speeds of 0.8 and 1.6 Gb/s. ECAL will run the GOH at 0.8 Gb/s. • Two protocols (G-Link and 8b/10b). ECAL will use 8b/10b for data and G-Link for trigger. • The laser diode, made by ST Microelectronics, based on a Mitsubishi die • Custom-made for CMS Tracker (linear response), suitable also for digital use in ECAL (rise time consistent with 800 Mb/s operation). • Die wafer lots are radiation-qualified (gammas and neutrons) before assembly into laser diodes. • Pigtail fiber is attached and calibration data are taken by ST. • Some specifications of the GOH: • Receives LV, clock, control and 16-bit parallel digital input at 40 Mb/s from FE board via connector. • Transmits serialized optical output at 1310 nm, 800 Mb/s, either protocol, via pigtail fiber. • Output signal power ~-6dBm, depending on bias levels chosen. (0 dBm = 1 mW)

11. Components: Transmitter - 2 Status of the GOH: • The design is complete, prototypes exist. • Driven by GOH evaluation board (a modified GOL eval board), gives a clean eye diagram at 800 Mb/s. • Functions well within the full data link (more on this later).

12. 12 12 96 1 Components: Fiber and Connectors • Fiber and connectors are adopted from the CMS Tracker system • All specifications consistent with use in ECAL as well (e.g. temperature, rad-hardness, attenuation, safety). • ECAL benefits from commercial work, qualification and acceptance test setups already a part of Tracker program. • ECAL’s contribution is the in-kind to staffing of acceptance tests. MFS adapter (Diamond) sMU-MFS fanout (Ericsson, Sumitomo, Diamond) MU-sMU adapter (Sumitomo) MFS-MPO multi-ribbon cable (Ericsson, Diamond)

13. Rx module 12 12 Digital amp. ASIC PIN photo-diode array Components: Receiver • 12-Channel digital NGK Rx • COTS device; ECAL tasks are to qualify and procure. • Specifications are consistent for use within the full data link, e.g.: • wavelength: 1310 nm • speed: up to 1.25 Gb/s • saturation: -5 dBm • sensitivity: -18 dBm • Functions well within the link (see following slides).

14. GOH Evaluation Board Rx Evaluation Board Deser Evaluation Board Counter Rx Err ~2 dB Attenuator Rx Out Scope Clock Generator Data Link System Tests - 1 • Early system tests involved: • GOH driven by GOH evaluation board • Rx on Rx evaluation board • Deser (G-Link or 8b/10b) on Deser eval board • counted Deser word errors (frame errors)… Noise data generated on neighboring channels Channel under study • …in this case, to study effect of crosstalk in the Rx • Conclusions: • Link functions well, BER < 10-13 should be obtainable • Rx is well within sensitivity spec (-18 dBm) • Crosstalk costs about 2 dB of the optical power budget

15. (Four) Trigger-generating pulses High = Des “Data Valid” High = Des “Rx Err” Data Link System Tests - 2 • First test of the Data Link with the prototype FPGA version of the Front-End board (mid-Feb 03): • Four triggers generate four bursts of data. • Each trigger involves the absence of one Control clock cycle. • Valid data seen by deserializer, except occasionally soon after the missing clock cycle(s). It is expected that this will be resolved when QPLL (CERN MIC) is integrated into Front-end and Off-detector boards. • The good news is that FE + Data Link works!

16. Optical Power Budget *Rx sensitivity spec is “-19 dBm typical, -18 dBm minimum”. We have so far observed better than –20 dBm. => At least 6 dB of margin in the power budget

17. Laser diode attachment and wire-bonding GOL BGA mounting Fiber clamp attachment PCB manufacture, passivecomponent mounting Wire-bond protective cover attachment GOH Assembly • GOH assembly involves a number of steps: • Manufacturing Specifications document for the GOH is complete. We plan to submit the RFQ as soon as possible to have the maximum amount of time to test the first 100 (Engineering Run).

18. GOH QA and Link System Tests - 1 • Construction of first 100 GOH (Engineering Run) will begin when all of these are ready: • RFQ completed and manufacturer selected • Laser diodes are available • Tested GOL’s are available We plan thorough tests to verifyfunctionalityand reliability. • The tool for these tests will be our Bit Error Rate Test system. Based on PC, CMS “GIII DAQ” cards and extensive Altera programming. • Principle is the same as for the frame error tests: plot BER vs. optical power, make a quantitative measure of a stress on the system in terms of a “penalty” on the optical power budget. • Advantages over frame errors are that it is bit-by-bit, independent of protocol. • BERT system has been working since beginning of the year. Presently runs at 100 Mb/s: BER of <10-13 in ~1 day. • Present effort is to automate it to perform many measurements with minimal human intervention.

19. GOH GOH QA and Link System Tests - 2 • The functionality tests we plan include: Temperature sensitivity: GOH, Rx, deserializer. Sensitivity to LV fluctuations/noise. Fine-tuning of the eye diagram via GOH compensation network components. Integration of the Link into the full ECAL electronics chain. • The reliability tests planned for the GOH include: Vibration Mechanical shock Thermal cycling (-20°C to +40°C) Pass/fail eye diagram test Irradiation Accelerated aging Long-term stability Manufacturer Data Links group • The goal of the tests is to uncover and resolve any design or manufacturing reliability issues that may exist, and to be able to sign off on the full Link system design by the October ESR.

20. Schedule Qualification is crucialbut is the only “compressible” task. Schedule concerns: May 03: If laser diodes or tested GOL are late, time for testing of Eng Run is reduced. Sep 03: If GOH design modification is necessary, little time to test it before the Pilot Run.

21. Summary • Development of an optical link for a HEP experiment is a fascinating project, invariably involves a lot of co-development with industry. • ECAL Data Link system design and project have made a lot of progress in a short time. Starting with good components helps.