Thoughts on Data Transmission US-ATLAS Upgrade R&D Meeting UCSC 10-Nov-2005 A. A. Grillo

1. Thoughts on Data Transmission US-ATLAS Upgrade R&D Meeting UCSC 10-Nov-2005 A. A. Grillo SCIPP – UCSC Thoughts on Data Transmission

2. Old Guidelines for the Present SCT DAT • Avoid shared parallel write busses where one failed IC tends to take out the whole bus, i.e. use serial data paths. • Provide redundant data transmission paths onto and off of the detector. • Employ optical transmission for all high speed communication between the detector unit and the off detector electronics. • Saves some services volume (fibre vs. wire). • Minimizes interference between different detector units making scale up based upon test results of a small subset more viable. • Provide a mechanism to bypass a dead chip. • All electrical communication between ICs or between the detector units must be via low amplitude fully differential signals. • Minimizes electrical noise. Thoughts on Data Transmission

3. Present SCT Data Transmission System Apologies for the extremely old picture. Two data links, 1 clk, 1 comper module VCSELs not LEDs Ignore patch panel and fibre lengths Redundantclk & com Thoughts on Data Transmission

4. Interconnects on Each Module One Detector Module with Twelve ICs 2 serial outputs 2 serial outputs Clk & Com not shown but bussed in parallel to all ICs. Likewise for redundant Clk & Com. “N” normal“B” bypass Thoughts on Data Transmission

5. Transmission Data Rates & Multiplexing • The natural clock frequency of the readout ICs is the beam crossing frequency or perhaps x2 the beam crossing frequency. • Now at BC Frequency = 40 MHz • Could become 67 MHz, 80 MHz or even 100 MHz • Optical transmission now typically operates at GHz rates • If such rates are attainable with rad-hard components, fibre count will be reduced. • Operating the optical transmission at GHz rates implies multiplexing • Difficult to operate electrical transmission at those rates • Multiplex several serial chains with several ICs in each chain (figure). • The number of chains and the number of ICs in each chain will depend upon the details of the detector unit (e.g. stave). • Higher rates for each chain will add flexibility for layout options. • Each chain may require phase adjustment. Thoughts on Data Transmission

6. Possible Multiplexed Architecture Thoughts on Data Transmission

7. The Present Bypassing Scheme • The present bypass scheme has worked well but some difficulties: • The layout of the hybrid circuit has added complexity with the addition of four extra traces for each IC. The differences in capacitive load of the normal and bypass traces required slightly different designs for the two drivers/receivers in order to keep power to a minimum. • The connections to bypass an IC at the edge of the hybrid required very long traces. These long traces required a much more powerful differential driver (i.e. more power consumed) and these signals became the function limiting the operational speed of the IC. • Some of the redundant paths so complicated the topology of the hybrid layout that one was not implemented precluding the bypassing of one IC. Thoughts on Data Transmission

8. A Simpler Bypassing Scheme Define two directions for the data flow (e.g. left and right) If an IC fails, the data flow direction of ICs are arranged such that all to the left of the dead IC send data to the left and all to the right of the dead IC send data to the right (figure). Both left and right drivers drive only short path to adjacent IC Add third driver (powered only when needed) to drive long line to multiplexer or displaced section of chain. (Equivalent of master driver on present IC.) Number of drivers and hybrid traces are not reduced but trace routing is simpler and drivers are more efficiently matched to loads. Thoughts on Data Transmission

9. Bypassing a Dead IC Thoughts on Data Transmission

10. Configuration Read Back • The present SCT readout IC, the ABCD, does not allow for read-back of its configuration registers. • Tests for single event upset in a particle beam indicate that we should not have a problem if we periodically reload the registers. • The higher fluences of the upgraded ATLAS and the smaller geometries of the IC structures could create a worse problem. • Cannot tell in situ what the real error rate is; the re-load rate must be set to the worst case estimate. • Upgrade IC should provide for read back of configuration registers. • To minimize fibres, must use same output as for data. • Protocol developed for the SCT readout provided for such configuration read back but not implemented because of space required for extra circuitry. Need to re-examine the protocol to see if it can be improved, then implement in the new IC design. Thoughts on Data Transmission

11. Command Protocol • Present command protocol developed to allow triggers, short commands and long configuration commands using the same input stream. • A primary 3-bit identifying sequence was chosen in order to allow consecutive triggers to appear with only a gap of two beam crossing periods. (An ATLAS trigger requirement) • Later relaxed to a longer gap. The Pixel IC made use of a longer bit stream making it more immune to bit flips in the command sequence. • Keep longer pattern as Pixels (if still allowed by trigger) or increase data rate to allow longer sequence as long as SEU rate acceptable. • If beam crossing rate increases, the command data rate must increase proportionally. Thoughts on Data Transmission

12. Encoding of Clock and Command Signals • Clock and Commands carried on one fibre with biphase-mark encoding • Signal decoded on module into separate LVDS signals • Decoding could be in same IC as multiplexer • Would allow path for configuring multiplexer • Possible to multiplex more than one command stream on the same fibre. • Reduce time for frequent re-configuration (if necessary) • But … higher data rates will aggravate SEU problems • Need separate electrical drivers for each chain of ICs • Isolate failures • Possible need for phase adjustments for each chain • Could be advantage to maintain encoding in electrical chains and decode at each IC (figure). • Reduce number of drivers and dual traces Thoughts on Data Transmission

13. Keeping Clk/Com Encoded in E-Streams Thoughts on Data Transmission

14. Redundant Clock & Command Signals • Important to maintain redundant clock and command signals. • Present scheme of sharing clk/com fibre of adjacent module as redundant stream creates some problems. • Electrically couples adjacent modules destroying isolation • Electrical connection between modules is mechanically awkward. • Preferable to have two clk/com fibres per detector unit to maintain electrical isolation. Thoughts on Data Transmission