Belle-II bKLM Readout System Concepts, &c.

1. Belle-II bKLM Readout System Concepts, &c. W. Jacobs, G. Visser, A. Vossen Indiana University CEEM 4/8/2011

2. by Yusa-san (with some added info here) 16 “octants” 16 crates 1356 ch/crate 15 FEE boards/crate 2 cables/board 23 cables/crate fully utilized 7 cables/crate only 36 ch utilized current KLM Barrel : 21856†strips Endcap : 16128 strips Total : 37984 strips †TDR: 21696 ?? New readout: The organization is good, maintain same! Reuse all cables!

3. Readout Crate – Belle bKLM TDC cables Signal cables Controller Vanilla VME-J1 21-slot backplane BusTronic 101VMEJ121 OK – so here it looks certainly like 16 (not 15) FEE boards per crate – I don’t understand why… But, this is a small point that we can clear up sometime, I’m sure. There is some nonstandard power supply connections, need more info there but expect is fine.

4. Cables (re-use) bKLM layers in magnet steel Length 5.1 – 6.1 m KEL #8822E-100-171-F (#8821E-100-171-F) IU has purchased 5 for testing, and 12 board connectors #8830E-100-170L-F

5. Signals Z0= 50 Ω microstrip Z0= 113 Ω twisted pair 50 Ω Length 5.1 – 6.1 m 50 Ω Existing scheme (above) – NOTE ground at both ends of cable. Voltage/noise externally imposed between these grounds will simply add to signal+threshold input to comparator. Not great. Preferred scheme would break this ground connection at FEE boards: Z0= 50 Ω microstrip Z0= 113 Ω twisted pair 50 Ω Length 5.1 – 6.1 m 50 Ω

6. Possible frontend circuit MAX9010 Sync all hits to clock @ e.g. 250 MHz. Then capture common timer into register-per-channel • Optionally can tie the “ground” side of inputs • To ground • Simply together • To ground through capacitor • Or just leave them untied here (probably best) Have parts, plan on a prototype soon. Evaluation with real cable and mock detector, and signal from Tek AFG3252.

7. For the discriminator

8. Just some working notes / thoughts on backplane & trig data path • Use the backplane synchronously w/ 20MHz clock on SYSCLK, VMEH22501’s on all our data lines, clock rx w/ ADCMP600 or similar, clock tx needs some thought • Control board in slot 9 with 8 FEE boards to the left and 7 (or 8) to the right • Perhaps, have the fibers & trig/clk cable come from rear of control board? Through a little mezz board? Or if on front panel at least try to be vertically separated from the FEE board signal connectors, so the thing is not too painful to work on. • Trigger/reset commands will be encoded into the signal on SYSCLK, e.g. by pulse width or something like that. So SYSCLK is not directly for use as the data transfer clock, rather just a reference for local clock (DCM or whatever). Also of course SYSCLK is not the ~250MHz TDC clock but just a reference for it. The incoming reference clock from Belle-II timing distribution is divided down on the controller board to feed SYSCLK. • Trig datapath scheme: • All FEE boards have of course a global sense of time • So, let’s simply time slice the backplane for trigger data forwarding to the controller. (Maybe include all data in same path, let the controller decide what is trigger data and what is daq data.) No need for a “bus master” to control the backplane! • Don’t want trouble from a dead FEE board, so default is send nothing, have to configure each board with the time slice that it can use. FPGA config failure, or board not responding to config,  No data from this board but others ok. Need to check VMEH22501 details on power failure effect on backplane, this is a general question independent of protocol but not much options there if we don’t change the backplane. Can segment power/fusing on the FEE boards to try for maximum uptime on interface power. • With this scheme, timeslice widths need not be identical  system can be tuned to balance latencies even if the hit loading has some systematic variation across the FEE boards (which is likely I bet since crates serve “half-octants” and hit loading presumably varies with the layer number in detector…) • Saving a couple of lines for controls purposes, etc., let’s use a 64 bit main data bus: D(16), A(24), AM(6), IRQ(7), BBSY, BCLR, ACFAIL, DS(2), WRITE, IACK, BR(4). Or drop to 60 bits if seems better. • Remaining bussed lines: DTACK, AS (reserve for clock type thing); SYSFAIL, BERR, SYSRESET, LWORD (use for controls), SERA/B (termination status tbd on these backplanes) • So, total data output from crate ~150MB/s , neglecting any inefficiencys in the detailed protocol for time-slicing the backplane. Realistically, lets say 90% of that, 135 MB/s. • Obviously, have to decide bit width for trigger data out coding to know what hit rate we can support. But, for instance if we take 16ns resolution on the trigger output then 2bytes has a 1ms rollover. I would guess that suffices for unambiguous input to the trigger system •  cont.

9. Suppose we allocate 75% of the bandwidth for trigger data out (just a stab at a reasonable figure I think). • Thus ~50 MHz total hit rate could be supported from the crate for trigger data output. Which corresponds to ~32 kHz hit rate per channel (if all 96 channels used on 16 FEE boards, which is surely overestimating). • This does seem way in excess of any possible requirements for bKLM. So I’m feeling pretty comfortable about this now. • Probably we dial down the backplane transfer rate and/or bit width in the real design, to save money/power and increase reliability. • BOTTOM LINE: • 16 (or 15) FEE boards per crate • 96 channels per FEE board • 1 controller board per crate, probably in slot 9 • 1 DAQ fiber and 1 trigger fiber per crate (from controller board) • 1 trig/clk input (copper) per crate (to controller board) • 16 crates in full bKLM system • However… 

10. However: • Latency for trigger data output (neglecting any link protocol contributions): Figure 8 bus cycles per timeslice, latency is ≤ 15(boards)*8(cycles)/20MHz = 6μs. This would be unacceptable. Need concrete info on the latency requirement to bKLM crate output. • What is the latency through the trigger data link? • How much time is required in trigger system with bKLM data? (This may be an increasing function of the number of links it comes in on, of course. So reducing frontend latency by using more links, needs careful optimization.) • Obviously no matter how we do things, we can’t support very low latency with an arbitrary hit pattern. Can only guarantee low latency for sparse hit patterns. Can we (do we) simply drop hits from trigger output if we can’t meet some predefined latency for them? Or else how to handle this? • Of course, can try to improve by reducing the bit width, at least internal to the crate. Only need bit width so that rollover doesn’t occur withing the latency time.  Solve a nonlinear integer equation for optimal bit width (given the time resolution, 16ns or whatever). On the controller the timestamps can be expanded before sending on the link so this all is transparent to trigger system. • For example, 6μs/16ns requires 9 bits, so can send 7 hits per bus cycle, so maybe can reduce timeslice to 3 bus cycles and so latency to 2.25μs (and so bit width to 8, so 8 hits per cycle, …) • I am just now (before the meeting) noting that I did not include channel number encoding, this of course has to be added, more bits, more latency • To reduce latency, we can try to increase backplane bandwidth… Some possibilities: • point-to-point high speed serial backplane (VXS-type of thing) – expensive, development-intensive • put two VME-J1 backplanes in the crate and use then ~128 bit parallel bus • explore with labwork how fast we can really run the backplane w/ 22501’s (20MHz clock is probably very conservative) • use different chips (perhaps GTL of some sort) and modified terminations try to run the backplane even faster • in any case, we control all the board designs, do not need a general purpose backplane solution, we don’t have this constraint that a regular VME board has (that it must work together with any other VME compliant boards); our boards only have to work with our boards