Simple ideas on how to integrate L2CAL and L2XFT ---> food for thoughts

Simple ideas on how to integrate L2CAL and L2XFT---> food for thoughts Ted May 25th, 2007

Outline • Comment on the final L2 upgrade commission efforts • Some basics on how FILAR/S32PCI64 works • The problem with L2CAL and L2XFT integration due to large data package size • Ideas on possible quick solutions

Comment on the final commissioning efforts • We have a wonderful team: the L2 United team • A strong/powerful team in the past two weeks, even with two key people (the SPLs, Vadim&Gene) on vacation • It has been a real pleasure working with them in the past two weeks: people who spent most of their time on final commission in the past ~ two weeks

The L2 tough ladies

S32PCI64: single channel FILAR

The four-channel FILAR

S32PCI64: With transmitter firmware: --> Solar With Receiver firmware: --> single channel FILAR Four-channel FILAR (Rx) http://hsi.web.cern.ch/HSI/s-link/devices/FILAR

L2 Decision Block: Conceptual view CAL1 CAL2 CAL3 CAL4 Reces SVT Muon/XTRP/L1 Old cluster XFT1 XFT2 XFT3 Solar Filar Filar Filar New L2 decision node To L2TS Pulsar L2D + (TL2D if L2A) Control Node process Spare node

Inside the Box decision node Spare node

So what is the the problem then? Both new data paths have large data volume: L2XFT (grow with luminosity, could be very large) + L2CAL (fixed length) data words CAL1 == 104 CAL2 == 104 CAL3 == 156 CAL4 == 104 Reces SVT Muon/XTRP/L1 Old cluster XFT1 > 120 XFT2 > 120 XFT3 > 120 To L2TS Pulsar L2D + (TL2D if L2A) data packages fighting over 2 PCI buses, when traffic jam occurs, data wait in the input FIFO Input FIFO could overflow IF data packages too large ---> missing data words --> CPU detects this --> L2DTO Control Node process Spare node

The actual setup with beam • had problem when both L2CAL+L2XFT are included • show up as XFT data package corruption --> L2 DTO • no problem when only L2CAL is included PCI bus 1 PCI bus 2 CAL1 CAL2 RECES CAL3 SVT Muon/XTRP/L1 XFT3 XFT1 XFT2 Old Cluster CAL4

with L2CAL only, i.e. without XFT: --> do not expect FIFO overflow problem PCI bus 1: on this solar bus, the worst case is likely when L2A with TL2D bank and scaler info sending out from solar… up to ~ 1K words SLINK package… During the time this package is transferring over PCI bus 1, the largest CAL3 data has to wait in the input FIFO. But the maximal number of input events is 3 (since 1 is already on its way L2Aed). 3 x 156 = 468 < 512 FIFO depth… impossible to fill up the FIFO. PCI bus 2: worst case is 4 L1A in a row… The largest data package is CAL4 == 104 CAL4: 104 x 4 = 416 < 512 No problem CAL1 == 104 CAL2 == 104 RECES CAL3 == 156 SVT Muon/XTRP/L1 XFT3 XFT1 XFT2 Old Cluster CAL4 == 104

Can we solve this problem? • The Pulsar-based system is very flexible • zero-suppression, fast abort, enable back-pressure, use dedicated PC for dedicated buffer event … etc • Many ways to deal with this, but I will ONLY talk about the simplest possible solutions…(with the potential to be done within weeks, not months). • Will only show two simple ideas today • (1) let large XFT data have higher priority over PCI bus • (2) use another PC for the XFT data, to do fast-abort or data suppression, ROI (Region-Of-Interest)…

How to let XFT data packages have higher priority over the PCI bus? • Current way of PCI handling for each data channel: • Send requests for up to 4 (effectively) buffer events at begin of run • This means that not only the current event package, but also packages from next event(s) could compete over the PCI bus if they arrive early: in other words, whoever arrives early gets the bus… • This made sense in the past kuz algorithm time is almost zero, and all data path had small size How to let largest data path have highest priority? • This can be done by simply playing tricks with the request FIFO: • Only send requests for up to 4 buffer events for largest data paths at the beginning of run. Do not send request for all other (smaller) data paths • For each event, only after the largest packages arrived to PC memory, send request for the rest of data paths, but only for the current event. Before that, all other smaller data packages will have to wait in their input FIFO: not allowed to fight over the PCI bus. This should reduce the possibility of FIFO overflow, but may not fix the problem in a fundamental way (as luminosity increases), and likely delay L2CAL path. But it is the simplest thing to do…this will be done anytime soon…

There is simple way to possibly solve the problem: use the spare PC to receive XFT data and do “zero-suppression” in software … then send to main PC … This sounds more complicated than it actually is … data words CAL1 == 104 CAL2 == 104 CAL3 == 156 CAL4 == 104 Reces SVT Muon/XTRP/L1 Old cluster XFT1 > 120 XFT2 > 120 XFT3 > 120 To L2TS Pulsar L2D + (TL2D if L2A) L2 decision node Solar Filar Filar (Single channel)Filar Control Node Process controls both PCs • Simple implementation: • Send empty package if no need for Stereo tracking • Send aw xft data when needed • (based on L1 bits) • Could also do ROI • More involved implementation: • Do 3D tracking right here • for tracks about certain Pt …. your wild ideas here ,,,

Spare PC could use 2 single channel Filars (2048 input FIFO depth) to receive the largest XFT packages… The decision node could also use single channel Filar to receive the xft package. data words CAL1 == 104 CAL2 == 104 CAL3 == 156 CAL4 == 104 Reces SVT Muon/XTRP/L1 Old cluster XFT1 > 40 XFT2 > 160 XFT3 > 160 Load balancing To L2TS Pulsar L2D + (TL2D if L2A) L2 decision node Solar Filar Filar (Single channel)Filar Control Node Process control both PCs Solar Filar (single channel)Filar (single channel)Filar The spare PC

This all looks more complicated than it is--- so what is really involved then? • Step 1: how to ask the existing control node process to also control the spare PC? • Daniel’s design of the control node can naturally handle multiple PCs (a L3 based design) • In Daniel’s words: “just use the CardEditor to change number of PCs from 1 to 2, then input the spare PC IP address”. • Step 2: what do we have to change for the existing code for L2 decision node? • Nothing, if one simply use the spare PC to do fast abort based on L1 bits • It doesn’t know whether the XFT package is from a PC or a Pulsar merger • Step 3: move XFT fibers from the decision node to the spare node, as well as the copy of muon/xtrp/L1 fiber. • Step 4: could use single channel Filars if so desired (software is the same) • Step 5: copy over the decision node software to the spare PC, remove most of the algorithm stuff, do fast-abort here. If the event doesn’t need stereo tracking, send empty package. Otherwise, send the combined raw data. • Later on, could add more functions for zero-suppression, such as ROI. • Or, if you really want, simply do stereo tracking right here in spare PC…e.g. for muons … • Your wild idea here… • We may want to order 2 more PCs soon, as spares…

System commissioning is fun … • I am leaving tomorrow, for two weeks • This is just one of the ideas I have had • Didn’t want to talk about it, but young people kept asking • Meant as food for thoughts, to encourage young people to brainstorm Our system is more flexible than we think About commissioning: To have a system more or less working is not hard, To have zero error is hard, To keep it that way is even harder… Whatever you do, think about how to win the peace afterwards…

Simple ideas on how to integrate L2CAL and L2XFT ---> food for thoughts