1 / 21

ROD DAQ Workshop February 6-7, 2003

ROD DAQ Workshop February 6-7, 2003. DSP Software Update. Isr and Data Handling. MDSP initialization, Rev B & Rev C RODs. Histogramming code. Douglas Ferguson UW-Madison & LBL. ISR and Data Handling. Event Based Data:. Event based data allows the DMA ISR to complete

matana
Download Presentation

ROD DAQ Workshop February 6-7, 2003

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ROD DAQ Workshop February 6-7, 2003 DSP Software Update • Isr and Data Handling • MDSP initialization, Rev B & Rev C RODs • Histogramming code Douglas Ferguson UW-Madison & LBL

  2. ISR and Data Handling Event Based Data: Event based data allows the DMA ISR to complete its work quickly, since the event’s length is always found at the end of a frame. B0F00000 EE1234EE 00000009 ………… ………… ………….. ………… ………… ………… 21008E91 9CF09EB0 ……….. ………… ………….. ………… ………… ………… ………….. ………… ………… ………… ………….. ………… ………… ………… ………….. ………… ………… Previously the router inserted a marker word (which cannot appear in normal data) of 0x40ff llll, where llll is the event length in words, at the end of the frame, if it encountered an E0F in that frame. If the event ends on the frame boundary, the marker word is inserted at the end of the next (empty) frame. The ISR was responsible for checking the event for errors by calculating the position of the error word in the event trailer. The ISR also had to determine which router trap had sent on the event by looking at a word in the event header. The new router code dispenses with those checks by using a new marker word of 0x40xy llll. llll is as before and x = 0 or 1 indicates the router trap. If the event has no errors, y = 0xF; if there are errors y = 0xE. ………… ………….. ………… ………… ………… ………….. ………… ………… ………… ………….. E0F0000 ………… ………… ………….. ………… ………… ………… ………….. ………… ………… ………… ………….. ………… ………… ………… ………….. ………… 40xy0127 Word 27  Inserted by router  1 frame= 0x100 words

  3. ISR and Data Handling (cont) ISR code has been streamlined, resulting in modest speed improvements for normal events & much improved handling of cases where it was necessary to place data in slow memory: • Previously for each incoming event, the ISR filled in a set of frame registers (in • addition to the event data which goes onto the event queue). These permitted events • to be processed by tasks & removed from the queue immediately (for multi-task • situations). • With the availability of 4 slave DSPs, this is less likely, and in addition the queue of • events is likely to be short under most circumstances, so the queue has been reduced • to a simple FIFO. The space in IDATA (the section of fast memory reserved for • significant program variables) which was reserved for the frame registers will now • be filled with the event queue data (up to 256 events max), which has been compacted • into two words per event: Data word 1: 0x II LL XX ZZ, I= IDRAM starting frame, L= # of IDRAM frames, X=SDRAM starting frame, Z= # of SDRAM frames. Data word 2: 0xEE TT LLLL , E= error bits: 31= overflow, 30= any error, 29= header err, 28= trailer err, & 27= data err, T= the trap function mask, from event trap setup. L= event length in words,

  4. ISR and Data Handling (cont) • Previously if the fast memory became full, the ISR would dump all frames in • fast memory into slow memory, & fix any pointers that need updating as a result. • This was an attempt to minimize time spent re-organizing data when receiving • large events (the only situation in which an overflow is likely), but resulted in • accounting headaches. Additionally, in the unlikely case that an overflow occurred • while a set of smaller events were still being processed, there was a performance • penalty. • The new scheme copies an incoming event frame by frame into slow memory once • the fast memory fills. Once an event has begun dumping frames into slow memory, • it will continue doing so even if fast memory frames become available meanwhile. • (Succeeding events, however, can take advantage of the newly available memory).

  5. MDSP Initialization & ROD boards • The new Revision C boards are significantly better at event handling than the • Rev. B boards. On Rev. B boards, the slave DSPs’ clocks cannot be phase-locked • due to a fabrication error. This resulted in occasional errors while transferring the • data from the router to the DSP. Slave DSP 1 is unusable, and DSP 3 is unreliable. • On the new boards, slave DSP clocks are in a phase-lock loop with the router. This • solves the data transfer problem, with one caveat: The router FPGA has four global clocks; one is reserved for the router itself, and three others are distributed to slave DSPs 0, 1 and 2.  DSP 3 shares its clock with DSP 0. The clock can either be in phase or 180o out of phase. If it is out of phase, router errors are still possible. 

  6. MDSP Initialization & ROD boards (cont) • To correct for this, code has been inserted inside the master DSP’s initialization. • The MDSP checks registers in the router indicating the condition of the clock on • slave DSP 3 (these are located at 0x0040240C, 4C, 8C and CC for DSP 0 – 3 • respectively; if bit 12 is set, the clock is in phase). If the clock is out of phase then the MDSP attempts to correct this by sending a signal to the rod controller FPGA that it should reset the slave’s clock. Each time this is done there is a chance that the slave DSP clock will be put in phase; the MDSP will attempt up to 10 times to correct the clock. If it is still out of phase by the 10th try (normally the clock is corrected by within 1-3 attempts), the MDSP will give up and report an error. In this case the DAQ should correct the clock manually by sending the new resetDsp primitive to the MDSP. Additionally, if at any point slave DSP #3 is reset via VME (without a board reset), this primitive should be run (documentation forthcoming). As a side benefit, the MDSP also configures the slave DSPs’ EMIF registers upon startup, so that all areas of memory may be accessed freely via VME; it is no longer necessary to load them manually.

  7. Histogram Control • Is now designed around the concept of “module groups”, which are a group of modules • that act in a similar fashion during a scan, and receive the same trigger. (A module’s group • is determined by the data in the corresponding structure in the structure set). Four module • groups are definable now (0-3); this will be increased to 8 in the next version. Configuration Parameters: slvBits: A bitfield indicating which slaves we are setting up. Bit 0 slave 0, etc. configSctSet: Indicates which structure set is being used. INIT_MODULE_CONFIG, WORK_MODULE_CONFIGandSPARE_MODULE_CONFIG (primParams.h) dataSet: Indicates which type of data will be sent out during configuration (MODULE_BASIC, MODULE_TRIM and MODULE_ALL) Will be made automatic in the next version. (primParams.h) useRangeList: A debugging tool, will be removed soon. Set to FALSE for now rangeList[2]: A debugging tool, will be removed soon. Ignored if useRangeList = FALSE.

  8. Histogram Control (cont) cmdBuff: Indicates which command buffers (== serial ports) to send configuration data from. Can be either 0, 1 or both (= 2). scanRegister[2]: The type of variable which will be scanned across during this histogramming run, eg. ST_QTHR. Two range lists are definable (discussed below), the two variables correspond to the different lists. *dataPtr[2]: Pointers to the two range lists. These must either be pre-loaded into memory (and the pointer supplied here), OR the pointers set to the DEFAULT value (= 0xFFFFFFFF, from utilities.h), and the two lists included in the primitive list (inside the start task primitive, appended immediately after the TASK_STRUCTURES_INunion. The range lists give the floating point values of the scanRegister at each bin; a module group may use either of the two. groupRangeMap[2]: Bitfields indicating which module groups use which range list. groupSPMap[2]: Bitfields indicating which module groups use which serial port. groupDSPMap[4]: Bitfields indicating which module groups use which that SDSP.

  9. Histogram Control (cont) Repetitions: The total number of events to send in a given bin. nBins: The number of bins in this scan. bin0: The bin to begin filling in upon start; for use in appending scans. globalCtrl: Flag indicating whether this scan is under local (MDSP) or remote (SBC) control. Remote control will be implemented using control signals from the SBC to the MDSP which force it to step in sync with all other RODs in the crate. syncLevel: Used with the global control flag; indicates the needed amount of ROD synchronization. To be defined.

  10. Histogram Control (cont) Trigger Parameters: calLineLoop: If set to TRUE, the master DSP will loop over the calibration lines before continuing on to the next bin. distributionToggle: If set to MODULE_GROUP, the master DSP will set the command line and data link masks appropriately for each module group. Groups are looped over for each event. If set to ROUTER_DISTRIB, all groups receive their triggers simultaneously. This will be removed in the next version, and the distribution will be set using the groupDSPMap varaibles. triggerSequence[2]: Two 6 function command lists; list 0 is for serial port 0, list 1 is for port 1. • Command lists are structures defined in primParams.h: • #define N_CMD_LIST_CMDS 6 • struct CmdList { • UINT16 cmd[N_CMD_LIST_CMDS]; • UINT32 data[N_CMD_LIST_CMDS]; • }; Each cmd can be any of the commands listed in ABCDchip.h (L1 trigger, calibration pulse, delay, etc.) If a command takes data (eg. delay), it is put In the corresponding data field.

  11. Histogram Control (cont) incCmd[2], incData[2]: If incData & incCmd are non-zero, the command given by incCmd inside the corresponding trigger sequence list is incremented by incData between bins. For cross-talk studies.

  12. Histogramming Registers Slave DSPs Master DSP Histogram CMD/STAT 0 (really status) Histogram CMD/STAT 0: Error count Cal-line current bin. C N Cal-line current bin. En En 16 1 7 bits 8 bits 1 1 1 7 bits 8 bits En= cal line enabled. C= new cal-line, N= New bin, En= cal line enabled. Histogram STAT 0/1: events received Histogram CMD/STAT 1: tot # of events 0: Histogram STAT 0 DSP 3 DSP 2 16 16 event processing time (micro-sec) 1: DSP 1 DSP 0 errors in bin D P E R 16 16 R: Ready E: Expecting Bit 8 Histogram CMD/STAT 1 (really status): Event statistics (to be defined). P: Processing D: Done Histogram STAT 1: # of events received

  13. Histogramming Task States Histogram Task Histogram Control Task INIT INIT • Task variable setup. • Task variable setup. • Check HRDY in slaves. • HSTAT 0 RDY HI Slave Polling (all states) • Copy slave HSTAT 0,1 to local • variables. Update own HSTAT 0. READY NEWBIN • Zero out events received variable. • Zero HSTAT 0,1. Place current • bin & set in HCMD. • Wait for #repetitions in • HCMD != 0. Then: • Zero out current repetitions • Set bin, histogram set, # rep. • this bin; copy bin & set to • own HSTAT 0. • Place current bin set, # rep in • all slave HCMD. • Create & send configuration • command stream. Prepare trigger • command stream. • Clear HSTAT 0 events received. • HSTAT 0 DONE LO, EXP HI. WAITEXP • Wait for all slave HSTAT EXP HI.

  14. Histogramming Task States (cont) Histogram Task Histogram Control Task PULSING EXPECTING • Wait for the # events received • by all slaves == current rep per • slave. (= curr. Rep./# slaves). • Check for timeout. Then: • Wait for events to arrive on • queue. Then: • HSTAT 0 PROC HI. • Send out one trigger per slave, • update current rep., HCMD. # rep • Histogram the event • HSTAT 0 PROC LO. • If Current repetitions < # rep. • per bin, repeat. • Increment events received, • HSTAT 0 events received. • HSTAT 0 PROC LO. WAITING • If events received < #rep. • this bin, repeat. Else: • Wait for all slave HSTAT 0 • DONE HI • HSTAT 0 EXP LO, DONE HI PREP • Clear HCMD #repetitions. • Increment # bins done. • state  READY • If bins done == # points: • state  DONE. Else: • Increment the data variables of • the configuration & trigger lists • state  NEWBIN

  15. Event Trap Setup • Will be internally generated in next version of the histogram-control task. • Event trap setup is an important primitive central to the event trapping; it • is used to coordinate the actions of the slave DSPs and Router. • Event trap setup should be sent from the host to the master DSP; the master • will then set the Router’s registers, and pass a copy of itself onto the slave • using the inter-DSP primitive list. • The router has a set of registers for each slave DSP, and a global register set. • Each DSP register set has two (mostly) independent traps. Trap 0 contains some • Extra flags which are common to both traps: Command Register 0 Command Register 1 Trap reset/load/flush Status Register FIFO word count Internal Delay Count Trap Match 0 Modulus/Remainder 0 Trap Match 1 Modulus/Remainder 1 Global Command/Status Register • The Trap reset/load/flush & internal delay count are set by the DSP and generally • should not be touched. The status & FIFO word count are read-only.

  16. Event Trap Setup (cont) • Input Parameters: slvBits: A bitfield indicating which slaves we are setting up. Bit 0 slave 0, etc. # of events: The total # of events to trap. Currently should be set to COLLECT_FOREVER (0). timeout: Timeout for the reply, when the MDSP sends the primitive to a slave. 0x2000 extRouterSetup: Flag indicating that the router is set up independently; if set the master DSP will not write to the router’s registers. Use with care! FALSE distribute: If set, the master DSP will distribute check that the input modulus (described later) is correct, and will distribute remainders among the input slaves. NO permitBackPressure: Bit in the global CSR. Should only be applied in calibration mode (!!) Allows the router to apply back pressure to the EFB if large events are coming through and the DSP needs time to move data into SDRAM. (The slave DSP sets the interrupt mask from DSP inside the SR, if so). TRUE format: Bit inside CR0 which controls the format in which events are sent to the DSP. This is common to trap 0 & 1. High= error format, Low= normal or “S-Link” format. NORMAL

  17. Event Trap Setup (cont) • Input Parameters (cont): dataMode: Bit inside CR0 which controls the whether the Router (for that DSP) outputs data in event mode (skipping to the end of a frame with a trailer, and writing 0x40ff +length, or in data mode, in which it waits until it has enough data to fill a frame. Common to both traps. FALSE sLink: Bit in SR0; if set the trap match is overridden and only the modulus & remainder of trap 0 are used to distribute events among DSPs. Common to both traps. FALSE • Trap Input Parameters: config: The type of events to trap. If IDLE, trap is unused; otherwise ATLAS, ROD or TIM. If sLink is set, Trap 1 should be IDLE, and trap 0 set to SLINK. ROD exclusionFlag: If set will trap all events *except* those in the match register. NOTE: for now this is disabled. match: The type of event for the router to match. Up to 4 copies of ETS, one for each enabled slave DSP. Only difference: DSP 0: match = 0 … DSP 3: match = 3

  18. Event Trap Setup (cont) • Trap Input Parameters (cont): function: Determines what tasks will process the events coming from the trap. Functions are additive: HISTO= 1, TRAP= 2, OCCUPANCY= 4, etc. Used only by the slave DSPs. modulus/remainder: Determines which matched events are trapped. Events are trapped if mod(trap match counter, modulus)= remainder. modulus = 1, remainder = 0. • DSP Input Parameters: releaseFrames: If set, the last task to operate on an event will release its frames as it processes them, freeing up space. FALSE trapStray/iterLimit: Stray events are events which shouldn’t be there, but were trapped anyway, either through an electronics glitch or operator error. They gum up the works if not disposed of. If trapStray is set, they are put into a special “observation buffer” (after the 7th SDRAM frame buffer). iterLimit determines how long they should be kept around before being removed/sent for observation. Suggestion: trap strays, and set iterLimit to a low number, say 2 or so. Don’t care.

  19. Histogram Setup • Will be internally generated in next version of the histogram-control task. Parameters: *base: A pointer to the base of the histograms in memory. Normally set to the default given in memoryPartitions.h: HISTOGRAM_DEFAULT_BASE rodType: The type of ROD this is: SCT_ROD(primParams.h) nBins: The total number of bins. padding[]: Adds delimiters to mark the histogram bins, if the arrangement is in blocks. calLine: Not used anymore; controlled solely by MDSP. Will be removed in next vers. arrangement: HISTOGRAM_BLOCKorHISTOGRAM_SLICE. (primParams.h) dataFormat: The format the data will arrive in. The formatters can be set to send the full hit pattern which arrives from the modules (3 hits), or send only the central hit. HISTOGRAM_CONDENSEDorHISTOGRAM_FULL. FULL will place hit data into three separate data sets, located sequentially after one another in memory. (primParams.h)

  20. Histogram Setup (cont) binSize: The size of the bins: HISTOGRAM_16BIT or HISTOGRAM_32BIT (primParams.h) validModules[2]: Bitfield indicating which modules are contributing to this slave’s events. Word 0 is for modules 0-31, while the 16 LSB of word 1 are for modules 32-47. Histogram space will be reserved only for modules flagged as valid modules. All other modules’ data if any arrives (by an error) is put into a single module/bin “invalid data” section which is appended to the end of the histograms. moduleRangeMap[2]: Bitfields indicating which modules use which range list. dataType[2]: Indicates the SCT variables which correspond to the two range lists. *xPtr[2]: Pointers to the two range lists. These must either be pre-loaded into memory (and the pointer supplied here), OR the pointers set to the DEFAULT value (= 0xFFFFFFFF, from utilities.h), and the two lists included in the primitive list (inside the histogram setup primitive, appended immediately after the HISTOGRAM_SETUP_INstructure.

  21. Start Event Trapping • Will be internally generated in next version of the histogram-control task. • Event Trap Setup must be run beforehand for this slave DSP. • Start event trapping is a simple primitive with no parameters. It is used to tell • the slave DSP to trigger the router’s enable trapping flag. This register is • connected to the SDSP’s serial port (both ports are configured for general • purpose I/O). The state of this register is visible in the router’s status register • for that slave DSP. Start Histogram Task • Will be internally generated in next version of the histogram-control task. • Histogram Setup must be run beforehand for this slave DSP. • Start Task is used to set the histogramming task running on the slave DSPs; • this must be done before the histogram control task is started on the MDSP. • The histogram task has two parameters: the total # of events to analyze (set to • COLLECT_FOREVER (0), as in Event Trap Setup), and a task control flag (set to • MASTER_HREG (0) ).

More Related