Jonathan Hargreaves, JIVE

1. JIVE UniBoard Correlator External Review 3 September 2014 Jonathan Hargreaves, JIVE JIVE UniBoard Correlator External Review

2. Introduction The JUC correlator firmware • Architecture • Signal flow • Future development JIVE UniBoard Correlator External Review

3. Specifications JIVE UniBoard Correlator External Review

4. Overall Signal Flow JIVE UniBoard Correlator External Review

5. Control 1 • At the start of every scan the control computer must set • An application reset to all nodes • The start second • The bandwidth, frame length, integration time • Which input streams are lower sidebands • In the BNs, the products to export • Incoming data are stored in a 4 second circular buffer • The data stream (station, polarization & band) is determined by the UDP port number and IP address • The time slot is determined by the VDIF time code and frame number JIVE UniBoard Correlator External Review

6. Control 2 • Data processing begins after 2 seconds of data have been placed in the buffer • The UniBoard does not check if all the data has arrived … it waits for the control computer to tell it • Every ‘second’ the control computer tells the UniBoard how many integrations to process • The UniBoard indicates via a status bit when it has finished processing • The control computer interacts with FN0. FN0 tells the other FN1,2 & 3 when to start and stop processing data • The UniBoard does not know if it is processing disk packs or eVLBI • Gaps in the data are handled by the validity bits JIVE UniBoard Correlator External Review

7. Diagnostics • Several diagnostic registers are read and logged by the control computer during a scan • Number of frames received per input stream • Number of unmatched frames received • Number of delay model packets received • Number of packets received and transmitted by 10GbE ports • Counts of the FFT frames generated by each FN and received by each BN • A sample statistics module has been written but not yet integrated into the FN firmware JIVE UniBoard Correlator External Review

8. Architecture • 32 stations x 2 pol • All products are calculated but only the necessary ones are exported • Stations are virtual: the control software can make a 16 or 8 station correlator with 2x or 4x the bandwidth • Native mode is 4 bands of 16MHz • Wider bands (32MHz, 64MHz) be supported with new firmware • Narrower bands (8MHz, 4MHz …) can be processed at a multiple of real time • Each band is channelized to 1024 frequency bins • Other channelizations could be supported with new firmware • The polyphase filter window function can be changed at run time • Maximum integration time is 1s • Limited by size of DDR3 module used for corner turning JIVE UniBoard Correlator External Review

9. Front Node Implementation Hierarchical Design DDR Pre Filter Structure Packet Receiver ETHSwitch 10Gb-Eth Mixer FFT Normalize Framer FBI Mesh DDR Delay Model Control Computer 1Gb-Eth 10Gb-Eth SOPC Packet receive FBI Framer Filter Bank Normalize 1Gb-Eth Delay Model SOPC Physical Design Floor Plan

10. PFB Architecture JIVE UniBoard Correlator External Review

11. PFB Window Function JIVE UniBoard Correlator External Review

12. 59 59 Integer sample delay Packet decoder d0 Fifo Delay Model Evaluation 51 UDP Offload 32 bits d0 1GbE port Fractional sample delay 28 27 A new set of coefficients is loaded at the start of every integration Evaluates Delay = d0 + d1t once per FFT Offset between delay and rate increase dynamic range Coefficients are currently 32 bits, will be 48 bits The second derivative coefficient is stored but not currently used 24 31 d1 Fifo d1 20 Output taps + 0 0 Delay rate register Delay accumulator JIVE UniBoard Correlator External Review

13. Delay Model Application • The integer (whole sample) part of the delay determines the address of the first sample in the integration • The next 8 bits are used to calculate a phase correction to the nearest 1/256th of a sample • The correction is applied to the frequency bins as they exit the FFT • If the delay correction rolls over a whole sample during an integration, one sample is skipped or repeated and a 90 degree correction is applied to the phase model JIVE UniBoard Correlator External Review

14. Phase correction + Evaluates Phase = p0 +Σ(p1 + p2 +2Σp2) once per sample Phase is applied in a quadrature mixer at the input to the polyphase filter Coefficients currently 48 bits, will be 64 bits P1 register + + Phase to mixer x2 + + P2 register P0 register JIVE UniBoard Correlator External Review

15. Back Node Implementation Hierarchical Design DDR_I CorrelatorEngine ETHSwitch Formatter 10Gb-Eth Corner Turner Mesh FBI Deframer Validity Accu DDR_II SOPC 1Gb-Eth Control Computer ValidityAccumulator DDR_I FBI Correlator Engine (Containing 132 MAC cells) Corner Turner Deframer 1Gb-Eth SOPC DDR_II Formatter 10Gb-Eth Floor Plan Physical Design

16. Correlator Multiply Accumulate Cell JIVE UniBoard Correlator External Review

17. Validity • 1 bit per VDIF frame stored in FN • A whole FFT is invalid if any contributing data are invalid • First six FFTs in an integration are invalid because pre-filter structure is filling up • Invalid FFTs are substituted by zeros so do not contribute to the products • One validity bit per FFT is carried across to BN and corner-turned with the data • Thirty-two bit validity accumulators calculate normalization factors for every product. JIVE UniBoard Correlator External Review

18. Current Developments • Widen delay/phase coefficients from 32/48 to 48/64 bits • Firmware written and under simulation/test • Two bands of 32MHz bandwidth • Under test • Sample Statistics • Firmware written, needs integration and verification • VDIF Frame length • So far only 5000 byte frames have been tested • Mixed frame lengths can be supported with further work JIVE UniBoard Correlator External Review

19. Future Developments • Support 1, 4 and 8 bit sampled data • One bit can be supported by converting to 2 bit at the input • 4 and 8 bit to be added when needed • VDIF epoch • One band of 64MHz bandwidth, other modes? • Stagger output data to alleviate bursting • Lower spectral resolution to reduce volume of output data • 1024 -> 64 points • UniBoard2 – 2x to 8x the bandwidth per board JIVE UniBoard Correlator External Review