EMBRACE station processing

1. EMBRACE station processing P. Picard Station de Radioastronomie de Nançay P. Renaud Station de Radioastronomie de Nançay C. Taffoureau Station de Radioastronomie de Nançay D. Benoist Station de Radioastronomie de Nançay V. Macaire SKADS funded L. Mercier SKADS funded W. Paule SKADS funded

2. RF beam A RF beam B Station processing inputs One RF beam signal is the phased sum of RF signals from 72 one pol. Vivaldi elements (analog RF beamforming) • Depending upon site configuration, for station processing one digitizer input is fed by a base element being: • one IF beam coming from one tile through an RF downconverter • one IF beam coming from a tileset of 4 combined tiles through an RF downconverter • The IF beams are in the 100 to 200 MHz frequency range after downconverting • Digitizer bandwidth: 100 MHz (200 Ms/s, 12 bits) sending a 2400 Mb/s data flow to each back-end digital input EMBRACE tile 72 two pol. Vivaldi elements 500 to 1500 MHz Analog RF beamforming for 2 beams (one pol.) Station processing input data rate for EMBRACE Westerbork site (144 tiles): 144 digital inputs for beam A (tiles) => 345.6 Gb/s 36 digital inputs for beam B (tilesets) => 86.4 Gb/s Nançay site (80 tiles): 20 digital inputs for beam A (tilesets) => 48 Gb/s 20 digital inputs for beam B (tilesets) => 48 Gb/s

3. EMBRACE digital beamforming Synthesis of a digital beam is done by a phased sum of all the digitized IF beams, each one being phase shifted by the proper value in order for the digital beam to point a sky direction. Using phase shift rather than true delays of IF beams signals is easier to implement, but works only in small bandwidth where a phase shift is equivalent to a true delay, requiring the use of bandpath filters before phase shifters Digitized IF beam A Antenna 1 Bandpath filter F, Δf Phase shifter Ф1 Digitized IF beam A Antenna 2 Digital beam F, Δf {AZ, El} Phase shifter Ф2 Bandpath filter F, Δf + Digitized IF beam A Antenna n Phase shifter Фn Bandpath filter F, Δf . complex data real data This generic system gives us a digital beam in a Δf subband. Duplicating this architecture with the proper bandpath filters and phase shifts allow access to many digital beams of Δf bandwidth (Δf = 195 KHz in EMBRACE)

4. Antenna Processor To reduce development duration, LOFAR station Back-End is used as EMBRACE hardware platform for station processing. Base processing element is the Antenna Processor (AP). One AP computes the phased sums of two antennas, same RF beam, for 248 data objects {[|AZ0, El0|,|AZ1,El1|], one subband} called beamlets (2 beams in one subband) Subband filter Subband select Beamformer Antenna n beam A Xin 248 x [Sre,Sim](Az0,El0,s) [Sre,Sim](Az1,El1,s) Xre,Yre Xre,Yre Xre,Xim Yre,Yim Xre,Xim Yre,Yim Xim,Yim Xim,Yim Antenna n+1 beam A Yin 512 subbands 2 antennas Up to 248 subbands All the required APs outputs are then summed to deliver the station digital beamlets One station beamlet is the phased sum of all station antennas, same RF beam, for two sky directions and one subband. A station delivers 248 beamlets for each RF beam. e.g. of digital beams configurations for RF beam A: 2 sky directions in 248 subbands (48.43 MHz if consecutive subbands) or …… or 8 sky directions in 62 subbands (12.11 MHz if consecutive subbands) or ….. or 496 sky directions in 1 subband (0.1953125 MHz)

5. Phased sum of antennas X and Y, subband s, towards sky direction {Az1,El1} Phase and gain shifts to apply to antennas X and Y, subband s, to point sky direction {Az1,El1} Complex samples of antennas X and Y signals in subband s Sum(X,Y,{Az1,El1})real Sum(X,Y,{Az1,El1})im Sum(X,Y,{Az2,El2})real Sum(X,Y,{Az2,El2})im g1cosφ1 -g1sinφ1g2cosφ2 -g2sinφ2 g1sinφ1 g1cosφ1g2sinφ2 g2cosφ2 g3cosφ3 -g3sinφ3 g4cosφ4 -g4sinφ4 g3sinφ3 g3cosφ3 g4sinφ4 g4cosφ4 Xreal Xim Yreal Yim = * s s s beamlet APs input parameters Subband filter: this polyphase filter delivers 512 subbands (~195 KHz wide) using a fixed set of 16 K coefficients to define filter shape. Subband select map: 248 subbands must be selected, with no constraints on these subbands (no required ascending nor descending order, multiple use of same subband…). All the APs must use the same subband select map at the same time. Beamforming weights: these weights are the digital phase shifts and amplitude shifts required for each antenna to synthezise a beam in a sky direction. For two antennas X, Y and one AP, beamforming weights to phase sum these two antennas in one subband and for two sky directions {Az1,El1} and {Az2,El2} are set in a 4 x 4 matrix and the beamforming process for one subband becomes a simple matrix multiply

6. X Form Beams + Form Beams Separate Subbands Select Subbands Antenna Processor datapath Subband width: 195.3125 Khz (input data sampled at 200 Ms/s) φ is one sky direction (Az,El) From previous AP in the chain From LCU (1s time frame) 248 x [PSre + jPSim]s,φ0 [PSre + jPSim]s,φ1 s in [0, 511] (18b complex) Subband Select map 248 X subbands 248 Y subbands Weights 248 X weights 248 Y weights two tiles or cells of combined tiles 512 X subbands 512 Y subbands 2 data flows X and Y (12b. real) 2 x 2400 Mb/s Station output 2 polyphase filter banks 16 K coef. 248 x [Sre + jSim]s,φ0 [Sre + jSim]s,φ1 s in [0, 511] (18b complex) 512 x [Xre + jXim]b [Yre + jYim]b b = 0 to 511 (18b complex) 248 x [Xre + jXim]s [Yre + jYim]s s in [0, 511] (18b complex) 248 x [FSre + jFSim]s,φ0 [FSre + jFSim]s,φ1 s in [0, 511] (16b complex) 1024 samples time frame => Data rate => 3.4875 109 b/s 7.2 109 b/s 3.4875 109 b/s 3.1 109 b/s Max[nb.s x nb.φ] = 496 Max[nb.s] = 248 Max[nb.s x nb.φ] = 496 Constraints => To next AP in the chain

7. Station processing board From previous board 4 x 2.5 to 3.125 Gb/s (Infiniband) 2 x 800 Mb/s control data 11 x 800 Mb/s X3 input 200 Ms/s 12b. Y3 input AP3 PHY 100 Mb/s raw Ethernet Monitoring and control RSP board 4b. MII X2 input 200 Ms/s 12b. Y2 input AP2 BP SERDES X1 input 200 Ms/s 12b. Y1 input AP1 8b. GMII X0 input 200 Ms/s 12b. Y0 input 1 Gb/s raw Ethernet Station data (partial) AP0 PHY CONF To next board 4 x 2.5 to 3.125 Gb/s (Infiniband) APs : FPGA using 90 nm process

8. Calculate Initial vector for Beam forming Calculate Tile array settings Compute Subbands Statistics Compute Cross Correlations Compute Beams Statistics Separate Subbands Select Subbands Form Beams Output Beams Station Control Unit Source coordinates Interferers coordinates Source coordinates Subband frequency Subband frequency Calculate Projection Matrices for nulling Array geometry Array geometry Monitoring and Control software Detect Interferer Nulling of Interferers LO1 beam A Local Control Unit LO1 beam B Tile array settings Calculate Calibration Correct for Calibration LO2 Store Store Store Output mode Time stamp Subbands to be processed External Correlator Interface N Tiles 2 RF beams Switch Antenna data From RCU Data recording Post processing 2N x 200 Ms/s 12b.

9. EMBRACE station data output Locally stored 1s averaged data, for each RF beam: • 1s averaged power of all 512 subbands for all antennas: subbands statistics • 1s averaged power of all 248 station beamlets: beamlet statistics • 1s averaged cross correlations of all antennas, for one subband High temporal resolution data (5.12 µs), for each RF beam: output of up to all 248 station beamlets on up to 4 x 1 Gb ethernet links Analog output for external analyser systems: • External Correlator Interface with two analog outputs • output bandwidth: up to 20 MHz • starting frequency: 0 to 40 MHz

10. F F+ 20.14 MHz Spectral domain of beam A (digital) 180 MHz 0 20 40 Analog output for external analyser External Correlator Interface: outputs two beams in 0-20 MHz analog signals 20 MHz bandwidth requires 103 consecutive subbands (20.14 MHz). Input of ECI: 103 beamlets of consecutive subbands and 2 sky positions (the same for each beamlet) Output of ECI: 2 x 1 data flow for the same 20 MHz bandwidth and 2 sky positions Digital processing Synthesis filter Digital to Analog converter + filter 0 Spectral domain of output beam A (digital) 20.14 MHz 0 Spectral domain of output beam A (analog) 20 MHz IF inputs of WSRT correlator (fringe stopping in IF processor)

11. Data output configurations Westerbork array station processing configurations 18 RSP boards for beam A and 5 RSP boards for beam B mode 0 Gb switch Half back-end RF beam A 144 inputs No analog output 1 No analog output 2 ECI tiles Data storage Data storage of up to 248 beamlets from RF beam A Data storage of up to 248 beamlets from RF beam B Half back-end RF beam B 36 inputs tilesets mode 0 mode 1 Gb switch Half back-end RF beam A 144 inputs Digital beam A(Az0,El0) in one 20 MHz bandwidth, to analog output 1 Digital beam A(Az1,El1) in the same 20 MHz bandwidth, to analog output 2 ECI tiles Data storage of up to 124 beamlets from RF beam Aand 248 beamlets from RF beam B Data storage Half back-end RF beam B 36 inputs tilesets Note: Same configuration available with RF beam B to ECI and RF beam A to data storage mode 0

12. Data recording, access to post processing P. Renaud Station de Radioastronomie de Nançay J. Borsenberger Observatoire de Paris F. Viallefond Observatoire de Paris Henrik Olofsson Observatoire de Paris S. Torchinsky Station de Radioastronomie de Nançay S. Pomarede SKADS funded Real time data recording Ongoing work on specific computer hardware configuration using COTS components to allow real time recording of at least 124 beamlets (2 x 1 Gb ethernet link) and recording software Storage capacity to record N beamlets ( GBytes or Tbytes) beamlets data flow 10 min 1 hour 5 hours 10 hours 62 93 MB/s 54.2325.51.63.2 124 185 MB/s 108.56513.26.4 186 278 MB/s 162.8976.44.8 9.6 248 370 MB/s 2171.36.4 12.8

13. Data tools Tools for acquisition software and post processing: Work on an Embrace data model in order to generate API in Python / C++ to deliver procedures to be used in test and observation software (F. Viallefond) Work to define an Embrace Measurement Set to be used in data acquisition and post processing tools.