170 likes | 469 Views
Next Generation ASKAP Hardware. Andrew Brown Lead Digital Hardware Engineer for ASKAP 8 th October 2011. What is ASKAP?. Australian Square Kilometre Array Pathfinder Located in Boolardy in Western Australia Site is currently under construction
E N D
Next Generation ASKAP Hardware Andrew Brown Lead Digital Hardware Engineer for ASKAP 8th October 2011
What is ASKAP? • Australian Square Kilometre Array Pathfinder • Located in Boolardy in Western Australia • Site is currently under construction • First stage of ASKAP is scheduled to be installed at the end of the year with commissioning starting next year • System Requirements • 36 12m dishes, maximum baseline of 6km • 188 element Phased Array Feed (PAF) on each antenna • Frequency range from 700 to 1800MHz • Beamformer with 30deg2 field of view • Instantaneous bandwidth of at least 300MHz • 16K channels of approximately 18KHz resolution • At least 30 dual polarisation beams with full stokes correlations • Support for finer zoom modes • Tied array outputs for VLBI/Pulsar processing • 5 second integration time • Custom digital and analogue hardware platform with commercial off the shelf antennas CSIRO. Next Generation ASKAP Hardware
ASKAP Design Enhancement (ADE) • ASKAP Hardware is good but can be improved • ADE is a chance to modify ASKAP design to address lessons learnt • Must have similar or better performance than current ASKAP hardware • Cheaper than current ASKAP design • Redesign of analogue hardware • Use RF over fibre rather than coax cable • Move to direct sampling away from dual heterodyne receiver allows all digital hardware to be placed at the central site • Redesign using latest FPGAs • Cheaper and more powerful than previous generations • Lower power consumption • Higher density FPGAs allows a decrease in the total number of FPGAs in the system CSIRO. Next Generation ASKAP Hardware
ADE Hardware • Next generation from current Redback-2 and Dragonfly-2 boards • Two new digital hardware platforms to be developed • Dragonfly-3 • Digitiser platform with 16 x 1.6GSPS ADC • Optical data output (optional optical input) • FPGAs for 1st stage Filterbank • Redback-3 • Signal processing platform (beamformer & correlator) • Optical input and output • Copper & Optical Ethernet output for data products • Each hardware platform consists of • 19 inch 1U custom PC server chassis including fan cooling • Optical input and output connectors as LC or MPO connectors • 300W ATX PC power supply • 1 Bullant card for data interfacing and control/monitoring • 1 signal processing card (Redback-3 or Dragonfly-3) CSIRO. Next Generation ASKAP Hardware
ADE System Flow PAF mechanical structure,PAF checker board, LNA, 700-1800 filter, Amplification, Tri-band filter, Variable attenuator, RFOF transmitter Fibre to central site CPU Beamforming System (6 1/3 per antenna) RFOF Rx, ADC andCoarse Filterbank (12 per antenna) Correlator CSIRO. Next Generation ASKAP Hardware
Dragonfly-3 Hardware • Four Kintex 7 FPGA (XC7K325T-2FFG900) • 50.9K slices, 16Mbit 840 DSP each FPGA • Dynamic configuration through control interface to Bullant • Independent over-temperature shutdown • 3 duplex 10Gbps serial connections between FPGAs • Larger FPGA (410T) can be used on same footprint to increase resources • Eight dual input ADC (ADC12D1600) • 12 Bit ADC with 8.6 to 9.4 ENOBs @ 1600MSPS • Each FPGA connected to 2 ADCs • Sampling clock distribution done off board • Two Avago 12x10.3125Gbps optical MiniPOD transmitter • Sufficient bandwidth to output all 768MHz from the digitisers at 9bit resolution • Two Avago 12x10.3125Gbps optical MiniPOD receiver • Not installed for ASKAP, could be used for other applications i.e. CASPER • Onboard fan control for FPGA cooling • Extensive monitor network • Each power supply has voltage/current/temperature monitoring • EEPROM for serial number and board type identification CSIRO. Next Generation ASKAP Hardware
Dragonfly-3 Data Flow 76.8Gbps 76.8Gbps 2 Dual ADC (4 x 1.6GSPS 2 Dual ADC (4 x 1.6GSPS Kintex-7 XC7K325T Kintex-7 XC7K325T ADC (2 x 1.6GSPS ADC (2 x 1.6GSPS 30Gbps 60Gbps 60Gbps 30Gbps 30Gbps MiniPOD 12x10Gbps MiniPOD 12x10Gbps 30Gbps 30Gbps 60Gbps 60Gbps 2 Dual ADC (4 x 1.6GSPS Kintex-7 XC7K325T Kintex-7 XC7K325T 2 Dual ADC (4 x 1.6GSPS ADC (2 x 1.6GSPS ADC (2 x 1.6GSPS 30Gbps 76.8Gbps 76.8Gbps CSIRO. Next Generation ASKAP Hardware
Dargonfly-3 Hardware Layout Optical I/O RFoF Receiver 300W ATX PS >80% Efficient ADC Kintex-7 325T Synthesiser MiniPOD TX/RX Control Board (Bullant) Fans CSIRO. Next Generation ASKAP Hardware
Redback-3 Hardware • Six Kintex 7 FPGA (XC7K355T-2FFG901) • 55.6K slices, 25.7Mbit RAM, 1440 DSP • Dynamic configuration through control interface • Larger FPGA (420T or 480T) can be used on same footprint • FPGAs are arranged as pairs, the 3 pairs on the board are identical • Signal processing can treat FPGAs as 3 pairs, 2 triples or 1 group of 6 FPGAs • All FPGAs are fully interconnected using high speed serial links • Twelve DDR3 SODIMM sockets • Two sockets per FPGA compatible with Xilinx MIG requirements • Support up to 1066 MT/s memory (based on Kintex 7 datasheet) • Sockets will support 8GByte memory sticks • Three Avago 12x10.3125Gbps optical MiniPOD transmitter • Three Avago 12x10.3125Gbps optical MiniPOD receivers CSIRO. Next Generation ASKAP Hardware
Redback-3 Data Flow DDR3 (2 x SODIMM) DDR3 (2 x SODIMM) DDR3 (2 x SODIMM) 40Gbps 136.4Gbps 136.4Gbps 136.4Gbps Kintex-7 XC7K355T Kintex-7 XC7K355T Kintex-7 XC7K355T 40Gbps 40Gbps 60Gbps 60Gbps 60Gbps 20Gbps 20Gbps 20Gbps MiniPOD 12x10Gbps MiniPOD 12x10Gbps MiniPOD 12x10Gbps 60Gbps 60Gbps 60Gbps 20Gbps 20Gbps 20Gbps 60Gbps 60Gbps 60Gbps Kintex-7 XC7K355T Kintex-7 XC7K355T Kintex-7 XC7K355T 40Gbps 40Gbps 136.4Gbps 136.4Gbps 136.4Gbps 40Gbps DDR3 (2 x SODIMM) DDR3 (2 x SODIMM) DDR3 (2 x SODIMM) CSIRO. Next Generation ASKAP Hardware
Redback-3 Hardware Layout Optical I/O 300W ATX PS >80% Efficient DDR3 SODIMM MiniPOD TX/RX Kintex-7 355T Control Board (Bullant) Fans CSIRO. Next Generation ASKAP Hardware
ADE Control Structure • Based on a dedicated control board (Bullant) with a Virtex-6 FPGA • Allows changes to control system hardware without impacting signal processing • Each signal processing FPGA has a dedicated 1Gbps down (to FPGA) and 2Gbps up (from FPGA) to Ethernet port(s) via the control FPGA. • Uses UDP Ethernet packets for communication • No processor involved - simplifies programming • Allows full bandwidth of Ethernet interface to be utilised • Maximum flexibility in data format/packet sizes etc. • Complex programming is put into the controlling computer CSIRO. Next Generation ASKAP Hardware
Bullant-3 Control Card • Virtex 6 FPGA (XC6VLX75T-FF784) • 11.6K slices, 5.6Mbit RAM, 288 DSP, 4 Ethernet MACs • Configuration through onboard 64Mbit PROM, runtime updatable, free space for other data approx 36Mbits • Multiple Computing IO Options • Vitesse Dual 1GbE PHY (Copper & Optical Interfaces) • Netlogic 10GbE PHY (Optical Only) • JTAG testing • Quick and automated testing and programming of devices at assembly • Front RJ50 connector for timing inputs • Fully buffered differential inputs for synchronisation clock and/or 1pps signal • 64Mbit NOR Flash • 1Gbit DDR3-1066 chip • Pin out compatible with Xilinx MIG • Compatible with 2/4 Gbit DDR3 chips • Onboard fan controller for 8 external case fans • Two 68-pin control ports for Dragonfly or Redback cards • General IO • 12 Leds • 2 Front switches CSIRO. Next Generation ASKAP Hardware
Timelines • Initial design of hardware underway • Bullant schematic design and layout is complete • Dragonfly and Redback initial concept designs are complete and schematic work has started • Prototype hardware complete early in 2012 • Final production hardware in June 2012 • Production of full ADE system schedule to start in July 2012 CSIRO. Next Generation ASKAP Hardware
CASPER on ADE • Hardware design intended to be released under open source license • Optical Links on Redback-3 and Dragonfly-3 run at 10G Ethernet rates • Could be used as 10G Ethernet interfaces with appropriate Ethernet MAC • Xilinx Kintex-7 FPGAs will be compatible with system generator and therefore CASPER libraries • Dragonfly-3 integrates 16 high speed ADCs with large FPGAs and 24 Ethernet compatible 10G optical outputs (similar to Roach or iBoB) • Redback-3 integrates large signal processing FPGAs with high bandwidth memory interfaces with 36 Ethernet compatible 10G optical connections (similar to BEE boards) CSIRO. Next Generation ASKAP Hardware
Implementing a CASPER Control System • ASKAP Control System does not use a integrated processor for hardware interfacing. • Could use the Bullant platform and include a soft core power PC on the Virtex-6 FPGA • Bullant includes DDR3 memory and flash data storage compatible with a Xilinx Microblaze core • Using Microblaze would allow Linux to be installed on Bullant • Could modify the Bullant platform to use a processor instead of an FPGA • Multiple signal processing boards (Dragonfly-3 or Redback-3 boards) could be controlled from a single master control card. CSIRO. Next Generation ASKAP Hardware
Contact Us Phone: 1300 363 400 or +61 3 9545 2176 Email: enquiries@csiro.au Web: www.csiro.au Thank you CSIRO Astronomy & Space Science/Electronics Group Andrew Brown Lead Digital Hardware Engineer for ASKAP Phone: +612 9372 4283 Email: andrew.brown@csiro.au Web: www.csiro.au/cass