1 / 25

Scalable Packetized Correlators

Scalable Packetized Correlators. Jason Manley, Aaron Parsons, Don Backer, Henry Chen, Terry Filiba , David MacMahon , Peter McMahon, Arash Parsa , Andrew Siemion , Dan Werthimer , Mel Wright. Outline. What is a correlator? Scalable packetized correlators: The architecture The hardware

devlin
Download Presentation

Scalable Packetized Correlators

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Scalable Packetized Correlators Jason Manley, Aaron Parsons, Don Backer, Henry Chen, Terry Filiba, David MacMahon, Peter McMahon, ArashParsa, Andrew Siemion, Dan Werthimer, Mel Wright

  2. Outline • What is a correlator? • Scalable packetized correlators: • The architecture • The hardware • The software • The cost • Closing thoughts • Walk through actual design • Questions and comments

  3. Interferometry…

  4. MWA XNTD PAPER FAST PAST LAR LWA

  5. Basic idea Vk ∑ ∑ ∑  Vjk Vjk Vkk ∑ Z-n Vik ∑ 90° Vii Amplitude Amplitude Time Time ∑  Vik Vi Vi ∑ ∑ Z-n Vij Vij 90° 90° ∑ Vjj Vj Vj ∑ ∑ Z-n  Vij  Vij

  6. “Actual” FX Correlator Z-n Z-n Z-n FFT FFT FFT ∑ ∑ ∑

  7. CASPER DSP backend concept

  8. Design Philosophy • Standardized processing hardware • Commercial interconnect • Asynchronous compute engines • Synchronization using common 1PPS • UDP output delivery over ethernet network • Correlator scales with your array

  9. CASPER FX Architecture

  10. Implementation

  11. Architecture to hardware mappingExample 8 Antenna system

  12. F Engine Operations • Two F engines per iBOB • Dual polarization design • Currently uses ASTRO library • Currently processes data at native clock rate (<200MHz IBOB or < 400MHz ROACH) DDC Channelize Quantize Reformat ADC

  13. Setup and Control • Clocks: • X engines each run off independent clock • Sampling synchronized at F engines, but clock not distributed to X engines • Synchronized using global 1pps signal at ADCs • Propagated to X engines using out-of-band signaling on XAUI links • Headers labeling 10GbE Ethernet packet data • System control: separate 100Mbps Ethernet network on BEE2 • F engines configured from BEEs through XAUI links • Control packets: CASPER UDP framework on BEE2 control FPGA • Execute Python scripts for configuration, control and debugging

  14. F engine development • 2008: • Coarse delays (cable length compensation) • Fringe-stopping & fine delays • Walsh code generation and phase switching • Real sampling (low bandwidth) • Parallel streams (high bandwidth) • Future: • Ability to output subset of band • Spectral zoom modes

  15. X Engine Operations • Using CASPER library • Scales with 2^N antennas • Fit as many X engines on an FPGA as possible (2x 16 ant on BEE2 usr) 10GbE Buffer X Eng Accum

  16. Backend Software • UDP packets received • Currently received, parsed and saved in MIRIAD file format by single computer. • Computing requirements dependant on experiment; • Usually single computer ok: 128 antennas, 1 sec integrations, 2k chan = 512MB/s

  17. Pending systems • Bench sys: 8ant, DP, 200MHz, 2k ch • PAPER: 128ant, DP, 100MHz, 2k ch • KAT-7: 8ant, DP, 256MHz, 2k ch • meerKAT: 80ant, DP, 1GHz, 16k ch • Bologna: 32ant, SP, 32MHz, 1k ch • GMRT: 32ant, DP, 400MHz, 4k-8k ch

  18. How does it scale

  19. FPGA Roadmap • Processing power doubling every two years • V4 = ½ power requirements of V2Pro* * Manufacturers claim - Xilinx Inc.

  20. Coming soon… • 10Gbps output optionally gives integrations ~10ms • More efficient use of hardware DSP slices • High speed, scalable, distributed data capture software • Walsh codes and phase switching • Phase rotation • 64 antenna design • Upgrade to 4096 channels • ROACH hardware: • <400MHz bandwidth • 16 384 channels • 128 antennas • no architectural changes

  21. Questions and Comments Visit the CASPER correlator page: http://casper.berkeley.edu/wiki/index.php?title=Correlator Add your own requirements: http://casper.berkeley.edu/wiki/index.php?title=International_Correlator_Collaboration Email me: jason_manley@hotmail.com

  22. PFB-FFT response

  23. Current usesPocket Spectrometer • Using ATMEL ADC’s at 2 Gsamples/sec • Performing 4 real FFT’s in 1 (complex) biplex pipelined FFT module. • 2048 channels • Uses just 1 ADC, 1 IBOB, and your laptop.

  24. ROACH block diagram

More Related