Dan Werthimer & Aaron Parsons University of California, Berkeley

1. Correlators, Spectrometers, Beam Formers and VLBI using general purpose FPGA boards, tools & libraries(how to build eight radio astronomy instruments in two years) Dan Werthimer & Aaron Parsons University of California, Berkeley http://seti.berkeley.edu

2. Our research group is really 3 groups • SETI (plus primordial black holes, HI mapping) • Public Participation Distributed Computing • CASPER – Center for Astronomy Signal Processing and Electronics Research

3. SETI Group David Anderson, Bob Bankay, Court Cannick, Jeff Cobb, Kevin Douglas, Josh Von Korff, Eric Korpela, Matt Lebofsky, Dan Werthimer

4. UC Berkeley SETI Programs

5. Public Participation Supercomputing Group David Anderson, Rom Walton, SETI Group • aka Distributed Computing • aka “edge resource aggregation”)

6. The SETI@home Client

7. SETI@home Statistics TOTAL RATE

8. BOINC: NSF • Berkeley Open Infrastructure for Network Computing • General-purpose distributed computing framework. • Open source. • Will make distributed computing accessible to those who need it. (Starting from scratch is hard!)

9. Projects • Astronomy • SETI@home (Berkeley) • Astropulse (Berkeley) • Einstein@home: gravitational pulsar search (Caltech,…) • PlanetQuest (SETI Institute) • Stardust@home (Berkeley, Univ. Washinton,…) • Earth science • Climateprediction.net (Oxford) • Biology/Medicine • Folding@home, Predictor@home (Stanford, Scripts) • FightAIDSathome: virtual drug discovery • Physics • LHC@home (Cern) • Other • Web indexing/search • Internet Resource mapping (UC Berkeley)

13. Where's the computing power? your computers home PCs academic business • 2010: 1 billion Internet-connected PCs • 55% privately owned • If 100M participate: • 100 PetaFLOPs, 1 Exabyte (10^18) storage

14. CASPER: Center for Radio Astronomy Signal Processing and Electronics Research Henry Chen, Daniel Chapman, Pat Crescini, Christina DeJesus, Pierre Droz Kirsten Meder, Jeff Mock, Aaron Parsons, Andrew Siemion, Dan Werthimer Radio Astronomy Lab Don Backer, Paul Demorest, Matt Dexter, Carl Heiles, David McMahon, Mel Wright, Lynn Urry Berkeley Wireless Research Center Bob Broderson, Chen Chang, John Wawrzynek SETI Institute Dave Deboer

16. Casper Origins • NSF proposal to build SETI spectrometer (2003) (added one paragraph: BTW, this can be used for other astronomy instrumentation, potential spin offs are ….) Reviewer’s comments (paraphased): ~”SETI is bullshit, SETI will never find anything, But these instruments are useful for the community, strongly recommend funding”

17. CASPER Real-time Signal Processing Instrumentation(NSF ATI) • Low NRE, shared by the community • Rapid development (8 instruments / 2 years) • Open-source, collaborative • Reusable, platform-independent gateware • Modular, upgradeable hardware • Industry standard communication protocols • Low Cost

18. MOTIVATIONATA, SKA, Focal Plane Arrays, SETI,need >> PetaOp/sec Instruments take a long time to build, very high NRE

19. Allen Telescope Array • 6.1-meter offset Gregorian (2.4-meter secondary) • rim-supported, hydroformed dishes

23. ATA-42 Operational This Summer

24. MWA XNTD PAPER FAST PAST LAR LWA The Radio Revolution

25. SKA Square Kilometer Array Inner core Station

27. The Problem with the CurrentHardware Development Model • Takes 5 years • Cost Dominated by NRE because of custom Boards, Backplanes, Protocols • Antiquated by the time it’s released.

28. Solution: • Modular Hardware • Low number of board designs • Can be upgraded piecemeal or all together • Reusable • Standard signal processing model which is consistent between upgrades.

29. Solution: use FPGA’s1 FPGA = 100 Pentium, 1/500 the power per op Moores Law for FGPA’s 3X improvement per year!

30. FPGA = Field Programmable Gate Arrayreconfigurable computing - 1 minute100 times faster than CPU, 5 times less powerinteger arithmetic, not good at F. Point highly parallel (500 multipliers per chip)harder to program (mathlab simulink)tools to abstract the hardware awaysignal processing libraries

34. Compute Module Diagram

35. Platform-Independent, Parameterized Gateware • What is Gateware? • Design logic of FPGAs (between hardware and software) • Need libraries for signal processing which don’t have to be rewritten every hardware generation. • Matlab Simulink!

37. Biplex Pipelined FFT • Uses 1/6 the resources of the Xilinx module.

38. FFT controls (Verilog and Simulink Libraries) • Transform length • Bandwidth • Complex or Real • Number of Polarizations • Input bit width and output bit width • twiddle coefficient bit width • Run-time programmable down-shifting • Decimate option

39. Filter Response: PFB vs. FFT

40. PFB vs. FFT

41. Additional PFB controls • Filter overlap • Width of filter coefficients • Window function for filter (hamming, hanning, etc.) • Import filter coefficients for custom filter performance • Both FFT and PFB available as Verilog modules • (no proprietary software, but not as portable • between chips/architectures).

42. Digital Down-Converter • Selectable # of FIR taps • On-the-fly programmable mix frequency • Selectable FIR coeff • Agile sub-band selection.

43. X-Engine Correlation Architecture (Lynn Urry, Aaron Parsons)

44. X-Engine Architecture:applied to an arbitrary sized antenna array

45. Hardware and Software Librarieslegend:

46. Applications

50. Global Interconnects • Commercial Infiniband switch from Mellanox, Voltaire, etc. • Packet switched, non-blocking • 24 ~ 144 ports (4X) per chassis • Up to 10,000 ports in a system • 200~1000 ns switch latency • 400~1200 ns FPGA to FPGA latency • 480Gbps ~ 2.88Tbps full duplex constant cross section bandwidth • <$400 per port