Conventional & Unconventional Applications of FPGA

1. Conventional & Unconventional Applications of FPGA Wu, Jinyuan Fermilab Oct, 2010

2. Fermi National Accelerator Laboratory

3. Colliding Experiments

4. Introduction • There is no clear distinction between conventional and unconventional applications of FPGA. • The range of FPGA application is very likely to be broader than we can image. • The outline of this talk: • The Starting Point • Topics on Averages • Using FPGA as ADC • TDC Implemented with FPGA • Serial Communication Between FPGA Devices • Doublet Finding in Trigger System • Triplet Finding in Trigger System Conventional & Unconventional Applications of FPGA

5. The Starting Point Conventional & Unconventional Applications of FPGA

6. Shaper LP Filter Spectrum of Original Signal LP filter ADC Input Sampling In ADC Aliasing w/o LP Filtering Cares Must Be Taken Outside FPGA (1) Band Limiting FPGA ADC Nyquist Frequency < (1/2) Sampling Frequency Conventional & Unconventional Applications of FPGA

7. The “Trend” vs. The Sampling Theorem There will be no hardware analog processing. Everything is done digitally in software. A shaper/low-pass filter is a minimum requirement. It sounds very stylish  Conventional & Unconventional Applications of FPGA

8. n Shaper LP Filter Cares Must Be Taken Outside FPGA (2) Dither FPGA ADC Resolution finer than the ADC LSB can be achieved by adding noise at ADC input and digital filtering. Conventional & Unconventional Applications of FPGA

9. Adding Noise for Finer Resolution • Mechanical pressure gauges usually do not track small pressure changes well. • The gauge readers may lightly tap the gauges to get more accurate reading. • The idea of dithering at ADC input is similar. Photo Credit: www.telegraph.co.uk, trinities.org Conventional & Unconventional Applications of FPGA

10. Why Band Limiting & Dithering are Ignored? • Pre-amplifiers usually have a naturally limited bandwidth and an intrinsic noise larger than the LSB of the ADC. • So a lot of time, band limiting and dithering can be “safely” ignored since they are satisfied automatically. • High bandwidth, low noise devices now become easily accessible. A design can be too fast and too quiet. • Do not forget to review the band limiting and dithering requirements for each design. Conventional & Unconventional Applications of FPGA

11. Topics on Averages Conventional & Unconventional Applications of FPGA

12. From Sum to Average 4 ＋） Integer Fractional • When N=2^k values are summed, word length +k bits. • These k bits are fractional bits of the average. Conventional & Unconventional Applications of FPGA

13. Gain of Measurement Precision 64 ＋） Integer Fractional    • When N independent measured values are averaged, precision improves by a factor of 1/sqrt(N). • If N=2^k, precision gain k/2 bits, (not k bits). Conventional & Unconventional Applications of FPGA

14. Weighted Averages c7 d7 e7 c6 d6 e6 c5 d5 e5 • The weighted average is a special case of inner product. • Multipliers are usually needed. c4 d4 e4 c3 d3 e3 c2 d2 e2 c1 d1 e1 y7 y6 y5 y4 y3 y2 y1 X X X S S S Conventional & Unconventional Applications of FPGA

15. Exponentially Weighted Average • No multipliers are needed. • The average is available at any time. • It can be used to track pedestal of the input signals. s[n]=s[n-1]+(x[n]-s[n-1])/N N=2, 4, 8, 16, 32, … s[n-1] x[n] 1/2K - + s[n] 1/2K Conventional & Unconventional Applications of FPGA

16. x[n] x[n] -x[n-K] S + s[n] s[n] Sliding Sum/Sliding Average   • For each input point, a sliding sum is computed. • It is preferable to implement sliding sum in recursive fashion. • Recursive implementation uses much less resources. Conventional & Unconventional Applications of FPGA

17. x[n] -x[n-K] + s[n] Sliding Average as a Low-Pass filter • Sliding average removes high frequency random noise. Conventional & Unconventional Applications of FPGA

18. The CIC Filters Sliding Sum Cascaded Integrator Comb (CIC) Sum of 2nd Order The Zero (e.g. 360Hz) • The CIC-2 filter is a weighted average. • Sliding Sum = CIC-1 sum. • The frequency response of CIC-2 sum is a sinc2(x) function that has 2nd order zeros and better stop band suppression. Frequency Conventional & Unconventional Applications of FPGA

19. x[n] x[n] -x[n-K] + -2x[n-K] + x[n-2K] s[n] u[n] -s[n-K] + + y[n] y[n] The CIC-2 Sum as a Low-Pass Filter Sliding Sum CIC-2 Sum • No Multipliers are needed to implement the CIC-2 sum. Conventional & Unconventional Applications of FPGA

20. Using FPGA as ADC Conventional & Unconventional Applications of FPGA

21. The Single Slope ADC FPGA • Analog signal of each channel from the shaper is fed to a comparator and compared with a common ramping reference voltage VREF. • Pulses, rather than analog signals are transmitted on the cable. • The times of transitions representing input voltage values are digitized by TDC blocks inside FPGA. • This approach sometimes is (mistakenly) refereed as “Wilkinson ADC”. Signal Source Line Driver Shaper ADC ADC Line Driver Shaper ADC ADC Line Driver Shaper ADC ADC Line Driver Shaper ADC ADC FPGA Signal Source Shaper TDC TDC TDC TDC Shaper V2 Shaper V1 Shaper T1 T2 VREF Conventional & Unconventional Applications of FPGA

22. TDC Resolution Requirement V2 V1 T1 T2 500 ns • Consider sampling rate at 2 MHz, the whole ramping cycle is 500 ns. • Arrange 409.6 ns for upward ramping. • To achieve 12-bit ADC precision, the TDC LSB is (409.6 ns)/4096 = 100 ps. • TDC with 100 ps LSB can be comfortably implemented in FPGA today. Conventional & Unconventional Applications of FPGA

23. Digital Noise During Digitization Shaper ADC V2 V1 T2 T1 Noisy Clean Noisy Clean • Typical ADC devices creates noise that may interfere the analog circuits. • The time interval for resetting of the common reference voltage may be noisy but analog signal is not sampled during it. • There is no digital control activities during ramping up of the common reference voltage. Conventional & Unconventional Applications of FPGA

24. Single Slope ADC Test: Waveform Digitization • Shown here is a demo of a 6-bit single slope TDC. • Sampling rate in this test is 22 MHz. • Both leading and trailing reference ramps are used in this example. • Nonlinear reference ramping is OK. The measurement can be calibrated. FPGA TDC TDC VREF 50 50 Input Waveform, Overlap Trigger & Reference Voltage 1000pF 100 Raw Data Calibrated Conventional & Unconventional Applications of FPGA

25. TDC Implemented with FPGA Conventional & Unconventional Applications of FPGA

26. Clock Domain Changing Multi-Sampling TDC FPGA Multiple Sampling Q3 QF c0 c0 QE Q2 • Ultra low-cost: 48 channels in $18.27 EP2C5Q208C7. • Sampling rate: 360 MHz x4 phases = 1.44 GHz. • LSB = 0.69 ns. c90 QD Q1 c180 Q0 c90 c270 DV T0 T1 Trans. Detection & Encode 4Ch Coarse Time Counter TS Logic elements with non-critical timing are freely placed by the fitter of the compiler. This picture represent a placement in Cyclone FPGA Conventional & Unconventional Applications of FPGA

27. TDC Using FPGA Logic Chain Delay • This scheme uses current FPGA technology  • Low cost chip family can be used. (e.g. EP2C8T144C6 $31.68)  • Fine TDC precision can be implemented in slow devices (e.g., 20 ps in a 400 MHz chip).  IN CLK Conventional & Unconventional Applications of FPGA

28. Two Major Issues In a Free Operating FPGA • Widths of bins are different and varies with supply voltage and temperature. • Some bins are ultra-wide due to LAB boundary crossing Conventional & Unconventional Applications of FPGA

29. Digital Calibration Using Twice-Recording Method IN • Use longer delay line. • Some signals may be registered twice at two consecutive clock edges. N2-N1=(1/f)/Dt The two measurements can be used: • to calibrate the delay. • to reduce digitization errors. CLK 1/f: Clock Period Dt: Average Bin Width Conventional & Unconventional Applications of FPGA

30. Digital Calibration Result N2 • Power supply voltage changes from 2.5 V to 1.8 V, (about the same as 100 oC to 0 oC). • Delay speed changes by 30%. • The difference of the two TDC numbers reflects delay speed. • Warning: the calibration is based on average bin width, not bin-by-bin widths. Corrected Time N1 Conventional & Unconventional Applications of FPGA

31. Auto Calibration Using Histogram Method • It provides a bin-by-bin calibration at certain temperature. • It is a turn-key solution (bin in, ps out) • It is semi-continuous (auto update LUT every 16K events) 16K Events DNL Histogram S LUT In (bin) Out (ps) Conventional & Unconventional Applications of FPGA

32. Good, However • Auto calibration solved some problems  • However, it won’t eliminate the ultra-wide bins  Conventional & Unconventional Applications of FPGA

33. Cell Delay-Based TDC + Wave Union Launcher The wave union launcher creates multiple logic transitions after receiving a input logic step. The wave union launchers can be classified into two types: • Finite Step Response (FSR) • Infinite Step Response (ISR) This is similar as filter or other linear system classifications: • Finite Impulse Response (FIR) • Infinite Impulse Response (IIR) Wave Union Launcher In CLK Conventional & Unconventional Applications of FPGA

34. Wave Union Launcher A (FSR Type) Wave Union Launcher A 0: Hold 1: Unleash In CLK Conventional & Unconventional Applications of FPGA

35. Wave Union Launcher A: 2 Measurements/hit 1: Unleash Conventional & Unconventional Applications of FPGA

36. 1 2 Sub-dividing Ultra-wide Bins 1: Unleash Device: EP2C8T144C6 • Plain TDC: • Max. bin width: 160 ps. • Average bin width: 60 ps. • Wave Union TDC A: • Max. bin width: 65 ps. • Average bin width: 30 ps. 1 2 Conventional & Unconventional Applications of FPGA

37. FPGA TDC • A possible choice of the TDC can be a delay line based architecture called the Wave Union TDC implemented in FPGA. • Shown here is an ASIC-like implementation in a 144-pin device. • 18 Channels (16 regular channels + 2 timing reference channels). • This FPGA cost $28, $1.75/channel. (AD9222: $5.06/channel) • LSB ~ 60 ps. • RMS resolution < 25 ps. • Power consumption 1.3W, or 81 mW/channel. (AD9222: 90 mW/channel) Conventional & Unconventional Applications of FPGA Wave Union Launcher A In CLK

38. More Measurements • Two measurements are better than one. • Let’s try 16 measurements? Conventional & Unconventional Applications of FPGA

39. Wave Union Launcher B (ISR Type) Wave Union Launcher B 0: Hold 1: Oscillate In CLK Conventional & Unconventional Applications of FPGA

40. VCCINT =1.20V VCCINT =1.18V Wave Union Launcher B: 16 Measurements/hit 1 Hit 16 Measurements @ 400 MHz Conventional & Unconventional Applications of FPGA

41. Delay Correction The raw data contains: • U-Type Jumps: [48-63][16-31] • V-Type Jumps: other small jumps. • W-Type Jumps: [16-31][48-63] Delay Correction Process: • Raw hits TN(m) in bins are first calibrated into TM(m) in picoseconds. • Jumps are compensated for in FPGA so that TM(m) become T0(m) which have a same value for each hit. • Take average of T0(m) to get better resolution. The processes are all done in FPGA. Conventional & Unconventional Applications of FPGA

42. The Test Module Data Output via Ethernet FPGA with 8ch TDC Two NIM inputs BNC Adapter to add delay @ 150ps step. Conventional & Unconventional Applications of FPGA

43. Test ResultNIM Inputs RMS 10ps 140ps 0 1 2 Wave Union TDC B BNC adapters to add delays @ 140ps step. Wave Union TDC B + NIM/ LVDS Wave Union TDC B Wave Union TDC B - LeCroy 429A NIM Fan-out Wave Union TDC B NIM/ LVDS Wave Union TDC B + Wave Union TDC B Conventional & Unconventional Applications of FPGA Wave Union TDC B

44. Wave Union? Photograph: Qi Ji, 2010 Conventional & Unconventional Applications of FPGA

45. Coincidence in Trigger System Conventional & Unconventional Applications of FPGA Oct. 2010, Wu Jinyuan, Fermilab jywu168@fnal.gov 45

46. Parameters in Coincidence Finding Disc Sampling Edge Detecting Delay Pulse Stretch Coincidence Logic Disc Sampling Edge Detecting Delay Pulse Stretch

47. Some Details Disc Sampling Edge Detecting Delay Pulse Stretch

48. Doublet Finding in Trigger System Disc Sampling Edge Detecting Delay Pulse Stretch Coincidence Logic Disc Sampling Edge Detecting Delay Pulse Stretch Conventional & Unconventional Applications of FPGA Oct. 2010, Wu Jinyuan, Fermilab jywu168@fnal.gov 48

49. Hit Matching

50. Example of Doublet Match, PET T D DT<A? Group 1 - Group 2 DT>(-A)? T D • Positrons and electrons annihilate to produce pairs of photons. The back-to-back photons hit the detector at nearly the same time. • Detector hits are digitized and hits at nearly the same time are to be matched together. • The process takes O(n^2) clock cycles.