NSLS-II Fast Orbit Feedback with Individual Eigenmode Compensation

1. NSLS-II Fast Orbit Feedback with Individual Eigenmode Compensation Yuke Tian, Li Hua Yu Accelerator Division Photon Science Directory Brookhaven National Lab EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

2. Outline • NSLS-II orbit feedback system • Technical requirements and specifications • NSLS-II site field vibration measurement • Slow and fast corrector locations • Fast orbit feedback with individual eigenmode compensation • Typical fast orbit feedback algorithm • NSLS-II FOFB algorithm with individual eigenmode compensation • Comparison of the two algorithms • Solving the calculation problems from two directions • NSLS-II fast orbit feedback implementation • Summary EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

3. NSLS-II technical requirements & specifications EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

4. NSLS-II site field vibration measurement EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

5. Slow and fast corrector locations Even Cell Odd Cell EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

6. Let’s design a FOFB system Orbit feedback calculation BPM data Corrector setpoint BPM BPM BPM BPM BPM BPM PS Controller PS Controller BPM BPM PS Controller PS Controller Tx/Rx BPM data around the ring BPM BPM Orbit feedback calculation PS Controller PS Controller BPM data Corrector setpoint BPM BPM PS Controller PS Controller BPM BPM PS Controller PS Controller BPM BPM PS Controller PS Controller Tx/Rx BPM data around the ring EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

7. Can we make a clean architecture ? 1. Try to deliver BPM data to a place that orbit calculation module have directly access. Don’t shove data around too much. 2. Similarly, try to deliver corrector setpoint from a place that orbit calculation module have directly access. 3. Looks we need a place that can:  Receiver local BPM data;  Tx/Rx BPM data to/for other cell;  Carry out FOFB calculation;  Tx corrector setpoints to PS control system. Cell Controller ! EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

8. Typical fast orbit feedback algorithm Compensator (PID etc) Controller R-1=VΣ-1UT Accelerator R=UΣVT R: response matrix R-1: reverse response matrix • The ill-conditioned response matrix will cause numerical instability. • Solution: 1) Truncated SVD (TSVD) regularization • 2) Tikhonov regularization EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

9. Typical fast orbit feedback algorithm For each corrector plane • Each of the corrector setpoint is calculated from a 1xM vector and Mx1 vector multiplication. If 8 BPM/cell in NSLS-II, M=240. Total: ~240 MAC. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

10. NSLS-II FOFB algorithm – compensation for each eigenmode • Fast orbit feedback system is a typical multiple-input and multiple-output (MIMO) system. For NSLS-II, there are 240 BPMs and 90 fast correctors. The BPMs and correctors are coupled together. One BPM reading is the results of many correctors. One corrector kick can also affect many BPM readings. It is difficult to design a compensator for all noises with different frequencies. • It is desirable if we can decouple the BPM and corrector relationship so that the MIMO problem can be converted into many single input single output (SISO) problems, for which control theory has many standard treatments. • Fortunately, SVD already provides a solution: it projects the BPMs input into the eigenspace, where each component is independent. We can design many SISO type compensators (one for each eigenmode) and apply the standard SISO control theory to treat each eigenmode problem in frequency domain without affecting other eigenmodes. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

11. NSLS-II FOFB calculation – compensation for each eigenmode UT Q(z) Σ -1 V Accelerator R=UΣVT • c1, c2, …, cN is the input projections in the eigenspace. • Q1(z), Q2(z), …, QN(z) is the compensator for each eigenmode. • We want to prove that Q1(z), Q2(z), …, QN(z) only corrects the corresponding eigenmode in eigenspace without affecting other eigenmodes. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

12. NSLS-II FOFB calculation – compensation for each eigenmode UT Q(z) Σ -1 V Accelerator R=UΣVT • In cycle n, • Since, • The pure effect of Qi(z) is on the ith eigencomponent. • The noise signals are also decoupled into the eigenspace. • This gives us freedom to suppress the noises in eigenspace using SISO control theory. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

13. Decpoupling or not ? Calculation compare Without decoupling: For each corrector plane, M multiplication and accumulation(MAC), followed by k MAC for corrections. Calculation (one plane): One corrector : M+k N correctors: N*(M+k) With decoupling: It takes NxM MAC to decouple the inputs into eigenspace. It takes N*k MAC for N compensators in each eigenmode. It takes N*N to get one corrector strength. Calculation (one plane): One corrector: N*(M+N+k) N correctors: N*N*(M+N+k) Assume N=90, M=240, k=3: Assume 50us of 10Khz FOFB time is used for the caluclations: N*N*(M+N+k)/50us=5.400 GMAC/s. For doing two plane FOFB in 10us: 5.4*5*2= 54 GMAC/s The high end DSP chip gives about 200-300 Millions floating point operations (MFLOP). EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

14. Solving the calculation problems from two directions Distributed FOFB system: two tier communication 30 cells in the storage ring All the BPM data is delivered to all the 30 cell controller within 12us, cell controller only needs to calculate the local corrector strength. This distributed architecture reduces the calculation by factor of 30. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

15. Solving the calculation problems from two directions FPGA’s powerful DSP performance Standard DSP/CPU process: sequential FPGA’s fully implementation One of the DSP48 block in FPGA FPGA ‘s DSP power is mainly gained through the parallel computation capability. The parallelism increases FPGA DSP power (vs genetic DSP or CPU) by a factor of more than 2000 (Virtex 6). EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

16. Solving the calculation problems from two directions UT Q(z) Σ -1 V FOFB calculate in FPGA: Accelerator R=UΣVT NSLS-II FOFB Model Θ 1 V1 U1T Q1(z) Θ 2 V2 U2T Q2(z) Θ N VN UNT QN(z) EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

17. Solving the calculation problems from two directions FOFB calculate amount in FPGA: Θ 1 V1 U1T Q1(z) Decoupling into eigenspace: Θ 2 V2 All components c1(n), c2(n), …, cN(n) is calculated in parallel (M MAC) U2T Compensation for each eigenmode: Q2(z) The k step compensations are done in parallel. (k MAC) Θ N Corrector strength calculation: VN All corrector strength is calculated in parallel: N MAC UNT QN(z) The total calculation is reduced to: M+N+k = 240 + 90 + 3 = 333 MAC. FPGA will finish it within a few us. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

18. NSLS-II FOFB Implementation Control System (EPICS IOC) Download through GigE interface BPM data RAM From SDI link MAC To PSC RAM RAM RAM RAM RAM RAM MUX MUX MUX MUX MUX MUX MAC RAM RAM RAM RAM RAM RAM Θ 1 MAC V2 V1 To PSC U1T Q1(z) MAC U2T Θ 2 Q2(z) MAC VN To PSC Θ N MAC UNT QN(z) EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

19. NSLS-II FOFB Implementation Floating point calculation vs fixed point calculation: FPGA computation is usually fixed point calculation, the quantization errors should be carefully controlled so that the calculation is accurate. FPGA fixed calculation block Matlab floating point calculation EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

20. NSLS-II FOFB Implementation Error with PID calculation for one eigenmode Decoupling calculation error EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

21. Status • The hardware design (PCB, chassis) for BPM, cell controller and PSC are all done. PSC is in production stage. BPM and cell controller are in the first article of production stage. • BPM and PSC FPGA firmware, EPICS drivers and database development are all done. All cell controller blocks (SDI, FOFB etc) are all. The cell controller integration is in progress. • Since we have the fast fiber SDI to deliver data around the ring, cell controller’s SDI link will also be used as fast machine protection system that deliver critical system (such as the vast valve signal from vacuum system) around the ring within much less than 1ms. This latency is impossible for PLC to achieve. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

22. Open source hardware Want it ? You got it. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

23. Open source hardware • We like to open the hardware we designed at BNL to the community. These includes all the PCB design, the FPGA firmware design and the EPICS IOC (driver, database) design. We welcome any suggestions and cooperation from the community. PSC PSI EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

24. Open source hardware • The digital front end (DFE) board (Virtex6 FPGA) can be used as a common platform for many systems since it provides the basic common elements such as large memory (1 GB DDR3), embedded CPU, Gigbit Ethernet port to EPICS IOC, high speed SDI link(use 2 of the 6 available SFPs), and many user IOs (>160) to the user daughter board. BPM Cell Controller Share the same DFE EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

25. Open source hardware BPM control panel and raw ADC data EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

26. Open source hardware BPM turn-by-turn data and 10KHz FOFB data EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

27. Open source hardware BPM 10Hz slow acquisition data EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

28. Summary • NSLS-II’s stringent emittance requirements need a efficient fast orbit feedback system. The two tier communication structure and the FPGA-based fast orbit feedback calculation architecure is designed for achieve the requirments. • Algorithm with individual eigenmode compensation is proposed. The typical MIMO feedback problem is converted into many SISO problems. This algorithm enables accelerator physicists to correct the beam orbit in eigenspace. • We compared the calculations for FOFB with and without individual eigenmode compensation. We found that the proposed NSLS-II FOFB algorithm needs a large amount of calculations. However, benifited from NSLS-II FOFB architecure, the challenge can be conquered. • We expect a successful application of the NSLS-II FOFB algorithm during the NSLS-II commissioning and daily operation. • BNL like to open all the designs to the community as part of the open source hardware initiative. EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011

29. Acknowledgments • FOFB algorithm • Igor Pinayev • Sam Krinsky • BPM/ Cell controller development : • Kurt Vetter • Joseph Mead • Kiman Ha • Alfred Dellapenna • Joseph De Long • Yong Hu • Guobao Shen • Om Singh • Bob Dalesio • PSC and PS design: • Wing Louie • John Ricciardelli • George Ganetis EPICS Collaboration Meeting, Hsinchu, Taiwan, June, 2011