Improving the Accelerator Toolbox through Implementation of Methods for Parallel Computing

1. Improving the Accelerator Toolboxthrough Implementation of Methods for Parallel Computing Evan Li Xiaobiao Huang SLAC National Accelerator Laboratory August 12, 2010 Evan Li, Brown University 2012 evan_li@brown.edu

2. What is Accelerator Toolbox (AT)? • Toolbox for MATLAB, specifically oriented toward solving problems in computational accelerator physics • Combines heavily matured numerical methods and algorithms for beam tracking with the user-friendliness of the MATLAB interface • Memory management and datatype conversion are handled automatically, allowing for increased efficiency and flexibility in beam tracking • Combines existing codes and frameworks in to a simple “all-in-one” package Evan Li, Brown University 2012 evan_li@brown.edu

3. Parallel Computing • Parallel computing (as opposed to serial computing) involves executing a problem by simultaneously running computations on multiple processors • Allows for management of large data sets as well as increased performance speed • Machines have steadily switched to this model for processing • To keep AT updated with modern standards, parallel implementation is both logical and essential Serial Computing Model Parallel Computing Model Evan Li, Brown University 2012 evan_li@brown.edu

4. Beam Tracking • The beam is expressed as a 6xN dimensional array • The beam contains N particles • Each particle is represented as a 6-dimensional column vector • Populates a 6-dimensional phase space • x1 and x1’: horizontal displacement its derivative (with respect to s) • x2 and x2’: vertical displacement its derivative • z: axial displacement • delta: off-momentum • Observed in a co-moving coordinate system with the beam Evan Li, Brown University 2012 evan_li@brown.edu

5. Pass Methods • Beam trajectory is evolved as it passes through the ring’s lattice of cavities and magnets • Most basic lattice elements • Drift Space: Just vacuum, no electromagnetic fields • Bending Element: Magnetic dipole, used to steer the beam • Focusing Elements: Quadrupoles, used to focus the beam • The pass methods for these elements can simply be expressed as linear transfer matrices • The motion of the beam in a lattice containing only dipoles and quadrupoles can be easily solved analytically Evan Li, Brown University 2012 evan_li@brown.edu

6. Chaos and the Dynamic Aperture • Unfortunately, sextupoles and higher order magnets have passmethods that cannot be described simply using a transfer matrix • When these magnets are introduced, the behavior of the beam becomes chaotic at large amplitudes, and the beam optics must be optimized to reduce chaotic behavior • To test the beam optics, we calculate the dynamic aperture of the beam • The dynamic aperture is a limitation on the transverse displacement of each particle • Particles outside of the dynamic aperture will exhibit unstable orbits and eventually collide with the physical aperture, i.e. the walls of the vacuum • Calculating the dynamic aperture is a computationally intensive process! • No analytical solution for beam motion, so particle trajectories must be individually calculated • Particles trajectories must be calculated for thousands of turns before the trajectory can be declared stable • Thousands of particles must be tracked to obtain a rough visualization of the dynamic aperture • The dynamic aperture must be recalculated every time the ring optics are adjusted, which can mean days of calculation before an optimal design is found Evan Li, Brown University 2012 evan_li@brown.edu

7. ringpass(RING, Rin, NT) • Calculates trajectory based on the beam lattice RING, the beam’s initial coordinates Rin, and the specified number of turns NT • Used to track particle trajectories for a huge number of revolutions • Calculations are done numerically and reiterated millions of times • Used in determining the dynamic aperture to address the problem of aperture optimization • Individual calls to ringpass can take several hours to complete • Parallelization of the problem allows for simultaneous calculation of multiple particle trajectories, dramatically increasing the efficiency of the process Poincare Map Evan Li, Brown University 2012 evan_li@brown.edu

8. Data from Ringpass Evan Li, Brown University 2012 evan_li@brown.edu

9. Running MATLAB Applications in Parallel • MATLAB isn’t open source, so developing tools for it can be difficult • Much of the difficulty in the project comes from finding methods for communication and message-passing between nodes • Parallel Computing Toolbox • Commercially developed by Mathworks • Each separate node requires a MDCS license: won’t use enough of the software features to justify the enormous cost factors • Introducing PCT as a system requirement would detract from AT’s utility as a widely available toolbox for computational accelerator physics • Tried a lot of other things, lots of misdirection, blech… • MatlabMPI • Open-source set of MATLAB scripts developed by MIT Lincoln Laboratory • Implements functions from the Message Passing Interface (MPI) library, which has become the standard tool for communication for running programs on multiple processors • Not as user-friendly: requires lower-level “housekeeping” tasks involving memory management and data parsing • Massive headache trying to run programs and debug :( • Uses file I/O to communicate between separate processes • Can work adhering to both shared and distributed memory computing structures Evan Li, Brown University 2012 evan_li@brown.edu

10. Measured Speed Increases • We were successfully able to demonstrate dramatic increases in processing speeds by dividing AT’s passmethods into multiple threads to be run on a small number of local processors • Measured processing times using MATLAB’s tic, tocfunctions: • Parallelization algorithms showed around 90% efficiency for each additional processor • Demonstrated processing speed increased by a factor of over 3.7 by parallelizing on a local office computer Evan Li, Brown University 2012 evan_li@brown.edu

11. Cluster Access • With the success of parallelization efforts at SSRL, including those of AT, there has been discussion of gaining access to larger scale computing clusters for tasks requiring high-performance processing • With plans to upgrade the SPEAR3 project in the near future, SSRL will be using AT very frequently to produce accurate simulations and predictions for beam behavior • Dynamic aperture optimization algorithms are essential in maintaining stable beam dynamics • If we take advantage of a 128-node cluster, a process that could take over 8 days could be reduced to a 15 minute process • This is an enormous upgrade for the computing power of the AP group at SSRL Evan Li, Brown University 2012 evan_li@brown.edu

12. Future Work • Passmethods for AT can be improved in a number of ways • Higher compensation for “fringe field” effects • More methods to simulate particle interactions within the beam • Algorithms for dynamic aperture optimization • Implementing more efficient algorithms, such as application of genetic algorithms in lattice optimization • Documentation • While AT continues to improve, its documentation has not been updated in recent years to reflect these upgrades Evan Li, Brown University 2012 evan_li@brown.edu

13. Evan Li, Brown University 2012 evan_li@brown.edu

14. Acknowledgements • Xiaobiao Huang • Department of Energy • SULI and SLAC National Accelerator Laboratory • Steve Rock, Eric Shupert, Shannon Ferguson, Christine Green Evan Li, Brown University 2012 evan_li@brown.edu