Performance of RHIC Vacuum Instrumentation and Control

1. Performance of RHIC Vacuum Instrumentation and Control H.C. Hseuh and L.A. Smart Collider-Accelerator Department Brookhaven National Laboratory April 11, 2005 • Outline: • Description of RHIC Vacuum I & C • Performance and Reliability • Summary RHIC Vacuum I&C Hseuh & Smart

2. RHIC Vacuum I&C Hseuh & Smart

3. BNL Collider-Accelerator Vacuum Systems 3.8 km x 2, <10-10 Torr 150m, 10-8 Torr ~500m, 10-9 Torr 800m, ~1x10-8 Torr 200m, ~2x10-11 Torr ~900m, ~10-10 Torr RHIC Vacuum I&C Hseuh & Smart

4. RHIC Vacuum Instrumentation (Excludes ~ 800 vacuum devices for the injector chains) ~ 1600 Vacuum Devices w/ remote control and monitoring Beam Vacuum Systems (Warm + Cold) 420 CCGs, 150 TCGs, 15 RGAs w/ E-M 170 SIPs, 320 TSPs 133 Sector Gate Valves Insulating Vacuum Systems 130 CCGs, 230 TCGs, 76 RGAs w/ FC ~ 42 TMPs (with isolation valves, TCG, CCG, …) Control and Monitoring 8 PLCs  4 FEC (VME Power PC)  CLC (Control Level Consoles) ~140 CCG Analog Signals to MADC(for fast logging) RHIC Vacuum I&C Hseuh & Smart

5. Control Architecture • Based on Programmable Logic Controller (PLC) • 8 Allen-Bradley PLC 5-40E or 5-20E processors • 2 Co-processor + expander setsin each PLC • OS-9 real-time operating system • RS-485 multi-drop serial interface ports for GC, IPC, RGA, TMP • < 5-second update time with ≤ 32 devices at each port • Inputs from GC and IPC set points • Outputs for valve interlock (thru 2 out of 3 set point voting scheme), Beam Permit Link (any valve closed), RF & Kicker interlocks • PC GUI at two locations w/ Data Highway links to all PLCs for diagnostics and downloading programs (independent of CLC) RHIC Vacuum I&C Hseuh & Smart

6. Control Architecture –cont’d • Distributed PLC Control • Relieves CLC of low-level tasks - simpler software & hardware • Front-end computer (FEC) • 4 VME based Power PC processors • 100 Mb/s Ethernet links to CLC and PLC co-processors • Console Level Consoles (CLC) • Control & monitoring via spreadsheets, graphics, alarms • Data Logging and LogView • Elog for machine operations • Elog (e.g. Vacuum Elog) with each subsystem RHIC Vacuum I&C Hseuh & Smart

7. RHIC Vacuum I&C System with RS-485 Networks ~ 640m RHIC Vacuum I&C Hseuh & Smart

8. GC, IPC, TSP, PLC, PC/GUI, … At service buildings PLC, Co-Processors, I/O Modules TMP w/ GC, SLC w/ RS485 serial link TMP, CCG, TCG & Valves on Cryostats RHIC Vacuum I&C Hseuh & Smart

9. LogView for diagnostics… e.g. ∆P vs. I & bunch length Most useful applications for Vacuum Spreadsheets Live monitoring and control, open architecture, no password Logging & View from minutes to 720 Hz with synchronized time stamps; compare ∆P with other machine variables with ~msec resolution Vacuum ELog Vacuum maintenance and operation details entered by technicians and staff for diagnostics and archives Graphic Display2-D, 3-D, waterfall… Bunch Length IR10 Pressure rise IR2 Pressure rise Fill 5160 RHIC Vacuum I&C Hseuh & Smart

10. Spreadsheet for Turbopump Operation Spreadsheet for RGA Operation RHIC Vacuum I&C Hseuh & Smart

11. Performance and Reliability of Vacuum PLC and Controls Interfaces • 8 PLC, Processors & BPL – very reliable for machine operation, device I/L… • PLC and GC are backed by UPS, IPC backed by Generators • 14 Co-processors - hung several times a week, but does not affect beam • Communication errors? Resulted in no logged data, • Remote soft reset (close/open ports) • Some required local manual reset • Must be checked daily • Common software for all subsystems at control console • Spreadsheet, Graphic Display, Alarm… • LogView allows comparison with other machine parameters • Machine ELog and Vacuum ELog – detailed archives for diagnostics RHIC Vacuum I&C Hseuh & Smart

12. Performance and Reliability of Vacuum Devices GC, CCG - < 3% failure per yr HV cards (thermistors ➱ abnormal readings) CC gauges (neon bulbs, ➱no ion current) IPC - ~ 5% failed in 2003, less in 2004 HV cards, chassis TMP - In-house designed Station Logic Controller Possible radiation–induced errors ~ once a week (FIV, PIV closed) Some need ring access for reset RGA - Single filament, custom software, 6 masses only ~ 5% filament failure per yr Hardware and software are no longer supported by vendor TSP - In-house design to accommodate long cable run With local step-up (I) transformer, manual operation Remote capable, but not implemented SGV - ~ 5% faulted (momentarily) in 2004, resulted in BPL faults Stuck open due to corrosion in solenoid RHIC Vacuum I&C Hseuh & Smart

13. Summary • Reliability of RHIC Vacuum Instrumentation are much better than other sub-systems over the last five years • PLC based control has worked reliably for vacuum and machine protection • Coprocessor RS485 interface is susceptible to communication errors • Results in lost data • Update time of > seconds per port • Augmented with analog signals to MADC for fast (msec) logging • Identical GC, CCG and TCG for beam and insulating vacuums • CCG neon bulb disconnected during bakeout • Conversion kit is available but expensive • IPC works relatively well • Repair delays due to custom products RHIC Vacuum I&C Hseuh & Smart

14. Considerations for the Future • CCG - Buy gauges designed for operation at bakeout temperatures • IPC, GC - Buy off-the-shelf controllers to avoid repair delays for custom units • PLC - Include analog inputs for fast gauge monitoring • TMP – Any commercial turn-key stations with integrated valves and gauges? • RGA – Dual filaments • Avoid custom software and interfaces • TSP - Any commercial units for long cable run(with local transformer?) • With remote control? • SGV – Corrosion resistantsolenoids? RHIC Vacuum I&C Hseuh & Smart