LCLS Maintenance Personnel and Procedures Paul Bellomo

1. LCLS Maintenance Personnel and Procedures Paul Bellomo

2. Topics • Scope and Perspective • Why we know we are ready • Department descriptions • ARTEMIS • The maintenance process • Examples

3. Scope and Perspective • Accelerator equipment • New electrical, mechanical, electro-mechanical and electronic equipment from the injector gun through the end of the tune-up dump TD-11. • New control room in Building 5 • New substations K10B, S520, MCC S20, panel-boards • Operations Directorate • 12 Departments, only 5 directly involved • Controls, CEF, Klystron/Microwave, Mechanical Fabrication, Power Conversion • Perspective • Maintenance departments

4. Why We Know We Are Ready • All departments • LCLS equipment possesses some new features, but is not totally unlike the equipment that SLAC currently services and maintains • Designers are the persons/groups that will maintain the equipment • Maintenance personnel are qualified and have been involved in the LCLS design from the start. Training requirements are modest and provided by LCLS, Department, or outside agency • Staff quantity is adequate to handle increased LCLS workload. The documentation (detailed later) needed to support the systems exist • Procedures (detailed later) needed to maintain the systems exist

5. Department Descriptions • Department Head – Hamid Shoaee, staff of 75 • Software and hardware sections • Software Section • Write low-level applications, EPICS drivers, displays • Maintain MATLAB infrastructure (licenses), ensure compatibility with all control room platforms • Provide network hardware and software • Provide control room infrastructure (alarms, displays, servers and workstations) • Development and maintenance are the same people

6. Department Description - Controls (Continued) • Hardware Section • Beam Containment System (BCS) • Prevents radiation from 'escaping' the shielding enclosure • BSOICs, BTMs, LIONS, toroids and current monitors • Machine Protection System (MPS) • Protect equipment from beam damage • VME processor-based protection ion chambers (PICs), RTDs, digital monitors • Personnel Protection System (PPS) • Protect personnel from radiation exposure, laser and electrical hazards • PLC driven systems • Vacuum - PLC based gages, valves, pumps and interlocks • Movers - Tables, mirrors, cameras, wire scanners

7. Department Head – John Weisend, staff > 150 • CEF Operations - Bernie Romero, staff of 28 • Electrical maintenance of substations and electrical distribution equipment • Beam-line manifolds and plumbing • Documentation • One-line diagrams • Panel-board schedules • Manufacturer drawings and instruction manuals • Beam line assembly and layout drawings

8. Klystron/Microwave Department • Department Head - Chris Pearson, staff of 80 • Microwave Engineering and Maintenance/AMRF, staff of 15 • Positron Source, Damping Rings, LINAC, LCLS LINAC and SLC RF systems and 2856MHz pulsed klystrons • PEP (HER and LER) and SPEAR 3 RF systems and 1.2 MW, 476MHz CW klystrons • Documentation • Rack front views and Captar rack profile drawings • Electrical interconnect diagrams, cable database • Electronic chassis diagrams • Manufacturer drawings and instruction manuals • ELPs, EWPs, bench test, hi-pot procedures

9. Mechanical Fabrication Department (MFD) • Department Head – Karen Fant, staff of 80 • Mechanical fabrication/maintenance – no design • Vacuum gages and pumps on the vacuum side, pneumatic valves, beam pipe and other components designed by others (LCLS staff) • Magnet cooling water hoses, disassemble magnets to find magnet problems and ground faults • On-call support of Operations • Documentation • Beam line assembly and layout drawings • Vacuum pump-down procedures

10. Department Head – Paul Bellomo, staff of 75 • DC magnet power systems and RF modulators • Cables, cable tray and raceways • Dedicated maintenance staff of 31 people, 24/7, 3-shift operation • Participate in power system commissioning • Documentation • Rack front views and Captar rack profile drawings • Electrical interconnect diagrams, cable database • Electronic chassis diagrams • Manufacturer drawings and instruction manuals • ELPs, EWPs, bench test, hi-pot procedures

11. ARTEMIS • Accelerator REMEDY Trouble Entry & Maintenance Information System • Web-based problem reporting, work scheduling and work status program • Report and track both hardware and software problems • Incorporates ISMS core functions • Five states • New Entry – report problems or schedule maintenance • In Progress – Jobs in progress or have more work to do • Scheduled Jobs – Input from the New Entry state • Review To Close – All known work completed, ready for management review • Closed – trouble report entered into archive

12. ARTEMIS

13. The Process

14. Example S20 Power Distribution Layout

15. Example One-Line Diagram Substation K10B 13.8kV:480V to S20 MCC to 6 Power Supply Racks Portion of S20 MCC shown Excerpt from ID-950-075-03

16. Example Rack/Equipment Profile

17. Example Electrical Interconnect (EI/ELP) Diagram

18. The PCD Process – Example 1 • LCLS bend magnet system fails unexpectedly at 3AM. The indications are that a power supply has failed. • LCLS Operations files an ARTEMIS and also notifies the PCD owl shift supervisor/maintenance manager (MM). Ops classifies the problem as critical, because the machine is down without this particular magnet. • MM evaluates the system, reviews past ARTEMIS reports and chooses appropriate technicians to troubleshoot the problem. • The MM, the technicians then assess the situation, from the technical, logistics and safety standpoints. The system is 480VAC input, 40VDC output. • Appropriate test equipment and PPE (Category 0 with Class 0 gloves) • The group notifies Ops that it is responding to the call. Ops turns the system over to PCD. PCD applies LOTO. ARTEMIS upgraded to “in process”

19. The PCD Process – Example 1 Continued • Work progresses power system will not turn on. After several hours, the solution is not readily apparent. • Ops escalates the problem to the designated system expert or Department Head. After 3 hours without resolution the Department Head and LCLS manager are notified. Work continues • The technical expert finds and solves a short in the power supply rectifier. Parts are field-replaced. • Ops is notified that the system is running. The system is returned to Ops. • Technicians file “review to close” ARTEMIS report, file e-log detailing the problem, the solution and parts used. • The PCD MM checks the parts inventory and if below a pre-determined critical level, orders replacement parts. • PCD closes the job internally

20. The PCD/MFD Process – Example 2 • A ground fault is discovered in a magnet system. Ops files ARTEMIS and notifies PCD. • The fault keeps the system off. It must be repaired immediately • PCD locks out the system. They troubleshoot the system and determine that the ground fault is in the magnet. PCD asks Ops to notify MFD. The ARTEMIS is annotated and responsibility re-assigned. • PCD replaces personal LOTO with group LOTO. The PCD MM is now responsible for the system. • MFD is called. MFD responds and confers with PCD. PCD demonstrates that the power supply cannot be turned on. • MFD apply personal locks over the PCD group locks. MFD apply other LOTO, verify magnet ground fault existence and perform work necessary to remove the ground fault.

21. The PCD/MFD Process - Example 2 Continued • The ground fault is found and repaired. The magnet is re-assembled and hi- potted. • MFD personnel remove their locks. PCD is re-called. • PCD replaces Group with personal locks. PCD hi-pots the system, remove their locks, turn the system on and release it to Ops. • PCD technicians file a “review to close” the ARTEMIS and file e-log entries • PCD MM conducts safety de-briefing and closes the job

22. Last Slide • The SLAC maintenance groups are ready and eager to safely and professionally support the LCLS project