An Overview of the Reliability and the Fault Trends of the SRS

1. An Overview of the Reliability and the Fault Trends of the SRS Cheryl Hodgkinson Daresbury Laboratory Synchrotron Radiation Department WAO, Trieste, 24th - 28th September 2007

2. Contents • Introduction • SRS Operational Statistics • Analysis of the Operational Data • Efficiency • Fault Statistics • Ageing and Obsolete Equipment • Maintaining Reliability • Capital Investment and Redundancy • Risk Analysis • Post Incident Investigation and Modifications • Preventative Maintenance and Scheduling • Conclusions and Acknowledgments WAO, Trieste, 24th - 28th September 2007

3. Introduction • The Synchrotron Radiation Source • 2GeV, 2nd Generation Light Source • Constructed between 1975 and 1980 • Began scheduled user operations in 1981 • Some SRS systems, mainly injector components were salvaged from a previous accelerator • Operational user facility for 26 years and scheduled for closure December 2008 WAO, Trieste, 24th - 28th September 2007

4. 2001 -2007 • Dominated by water – vacuum incidents • 01/02 (65%) SCW 9 Absorber • 04/05 (85%) RF Cavity Window & Quadrupole Vessel Absorber • 06/07 (78%) TVM 9 • Efficiency in 06/07 was also significantly effected by Linac RF and HT systems and Gun Cathode issues. • Periods between incidents in excess of 90%. • 1984 – 1992 • A number of upgrades • HBL • Undulator • SC Wavelength Shifters • Erratic efficiency profile due to new systems 1993 - 2001 Stable period of operation Gradual increase in efficiency to figures in excess of 90% SRS Reliability Data WAO, Trieste, 24th - 28th September 2007

5. Vacuum Incidents SRS Fault Statistics WAO, Trieste, 24th - 28th September 2007

6. Vacuum Incidents • Usually an isolated single failure • Difficult to predict and prevent • failed RGA head • Quadrupole absorber – failed flowmeter • Common Theme? • SCW Absorber & TVM 9 • Both very different failure mechanisms • Both historical – not needed in the current SRS configuration WAO, Trieste, 24th - 28th September 2007

7. Beam Loss Cause Unknown RF Klystron PSU RF System Controls SRS Fault Statistics WAO, Trieste, 24th - 28th September 2007

8. Ageing and Obsolete Equipment • Early Equipment • Built in-house • Repairs in-situ (time) • Can be repaired • Later Equipment • Plug-in • Replaced with a spare • Return to manufacturer for repair • Support issues WAO, Trieste, 24th - 28th September 2007

9. Ageing and Obsolete Equipment • Concern regarding condition of in vacuum water circuits • 17 year old absorber removed from storage ring • Area’s milled away to examine any wall thinning RESULT No evidence of wall thinning or erosion WAO, Trieste, 24th - 28th September 2007

10. Ageing and Obsolete Equipment • External water circuits • Elbows • Near feedthroughs • Potential for vacuum incident during repair • Potential to cause more serious failure WAO, Trieste, 24th - 28th September 2007

11. Ageing Machine – It’s not just equipment! • It’s about people too! • Loss of design and build knowledge, as staff retire • Less documentation for older equipment, drawings, maintenance schedules, etc. • Can’t teach 20 years design, build and operational knowledge. • Succession planning important • You only know what’s been missed when the fault occurs • Lapses in maintenance schedules • Longer fault rectification WAO, Trieste, 24th - 28th September 2007

12. Capital Investment and Redundancy Old RF PSU • Capital investment • Must be bid for against other priority items • Only essential systems and components which would significantly improve efficiency • Spares • Hot • Power Converters • Storage Ring Klystron • Vacuum vessels New Commercial Unit WAO, Trieste, 24th - 28th September 2007

13. Risk Analysis • Benefit difficult to assess • The biggest risk is the one you don’t know about • At top level • Difficult to compare risk and focus resource • How do you compare extremely reliable long lead time items against components that fail frequently, but are a fast repair? • Does highlight major issues • Linac Klystron WAO, Trieste, 24th - 28th September 2007

14. Risk Analysis • Most beneficial at individual technical group level • Direct comparison can be made between equipment and fault hours contributed • Led to successful modernisation in key area’s based on safety and operational risk WAO, Trieste, 24th - 28th September 2007

15. Post Incident Investigation • Learning from incidents is important • Committee is always formed to recover and also determine actions to prevent reoccurrence. • Actions from recent faults • Modification to TVM cooling pipes • Revised interlock testing schedule • Test rig to test thermionic valves on-site WAO, Trieste, 24th - 28th September 2007

16. Preventative Maintenance and Scheduling WAO, Trieste, 24th - 28th September 2007

17. Conclusions • The SRS has provided reliable user operations for over 26 years • Vacuum incidents are isolated incidents, which do not indicate any form of systematic failure which would limit the life of the SRS • Only evidence of any systematic failure due to ageing is the failure of copper elbows on water circuits • The high operating efficiency demanded by the light source community can be maintained on an ageing machine using a variety of techniques. WAO, Trieste, 24th - 28th September 2007

18. Acknowledgments • The author would like to thank • Previous SRS Operations Managers for compiling these statistics • Current and past Operations Teams for providing the data • Steve Griffiths of EE&PS for providing his Groups Risk Analysis Data And Finally........... Daresbury Staff past and present for their contributions to SRS Operations WAO, Trieste, 24th - 28th September 2007