170 likes | 188 Views
Operational Experience @ SNS. Karen S. White Controls Group Leader 10/11/10. The Spallation Neutron Source. The world’s most powerful pulsed neutron source. Outline. Machine Performance Controls Performance Issues New Stuff. Accelerator Daily Performance.
E N D
Operational Experience @ SNS Karen S. White Controls Group Leader 10/11/10
The Spallation Neutron Source The world’s most powerful pulsed neutron source
Outline • Machine Performance • Controls Performance • Issues • New Stuff
Focus on Availability • Availability considered more important than increasing the beam power • Machine designed for 1.4 MW, but routinely operates at ~1MW to maximize availability and production hours
Controls Downtime • Includes MPS and PPS • FY09 = 116.8 hours • Dominated by PLC and PLC/IOC communications problems ~60 hr • PPS ~26 hr • MPS ~17 hr • Radiation Monitors ~14 hr • FY10 = 40.8 hours • DeviceNet ~13 hr • PPS ~12 hr • MPS ~6 • Radiation Monitors ~3 hr
PLC and PLC/IOC Communication • Late 2008, increase in PLC module failures and communication problems • Two failures impacted CHL resulting in trips and the downtime associated with 4K to 2K transition (16.3 and 19.4 hours) • Failure analysis revealed eight different contributing problems • Worked with Allen-Bradley engineers to understand failures and plan improvements • Analyzed control system response to failures in CHL and implemented improvements
Machine Protection System • Easy to “shoot the messenger” • Most downtime due to noisy input signals • Testing verified beam coupling related noise in the Ring Service Building • Several improvements were made to reduce the noise and help the MPS better tolerate the noise • Timing measurements showed many MPS nodes did not trip the beam within the required 20ms • Primary causes for MPS delay • Large internal capacitors installed for noise filtering • Open collector sublink drive circuit slow response (capacitors make this problem a lot worse)
Performance Issues – LLRF Controls • LLRF IOCs run automatic feed forward algorithm • Increasing beam power -> longer beam pulses -> more data • MVME2100 CPU could not processing all data for each 60Hz pulse • Upgrading CPUs to MVME5500
Increasing Controls Scope • Increasing support for Instruments • Applying accelerator controls technology where it makes sense for instrument construction and support • Instrument Personnel Protection Systems • Instrument CF*, Vacuum, selected device controls • Sharing engineering expertise for problem areas • Instrument work not in original Controls scope • Infrastructure not in place • Standards not in place – propagating accelerator controls standards *LonWorks talk Wednesday 9:30 am
Lessons • Take data – listen to what it tells you • Sometimes what appears to be “a problem” has more than one cause • Send clear messages for faults or errors • Monitor and evaluate bug reports; apply patches as needed • Plan for how the control system should respond to various types of failures • Important to strike a balance between new technology and need for high availability
CSS Tools • CSS applications for archiving (BEAUTY) and alarm handling (the BEAST) deployed in ops • Configured using RDB • Data and logging also stored in RDB • New DM, BOY under development to interoperate with B&B BOY – Thursday 11am and 3 pm The BEAST Thursday 5 pm
New Stuff • We now have the SNS Café in our building • Spring 2011, a nearby Guest House will open for visitors • Working on CD-2 for Power Upgrade Project • Working on CD-1 for Second Target Station