250 likes | 447 Views
SNS Project Status. Ernest L. Williams Jr. 27 April 2005 EPICS Collaboration Meeting @ SLAC. Outline. Installation Status Global Systems Network Timing MPS CCR Accelerator Systems Cryogenic Controls PPS Target Controls Operational Improvements Issues Summary.
E N D
SNS Project Status Ernest L. Williams Jr. 27 April 2005 EPICS Collaboration Meeting @ SLAC
Outline • Installation Status • Global Systems • Network • Timing • MPS • CCR • Accelerator Systems • Cryogenic Controls • PPS • Target Controls • Operational Improvements • Issues • Summary
The Spallation Neutron Source • The SNS construction project will conclude in 2006 • At 1.4 MW it will be ~8x ISIS, the world’s leading pulsed spallation source • An upgrade to 3MW has been approved for CD0 planning.
Communication Infrastructure is nearly complete • CD4 Fiber Backbone is complete • Tubes installed for future addition of “blown” fiber • “Global” cables nearing completion • Ethernet • Timing • MPS • All accelerator cable SROs entered • CISCO Switch firmware bug found (and fixed) Fiber Backbone Cable Infrastructure
All IOCs (including NADS) Multiple Fixed Rep Rates for RF Systems CLO Test Timing Network Infrastructure for Diagnostics, LLRF, XFD, and Power supply groups Line Synch monitor Delivered from BNL Setting up test network for XFD Chopper testing “General Time” added to EPICS Provides best time from multiple sources Assures monotonicity Accepted into “EPICS Core” SNS Event Link Monitor Event statistics Actual Events vs. Scheduled Events Timing System Installation is Keeping Up
Number of MPS Inputs has nearly Doubled(170 → 313) 170 • All linac MPS hardware installed – includes 27 new chassis • Copper phased out, using fiber everywhere • After CD4, corrector PS status will be added
Operations have moved to the CCR(Central Control Room in CLO) • Move took place in February as scheduled • Two console arcs are complete (ahead of schedule) • PPS and ODH systems moved (That was the worrisome part) • Software Development Area in use • All work planned for FY05 is complete • Servers are moved to “CER” (Central Equipment Room) • TPS also now in the CCR • Prototype Console moved to Target Control Room • Upgraded all multi-head OPIs • Faster processor • More RAM • Faster graphics adapter (Matrox G450) • X-windows and Java Apps have a much better performance PPS Equipment
Linac Controls Installation isKeeping Up with the Project Schedule • We have kept up (barely!!) with RF testing, RF Conditioning, PS testing, cryomodule cool-down, etc. • Medium-Beta SCL controls mostly complete • Cryomodule MB1 controls testing remains • Some “warm section” vacuum controls testing remains • Some MPS hardware installation remains • High-Beta SCL controls approaching completion • RF, Cryo, & Vacuum controls checkout are well along • Magnet power supply interface checkout has been integrated with overall power supply checkout • HEBT controls installation in progress • Automated procedures have helped keep schedule
Linac LLRF Installation is Keeping Up(This is the biggest task in terms of numbers and complexity) • Number of installed LLRF IOCs: 46 (up from 12) for 80 systems (out of a total of 96 systems) • Configuration automated via scripts, avoiding inconsistencies between SCL systems • From last ASAC: "Add Adaptive feed forward to firmware if required" • Was required for higher beam current • Not foreseen in current FCM, so successfully added to IOC software • recently refined to eliminate parameters that were hard to tune • Automation of RF ramp-up, warmup, loop closure • Allowed hands-off recovery from trips on the NC Linac • Needs adaptation for SCL • Waveform updates synchronized to timing system • Allows physics apps to compare data from several systems for the same beam pulse • Refining overview screens, giving better notice of trip reasons, active overrides, etc.
Cryogenic Control System nearly complete!! tested • All CHL Controls equipment installed and • All IOC and PLC programming complete • CHL Gas Management, Purifier and Warm Compressor • CHL 2.1K and 4.5K Cold Boxes • Medium and High Beta Cryomodules • HEBT Service Building IOC and PLC • 2K Cold Box Turn-on was automated • Cryomodules will complete in April Full RF Power First Liquid Cool Down Warm Up
The Cryogenic Control System layout 2 Servers 4 OPIs 11 IOCs 17 PLCs
Ring Controls is ahead of schedule:BNL effort completed in March • Handover Plan - Complete and in use (cooling water and ramping software scope moved to ORNL) • FSDs - Complete, in use and being updated as needed to keep them as-built • Design - Essentially complete. All drawings and specs needed are in use for fabrication and installation and being updated as needed to keep them as-built • Hardware Fabrication - WBS 1.9 work complete except for installing 1 PLC for cooling water skid. WBS 1.5 work complete except for 2 motion control racks, 2 PLCs for RF, and Vacuum racks • Procurement - All hardware procured except 1 PLC for water skid and some miscellaneous parts • Cabling - Schedule for Ring and RTBT cables depends on budget allocations • HEBT Communications cables pulled, being terminated • HEBT Process control cabling in process • Software - Vac, BLM, Mag PSC/PSI, HPRF, Beam Dump TCs ready for handover. All software ready for field testing by Jun 05 • Test Plans - Drafted for all systems except RF - Will be updated as needed during installation
PPS System Installation is keeping up(in spite of many configuration changes) • 75% of the PPS controlled devices in the Linac have been connected • The temporary Linac shielding has been removed and the Linac has been returned to one segment for phase 1.3 • PPS remote I/O racks for the HEBT are installed • Conduit & cable for PPS devices installed in HEBT tunnel • Installation of conduit & cable for PPS devices in Ring tunnel is 100% complete • The PPS control room racks have been relocated to the CLO CCR as a part of phase 1.3 (SCL commissioning)
The Target & Instrument PPS are alsothe responsibility of the Controls Team • Cable & conduit design for target building complete, installation underway using AIMSI crafts (beamline 2 and transfer service bay) and fixed price subcontractor (high bay) • Target PPS equipment design is 85% complete • Target PPS equipment fabrication is 70% complete • Equipment installation ramping up as electricians become available • Instrument PPS design for beamline 2 underway. Prototype user panel being fabricated
The number of IOCs has risen dramatically • 223 IOCs currently on Accelerator Network: • 109 VME / VXI --- vxWorks-based • 2 Linux-based “Soft IOC” Servers • Running 24 Soft IOCs (18 + 6) • 90 Intel-Pentium --- Win32-based IOCs (the “NADS”) • Wirescanners, Faraday Cups, BPMs, BCMs • Upgraded control system to EPICS R3.14.7 • Improved reliability in channel access. • Results in fewer white screens in the control room. • Upgraded OS for VME/VXI-based IOCs • vxWorks 5.5.1 (SNS CVS Release SNS07) • Better network driver and network stack configuration • Auto-retry on boot-up if boot server is unavailable. • Better memory management
Power Supply Group is Routinely UsingAutomated Testing and Calibration • Method uses EPICS and so assures “end-to-end” testing including control system • Resulted in large schedule reduction – one test instead of two • PS Calibration, current/field mapping and cycling profiles come from Oracle RDB • 9 SCL IOCs installed, 8 operational • 1 HEBT IOC installed and operational
Turn-on and Turn-off of HPRF Systemshas been automated State Diagram monitors the process for each HVCM Overall summary is displayed
The New Archive Engine is stable and amany-featured Java Archive Viewer is now in use • Archives now well organized with central configuration file and scripts for • generation of directory structure and skeleton files • start/stop/restart/monitoring of sampling engines • update of status web page • email notification of engine status and disk usage • Currently 11 distinct archives each running one or more sampling engines (total: 25). • (Conventional Facilities, Magnets, RF, Cryo, ...) • Data rate kept to ~2GB/day with occasional peaks at 4GB/day. • Limiting performance factor is currently neither CPU nor network but disk I/O. • New Java Viewer is stable and many features are being exploited • Uses XAL Framework • Web interface available • Plots actual data in various formats • Offers absolute and relative request times • Performs computations, exports data to spreadsheets • Investigating new uses via command line and web • Scroll, zoom, legends, titles, multiple plot styles, arithmetic and boolean operations
The new Archive Viewer (Chevtsov)has many useful features • Relative start/end times "-5 d(ays)" → "now" • Multiple value axes • Tool-tip info for individual data points • XAL Framework Support for calculations based on archived data Export to spreadsheets, progress bar Green squares in plot launch tool-tips for meta data, in this case daily restart of sampling engine.
Database Collaboration with APS has grown(SNS, APS, SLAC, TRIUMF, BESSY, FNAL,Diamond,PSI) • IRMIS: Integrated Relational Model for Installed Systems (v1.0 release in April) • Agreed “Core Schema” describes control system with three hierarchies: • Signal • Housing • Power • Uses the SNS Physics XAL “Framework” • Pelaia et al • RDB synchronized using “crawlers” • 106 SNS IOCs successfully “crawled” • EPICS versions 3.13.9, 3.14.6, 3.14.7 • 228,270 PVs found • SCL 112141 • DTL 33607 • CCL 27443 • MEBT 10918 • HEBT 8074 • “Crawlers” in development for all EPICS clients, IOCs, edm, Archiver, Alarms, etc.
Issues • Increased installation pace. • RF Processing and installation checkout means Control Systems Group must provide operational support. • Hardware/Software Configuration Management is highly correlated with reliability/availability. Work is in progress using a RDB, CVS, and good release control scheme.
Issues: How to manage increasing number of IOCS? • 223 IOCs currently on Accelerator Network: • 109 VME / VXI --- vxWorks-based • 2 Linux-based “Soft IOC” Servers • Running 24 Soft IOCs (18 + 6) • 90 Intel-Pentium --- Win32-based IOCs (the “NADS”) • Wirescanners, Faraday Cups, BPMs, BCMs • Upgraded control system to EPICS R3.14.7 • Improved reliability in channel access. • Results in fewer white screens in the control room. • Upgraded OS for VME/VXI-based IOCs • vxWorks 5.5.1 (SNS CVS Release SNS07) • Better network driver and network stack configuration • Auto-retry on boot-up if boot server is unavailable. • Better memory management
Summary • Installation is keeping pace • Required use of more automated configuration and testing, database use • PPS keeps pace notwithstanding changes • Cryo controls nearing completion • Will be ready for SCL commissioning • Ready to pick up on Ring controls • Operational Improvements as well • Automated HPRF turn-on • Automated 2K Cold Box turn-on • Improved Archive viewing tools