190 likes | 329 Views
Changes in Powering Interlocks I.Romera , TE-MPE. Thanks to: P.Dahlen, S.Gunther, R.Mompo, S.Ravat, H.Thiesen, M.Zerlauth. MPP Workshop – 11 th -13 th March 2013. Outline. Operation with FMCMs post LS1 Late dump detection by magnet powering systems
E N D
Changes in Powering Interlocks I.Romera, TE-MPE Thanks to: P.Dahlen, S.Gunther, R.Mompo, S.Ravat, H.Thiesen, M.Zerlauth MPP Workshop – 11th-13th March 2013
Outline Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS
Sensitivity to electrical perturbations 2012 FMCM dump cause Likelihood to trip Systems affected by EL perturbation Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS Sensitivity to electrical perturbations Proposed mitigations Electrical network glitches are one of the main dump causes from magnet powering systems 24 FMCM triggers last year due to glitches in the electrical network distribution A large fraction do not result in equipment trips and only seen by FMCMs! High sensitivity (<3x10e-4) on FMCMs is required to avoid effects on circulating beams Being RD1 and RD34 most sensitivecircuits which tripped in most of the cases RD1.LR5 is about 40% more sensitive than RD1.LR1 despite similar input amplitude
Mitigations put in place in 2012 Effect of 100mA current change on orbit in the arc Courtesy of T.Baer Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS Sensitivity to electrical perturbations Mitigations in place in 2012 Post L1 operation FMCM review carried out to study possible mitigations to the sensitivity problem [1] Thresholds relaxed by factor 3 on RBXWTV and RD34 still maintaining specification values[2] Simulations on RD1 shown that there is no margin for threshold change No significant availability improvement seen after mitigations as always in the shadow of RD1 Other mitigations were studied: Improving regulation characteristics of the PC, changing circuit impedance by adding SC solenoid and replacing thyristor-based by switch mode PC…
Post LS1 operation Exceed factor in FMCMs EL perturbations where only FMCM tripped (2012) Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS Sensitivity to electrical perturbations Mitigations in place in 2012 Post LS1 operation EPC will soon initiate a collaboration with EPFL to better understand the effects of such glitches After tests started in 2012, EPC is confident to provide a better rejection of elec. perturbations by only changing the regulation characteristics of the RD1 and RD34 power converters We expect that… ~40% of minor perturbation trips will be avoided in the future…
Late dump by CMS solenoid trip - 10.08.2012 ~7% beam loss in B2, almost nothing in B1 400um in B1H 200um in B1V 200um in B2H I_MEAS Trip of power converter V_MEAS 100um in B2V Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS CMS solenoid trip due to a cooling problem which had an effect on beams [3] Beams dumped only when I_MEAS at 40% of nominal value (3 minutes after solenoid trip) Orbit deviation up to 400um seen by interlock BPMs Very slow and distributed losses not reaching BLM thresholds Beams dumped on high voltage BLM interlock (via the SIS) Experimental magnets LHC Power Converters 60A orbit correctors
Late dump by LHCb trip - 19.08.2012 Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS LHCb trip led to slight beam orbit deviations before beam dump [3] Orbit changed by 20um during the last 200 turns Magnet Safety System (MSS) takes 30ms to send request to Beam Interlock System Experimental magnets LHC Power Converters 60A orbit correctors
Foreseen changes to Magnet Safety System Power ConverterFault Power ConverterFault 13 ms 13 ms Optocouplers Optocouplers 1 ms 1 ms MSS MSS2 FPGA Processing cRIO FPGA Processing < 1 ms < 1 ms Safetyrelays Optocouplers 15 ms 1 ms Total delay ≈ 16ms CIBU CIBU Total delay ≈ 30ms Courtesy of S.Ravat Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS MSS upgrade to MSS2: NI cRIO FPGA platform to replace obsolete fuse programmable FPGA Interlock will be provided in case of a Fast Discharge (CMS trip…) Optocoupler will replace the output safety relay and time to interlock reduced to 16ms Experimental magnets LHC Power Converters 60A orbit correctors
Late dumps by LHC Power Converters RTQX1.L2 I_OUT OVER CURRENT I_A I_MEAS I_REF V_REF Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS 2 events last year on inner triplet power converters On 15.06.2012 beams dumped due to high beam losses in SR7 [4] A broken diode inside RTQX2.L2 provoked voltage oscillations which put in conduction the FWD in parallel with Q3 magnet Large circulating currents between the 3 converters and RTQX1.L2 tripped by I_OUT_OVER CURRENT only 300ms after first current excursions observed Better sensitivity achieved by reducing I_ERR and I_POS limits in the FGC Experimental magnets LHC Power Converters 60A orbit correctors
Late dumps by LHC Power Converters Operation with FMCMs post LS1 Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS On 07.09.2012 beams were dumped on beam losses due to trip of RTQX2.R5 Interlock was not sent to the PIC due to radiation induced latch-up PIC interlock only received ~2 seconds after the dump by VS_FAULT on RTQX1 During LS1 the 2s FGC watchdog will be removed as part of FGC2 CPLD upgrade! Experimental magnets LHC Power Converters 60A orbit correctors
Late dump by loss of 60 Powering Permit FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS No hardware signals between Power Converters and interlock system Magnetprotection ensured byPower Converters and interlocks just to avoid unnecessary quenches of magnets and current leads and to help operations On 25.10.2012 communication problem with PIC => BLMs triggered in SR7 [5] Experimental magnets LHC Power Converter interlocks 60A orbit corrector interlocks
Late dump by loss of 60 Powering Permit What we saw… Logic implemented… FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS PC_PERMIT_60A would have never been lost with the correct PVSS logic due to a communication problem affecting a single PIC… PVSS logic will be corrected during LS1 Experimental magnets LHC Power Converter interlocks 60A orbit corrector interlocks
Renovation of interlocks in SPS FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS Renovation of SPS interlocks Changes in Access Powering Operational improvements Last June a problem with ring-line interlocks in SPS caused 16 hours of downtime! [6] Existing magnet protection system relies on 3 interlock systems (dating from 70’s) Spares for ring-line and auxiliary lines but no spares available for main interlocks Consolidation works done during 2012 and full renovation will be done during LS1 [7] Electromagnetic solution to be replaced by standard WIC interlock system
Changes in Access vs Powering Courtesy of L.Ponce – see Chamonix 2010 Courtesy of J.Wenninger FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS Renovation of SPS interlocks Changes in Access Powering Operational improvements Software-based solution put in place after the incident in September 2008 A more dependable implementation will replace the existing one during LS1 [8] New PIC-PLC to be installed by the LHC Access Control System (LACS) in the CCR Communication between LACS and PIC-PLC via hardwired signals PIC-PLC will publish access status to CMW and SIS will subscribe to the data
Operational improvements FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS Renovation of SPS interlocks Changes in Access Powering Operational improvements Protection mechanisms to avoid quench propagation to neighboring magnets During HWC campaigns such global protection is a bottleneck for testing! Proposal is to make the Global protection mechanisms maskable via PVSS Masking status clearly visible from PIC SCADA system
Operational improvements FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS Renovation of SPS interlocks Changes in Access Powering Operational improvements • To allow for some flexibility, not all circuit trips will directly request beam dumps: • RCD, RCO, ROD, ROF, RQS, RSS…do NOT directly dump the beam • Review of critical circuits for operation in view of new ATS optics… • In 2012, quadrupole correctors RQSX3 were included in maskable configuration
And some additional changes… FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS Renovation of SPS interlocks Changes in Access Powering Operational improvements • PICs: • 25% of the controllers to be relocated within R2E project (UL14/16 &UL557) [9] • Re-commissioning of UPSlinks following UPS renovation (see V.Chareyre talk) • HTS room temperature interlocks in RR53 routed through the PIC • FMCM: • FMCM in UJ56 relocated to USC55 within R2E project [10] • Controls renovation in SPS transfer lines (removal of obsolete ROCS-MUGEF) WIC: • Ti2/Ti8 - Installation of 800 interlock boxes to allow remote testing of magnet interlocks
Summary FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS • High sensitivity of FMCMs against electrical network perturbations: • Thresholds relaxed on RBXWTV and RD34 during 2012 • Voltage regulation improvements on the power convertersduring LS1 • Redundant protection to the BLMs will be restored after LS1: • PC_PERMIT_60A logic will be corrected in PVSS • Upgradeon Magnet Safety System • FGC2 CPLD upgrade to remove the 2s watchdog in case of FGC crash • No SEUs observed on PICs last year, thanks to temporary relocation in 2011 • SPS magnet interlocks were consolidated and will be fully renovated during LS1 • More dependable approach for the Access Powering interlocks
Thank you for your attention! FMCM operation Late dump detection by magnet powering systems Foreseen changes to powering interlocks of the LHC-SPS • References: • [1] FMCM review, TE-Technical meeting • [2] ECR-Change of Protection Threshold for FMCMs installed on the ALICE compensators and the twin aperture warm separation circuits, EDMS 1213337 • [3] 67 - MPP meeting, Minutes • [4] 63 - MPP meeting, Minutes • [5] 75 - MPP meeting , Minutes • [6] Problem with SPS interlock crate, MPE-COMS-167 • [7] TE-MPE-Technical meeting, Slides • [8] ECR-Change of the Interlocking of Powering and Access systems, EDMS 1246780 • [9] ECR-Activités R2E au point 1 durantLS1, EDMS 1263553 • [10] ECR-Activités R2E au point 5 durant LS1, EDMS 1263595