Andrew Ackerman NSLS II ESH August, 2018

1. Instrument Readiness Review at NSLS II 2018 Accelerator Safety Workshop; SLAC Andrew Ackerman NSLS II ESH August, 2018

2. Outline • Instruments (Beamlines) • Instrument Readiness Review process • Experience

3. Instruments 3

4. NSLS II • Accelerator Ready Review • ARRs • (FY14) • Commissioning: linac, booster, storage ring • Routine Ops • User Science

5. NSLS II Beamline (An Instrument)Integrated In-situ and Resonant Hard X-ray Studies (ISR) • ISR In Vacuum Undulator (IVU-23); long straight IVU-23 HFM 3 endstations DCM SSA DPP VFM DHRM KB

6. NSLS II BeamlineISR Instrument Layout Four hutches: two for optics, two for User data collection FOE Hutch C SOE Hutch D Beamline Layout cryo-cooler photon shutter photon shutter magnet diffracto-meter 6-circle diffracto-meter KB mirrors DHRM & SSA base diffracto-meter DCM HFM DPP VFM VFM VFM shielded beam transport shielded beam transport distance from center of straight (mm)

7. NSLS II Sources 3PW Assembly EPU57 EPU105

8. Beamline Hazard Analysis • Standard accelerator risks: • Electrical power • Fire • Pressure safety • Cryogens (LN2 & LHe) • Lead • Radiation • Bremsstrahlung – primary (ray trace) & secondary (Fluka); High energy, low power • Synchrotron – secondary (STAC 8); heat load (equipment & shield protection); low energy, high power

9. IRR Process 9

10. NSLS IIBeamline Instrument Readiness • Readiness “Systems and procedures established to address credited controls, documentation requirements, and personnel qualification“ ―Accelerator Safety Order • Ready to commission • Technical • Safe photon beam generation and delivery into the experiment endstation. • Science • Optimize performance • ‘Friendly’ Users • Beamline Instrument Readiness Review (IRR) – Technical Commissioning start • Point in time; Scope definition important Ready for User operations – separate process; procedure: review system • Beamline space – annual review; BNL electronic system • User science – beam time cycle review; another BNL electronic system

11. Building a BeamlineAn Extended Project (years) • Design & Plan • Science • Hazard control • Procure • Install • Travelers – checklists – meet specs • Integrated testing (no beam) • Assign and qualify staff • Role • Procedures • Instrument Readiness Review • ~ 6 weeks Getting Ready • Instrument Readiness Plan • Organize • Define: • Scope • Basis • System specific • Actions & documents • ~40 criteria • Status tracking • Declare ready – signatures • Lines of Inquiry (LOI) for the review IRP

12. Instrument Readiness Plan (IRP) CriteriaDesign and Plan • Functional description (scope definition) • Design • Source, Front End, & Beamline components • Radiation Safety components • Top-Off Safety System (TOSS) • Bremsstrahlung & Synchrotron Ray traces • Secondary radiation scatter analysis (FLUKA & STAC 8) • Plan • NEPA evaluation • Un-reviewed Safety Issue (USI) resolution • Legacy open action item resolution • Instrument specific procedures • Commissioning plan • Radiation survey plan

13. Instrument Readiness Plan (IRP) CriteriaInstall and test • Radiation safety components • Shielding, masks, labyrinths, shutters … • Area Radiation Monitors (ARMs) • Personnel Protection System (PPS) interlocks • O2 Sensors • Hutch structures (life safety) • Electrical power distribution • Utilities (water, LN2, LHe) • Beam transport components • Equipment Protection System (EPS) • Controls • Vacuum Safety Critical Other

14. Instrument Readiness Plan (IRP) CriteriaAssign and qualify staff • Assign roles and Job Training Assessments • Lead Beamline Scientist • Authorized Beamline Staff • Cognizant Space Manager • Beamline Support Staff • Assign procedure training • Read and sign • Demonstration

15. Review Team Charge • 7 person team – BNL & NSLS II • Web page • Team collects information: • Interview staff • Read documents • Inspect the instrument • Team provides: • Review details (staff, document, topics) • Observations (noteworthy practices; suggested improvements) • Findings (pre & post) (tracked to closure) • “Recommended” Corrective Actions Pre-start means (Required for safe beam operation)

16. Experience 16

17. Beamline IRRs • 09/14: CSX 1&2 • XPD • 10/14: HXNSRX • IXSCHX • FY 16 • 10/15: LIX • 03/16: FMX/AMX • 04/16: ISS • 06/16: ISR • XFP • ESM • 08/16: TES • CMS • FY 17 • 10/16: SMI • 11/16: SIX • NYX • 07/17: R&D • BMM • 09/17: XFM • QAS • FY 18 • 10/17: FXI • 02/18 SST 1&2 • 03/18: PDF 27 Total 8/07/18

18. Subscription Rates (2018-1) 2.4 2.3 2.8 2.0 1.0 2.3 1.4 1.2 1.5 1.4 1.1 1.0 1.0 1.2 1.8 1.0 2.0 3.3 1.9 Proposal subscription rate = proposals submitted / proposals allocated

19. NSLS-II Facility Users (as of April 24, 2018) FY15 • unique users: 110 • 100% were first-time users FY16 • unique users: 477 • 83% were first-time users FY17 • unique users: 1037 • 69% were first-time users FY18* (as of 24-April-2018) • unique users: 987 • 48% were first-time users Users

20. NSLS-II Publications (as of May 10, 2018)

21. ExperienceNSLS II Beamline IRR The IRR is a big effort (Beamline and Accelerator staff) • Intended • Significant risks (personnel and project) • $10 – 15M instrument • High expectations – Users are coming • NOT an agile process – 6 to 8 weeks preparation • Weekly status meetings (needed, but not popular) ~ 45 managers / engineers / scientists • Varied documents to complete, file, and post on web (some) • ~ 40 criteria – signatures • Report and follow up

22. ExperienceNSLS II Beamline IRR It is hard to be ready on time • Must schedule a date in advance – deadline • Project deliverables / milestones • Best = source to end-station • In practice – Source & FE ready first – split into two IRRs • Science wants review once something is ready (adjust scope) • Biggest challenges • Vendor supplied items • Lead shielding components • Optics • PPS design / installation • Radiation safety analysis (ray traces and scatter) • All documentation (one week in advance) • Ray traces • Final layout capturing all radiation safety components • Mechanical survey reports • Discrepancy reports • Adjust scope to meet date: something is ready • Self-identified pre-start – we’re almost ready

23. ExperienceNSLS II IRR Readiness brings resources that leave after the review. Strategy • Clear IRP criteria definition = difficult; important • Multiple instruments reviewed together = stress installation resources • Review team selection • NSLS II / BNL • BNL SMEs; NSLS II scientist, ESH, QA • Added BNL review team leader (2017); new eyes and perspective • PPS, EPS, Controls – unique for reviewers • Review meeting time? ~ 1.5 days / instrument • Walkthrough is productive – seeing stimulates conversation • Tailored Review Plan: • IRR threshold? Equipment Readiness Review (ERR)?

24. ExperienceNSLS II Beamline IRR Findings • Configuration control • Labeling, photos, checklist • Utility mechanical support and labeling • Bump and trip hazards • Documentation for pressure testing. Utilities, windows (Kapton, diamond, Be) • Hutch Personnel Protection System (PPS) hardware field location • Forced search buttons locations & emergency stops • Installation sequence forces early decisions

25. END