LUSI Controls and Data Systems: Integration and Implementation

1. LUSIControls and Data SystemsW.B.S. 1.6 Gunther Haller Project Manager July 15, 2009 Plenary Presentation

2. Controls W.B.S. Scope Near Experimental Hall X-ray Transport Far Experimental Hall 1 3 2 5 4 6 XCSMono SXR AMO XPP XCS CXI H6 Installation Part of LCLS Part of LCLS MEC ARRA Funds • Separate WBS 1.6 to combine all LUSI control & data needs due to commonality in requirements, design, implementation, installation and integration • XPP, CXI, XCS, Diagnostics & Common Optics • Common control and data systems design for LUSI and rest of photon beam-line instruments (AMO, SXR, FES)

3. Scope – WBS 1.6 Control & Data Systems • Included in W.B.S. 1.6 • All controls & DAQ, labor and M&S, for XPP, CXI, XCS instrument components and diagnostics/common optics • Includes controllers, racks, cables, switches, commissioning • Data-storage and processing for FEH instruments • Input-signals to LCLS machine protection system link-node modules • Provided by LCLS X-Ray End Station controls (CAM is G. Haller) • Personnel protection system • Machine protection system (LCLS modules, fibers) • Laser safety system • Accelerator timing • Femto-second laser timing • Network architecture & security • Data-storage and processing for NEH • User safeguards • Laser controls • CXI 2D detector controls • Interfaces described in • 1.1-517 ICD between XES and LUSI (released document)

4. 1.6 WBS to Level 4 Example XPP

5. ESD’s and ICD’s • Two types of controls documents required for each instrument • Engineering Specification Documents (ESD’s) • Interface Control Documents (ICD’s) • XPP • SP-391-001-21 XPP Controls ESD • SP-391-001-22 XPP Controls & DAQ ICD • SP-391-001-23 XPP DAQ ESD • CXI • SP-391-001-13 CXI Controls ESD • SP-391-001-14 CXI Controls & DAQ ICD • SP-391-001-18 CXI DAQ ESD • XCS • SP-391-001-24 XCS Controls ESD • SP-391-001-25 XCS Controls & DAQ ICD • SP-391-001-26 XCS DAQ ESD • Diagnostics • SP-391-001-19 LUSI Common Diagnostics & Optics ESD • All documents released at • http://confluence.slac.stanford.edu/display/PCDS/LUSI+Document+Page

6. Reviews • Process Flow for Reviews • Instrument Preliminary Design Review • Controls Preliminary Design Review • Instrument Final Design Review • Controls Final Design Review • Preliminary Design Reviews (PDR) for Instrument Controls and Data Systems held after instrument PDR and before the Instrument FIDR • XPP Controls and Common Diagnostics PDR’s • Held February 7, 2009 • http://confluence.slac.stanford.edu/display/PCDS/XPP+Preliminary+Design+Review • CXI and XCS PDR’s • Held May 11, 2009 • http://confluence.slac.stanford.edu/display/PCDS/CXI_XCS-PDR • Instrument FIDR’s have recently been completed • Final Design Reviews (FDR) for Instrument Controls and Data Systems planned for early August 09 • XRay End-Station (XES) Reviews under LCLS XES, not part of LUSI • Common services: e.g. Networking, DAQ, PPS, LSS, MPS • http://confluence.slac.stanford.edu/display/PCDS/XES-Review-Sept-4-08

7. Controls and Data Sub-System • Controls • Software: EPICS/Python/Qt • Vacuum • Motion • Viewing • Power Supplies • Racks and Cabling • Other items • Data Systems • Challenge is to perform data-correction and image processing while keeping up with continuous incoming data-streams • LUSI benefits that SLAC Particle Physics and Astro-Physics group is involved which has substantial experience acquiring, processing, and archiving large data volumes at high rates • Use common dataflow/processing/storage & offline interface DAQ for instrument components in the real-time detector data chain (BNL & Cornell 2-D detectors, future SXR detector, waveform sampler, etc) • Minimizes development, production, commissioning, and maintenance effort

8. Data System Architecture Instrument specific Photon Control Data Systems (PCDS)‏ Beam Line Data L1: Acquisition Digitizers, Cameras, 2D Detectors L2: Processing Timing L0: Control L3: Data Cache To SCCS Offline • Level 0: Control • Run & configuration control • Run & telemetry monitoring • Level 2: Processing • Pattern recognition, sort, classify, alignment, reconstruction • Level 3: Online Archiving • NEH/FEH local data-cache • Local cache can buffer up to 4-days worth of data • Offline will transport data to tape staging area in SCCS Computer Center • Level 1: Acquisition • Image acquisition, calibration • Event-building with beam-line data • Correction using calibration constants • Data reduction (vetoing, compression)

9. Example 2D-Detector Control and DAQ Chain Fiber ATCA crate with SLAC DAQ boards, e.g. the SLAC Reconfigurable Cluster Element Module Cornell CXI 2D detector-ASIC SLAC FPGA front-end board Brookhaven XPP/XCS 2D detector-ASIC Beamline Instrument Detectors • ATCA • Advanced Telecommunication Computing Architecture • Based on backplane serial communication fabric, 10-G E • 2 SLAC custom boards (also used in other SLAC experiments) • 8 x 2.5 Gbit/sec links to detector modules • Dataflow and processing • Managed 24-port 10-G Ethernet switching • Essentially 480 Gbit/sec switch capacity • Naturally scalable Main LUSI instrument custom integrated circuits from Cornell & Brookhaven are already connected at SLAC to SLAC LCLS high-performance DAQ system

10. Controls: Cost Total Budget: $7,380,671 Engineering $3,032,110 Proc/Fab/Assy/Instl: $4,348,561 * Within 5% of plan

11. Recommendations from Last Review • Recommendation • We recommend regularly scheduled meetings between the beamline scientists, diagnostics physicists and control system design engineers. • LUSI agrees and the project will schedule regular meetings including the beamline scientists, diagnostics physicists and control system design engineers. • Action • Weekly meetings are occurring with the Cornell detector group for mechanical and electronics integration • Weekly XPP meetings have started after the XPP FIDR, four months ago. • Controls invited to CXI and XCS instrument meetings, attend when relevant topics are put on agenda • Start dedicated controls meetings late August 09

12. Risks • Risks • Late changes to design due to evolving user requirements • Mitigation • Adhere to BCR process. • Participate in Experimental Area design process • Release and update Engineering Requirement documents

13. Safety • Safety • Laser Safety and Hutch Protection Systems provided by LCLS XES, hutch 3 (XPP) is the third hutch to be operated • Same for User Safeguards (oxygen deficiency monitor, radiological sources) • Same for Electrical Safety • All cables/equipment are rated for their use • All equipment will be NRTL listed or inspected and approved under SLAC's Electrical Equipment Inspection Program • Robot safety in progress

14. Overall Status • Control and data systems hardware and software components to be provided are agreed on and documented • XPP controls & data systems items are being ordered • No long lead items or expensive (> 100k$) orders • Started data processing coding for XPP, e.g. • Electronics gain correction, dark image correction, flat field correction • Event filtering (veto based on beam-line-data, sparcification by predefined region-of-interest) • Event binning (e.g. sorting, averaging) • Following services required by XPP are already in place in hutch 3 or soon will be in place, months before required by XPP (provided by LCLS X-Ray Endstation Systems) • Hutch Protection System, Laser Safety System, User Safeguards • Machine Protection System • Network • Timing (accelerator as well as femto-second laser) • Racks including AC connections and cooling • Data processing and storage • Offsite data access and transport

15. Summary • Interface and requirements documents released • Clear what needs to be done • No issues, design meets requirements • XPP, CXI, XCS controls preliminary design reviews already completed • Most items similar to XTOD and AMO which both already had Final Design Reviews for Controls and Data Systems (XTOD is being commissioned, AMO will follow in August 09) • Design mature • Design complete in respect to what will be used and quantity of items to be ordered • Many items are already used (hardware and software) in XTOD and AMO, ahead of XPP, XCS, CXI • No technical risk • Started ordering XPP control and data system items • Team • Engineers and technicians from PPA Research Engineering Group, sufficient man-power available for LUSI • Details in breakout session