Overview of Software Developments in J-PARC / MLF

1. Overview of Software Developments in J-PARC / MLF T. Otomo J-PARC center / KEK

2. J-PARC Facility Hadron Beam Facility Materials and Life Science Experimental Facility Nuclear Transmutation 500 m Neutrino to Kamiokande 3 GeV Synchrotron (25 Hz, 1MW) 50 GeV Synchrotron (0.75 MW) Linac (350m) J-PARC = Japan Proton Accelerator Research Complex Joint Project between KEK and JAEA (former JAERI)

3. Materials & Life Sceince Experimental Facility Neutron Beam Lines (23 total) Neutron Beam Lines (23 total) Target Station Scattered Neutrons Experimental Devices Proton Beam Calculated (about 13 meV) Neutron intensity Neutron Scattering Area 6Li - 7Li (Background) 5/30/2008 14:25 Muon Experimental Area Time-of-flight for neutrons (ms) Proton Beam

4. Beam Lines under construction commissioning 7 construction 5 Fundamental physics BL05 BL04 Neutron-nucleus reaction instrument BL08 BL03 High resolution powder Ibaraki biological diffractometer BL12 BL01 High resolution chopper High Intensity Chopper BL14 BL21 Cold Neutron Disk-Chopper Spectrometer Total diffractometer BL20 Ibaraki materials design diffractometer BL16 BL19 Horizontal reflectometer Residual stress

5. Basic computing requirements at MLF • High throughput of large data analysis • On-line analysis of GB order data during an experiment • Interpretation software (simulation) will be used on an experiment • Data analysis affect on effective flux of neutron (muon) • Fast and reliable data analysis and experiment systems are required • Variety of experimental approaches • Extreme experiments will become conventional • Flexible instrument control • Naïve experimental conditions • “Computer Assisted Experiment” • Large number of user • Several ten thousands of cumulativeusers / year • Database for user program should be implemented • Computing environment should be user-friendly • Security • User identification and authorization are essential to enable flexible access to MLF/J-PARC • Collaboratory system will open new style of experiment • Computing environment is one of essential infrastructures of MLF

6. Software components to be constructed (ver.1) : Under research DAQ Devices Logging Remote access Network Security Collaboratory Working desktop (Python) Simulation Analysis Visualization Experiment Ab Initio Monte-Carlo etc Analysis GnuPlot IRIS exp, Matlab, etc Reduction Data File Admin DB Experiment DB SAN / Grid Storage

7. Software Developments • Analysis (Data reduction) framework (2003-) • Manyo-lib • “Working Desktop” design (whole MLF framework) (2003-2007) • Use case (> 200 cases), data-flow-diagram • DAQ software with LabVIEW (2004-2006) • Modified KEK-KENS DAQ to network-based control system • DAQ framework (2006-) • DAQ middleware • Beam line specific software(2006-) • Powder, Chopper, Single crystal, etc. • Visualization package(2007-) • Viewer of Manyo-lib data-container

8. Software Developments (contd.) • “Working Desktop” developments(2007-) • Client-Server model (XML/http), GUI, data I/O • Device control • Commissioning (1) on Day-1 (May, 2008) • Starting from BL20 (Type: Powder diff, Detector:3He, Data: event mode) • Commissioning (2) (June, 2008) • On-line histgramming from event data • Off-line histgramming • I(d), S(Q,E) • Commissioning (3) (Dec., 2008) • DAQ + Analysis + Gonio control

9. MLF Computing Environment “Manyo-lib” “DAQ middleware” Common Library Common Library Data File API Library Instruments specific Lib. Device control NeXus File Powder Diff. Single Xtal DAQ electronics drivers Chopper XML File “Working Desktop” etc. Res. Stress Common User Interface DataBase Visualization DAQ Script Common User Interface Common User Interface Analysis script DAQ Log 2D 3D >3D Analysis Log DAQ specified network interface “SiTCP” Standard data format for neutron, x-ray, and muon science Simulation Software Common framework for neutron data analysis

10. Framework component (1) • “Working Desktop” • Implemented by Python • User interface • CUI and GUI • Interface to software component • Help to Edit XML files • Parameters of Devices, Experimental condition, Data plotting, Analysis • Execute experimental sequence • Support network distributed environment • Generate XML and send/receive with http

12. Framework components (2) • DAQ (Experiment component) • Introducing “DAQ-middle ware” • Network-based device control middleware • Event mode DAQ • Control of “standard” devices • NeXus • Analysis • Data reduction library = “Manyo-lib” • C++ framework; it can work as an application • Wrapped into Python • Standardized units system will be used • Hieratical data structure • Data Container (internal data structure) with STL • Both components works w/o the framework (working desktop)

13. DAQ system Implemented by Manyo-lib: easily modified for experiment purpose Software framework Working desktop XML/HTTP Begin Configure End Status Begin End Web Server DAQ Operator Device manager DAQ middleware XML/HTTP DAQ Status Logger NEUNET Dispatcher Gatherer Monitor SiTCP SiTCP SiTCP SiTCP SiTCP SiTCP SiTCP

14. Event data acquisition with Ethernet Time, Position Time, Position Time, Position CPU UI Time, Position Time, Position Time, Temperature etc. CPU DAQs DAQ electronics Detectors Network Storage Sample environment, Moderator, etc.

15. Analysis / Visualization system Software framework Working desktop Histograming XML/HTTP Smoothing EventToHistogram Plot Smoothing Analysis manager Web Server XML/HTTP MPlot etc. MPlot Manyo-Lib Common module Smoothing etc. User module EventToHistogram He-PSD, Scintillator

16. Manyo-lib can be directly used by Python • C++ (Manyo-lib) wrapped into Python by SWIG

17. DataBase for meta-data storage R&D Chain Management system (Quatre-i Science Ltd. ) Prototype will be developed in 2009

18. Software commissioning • He detector system • BL20 at Day-1 (May, 2008) • BL01 (June, 2008) • BL14 (Jan., 2009) • BL08, BL21, BL12 (Spring in 2009) • BL15 • ZnS scintillater system • BL19 (June, 2008) • BL03 in FY2008 • MPGC system • BL16 in FY2009

19. DAQ control and online histogram Framework (Skelton) （EngDevManageSkel.py） “Begin”&”End” Command (Stub) (EngDaqBegin.py, EngDaqEnd.py) Pixel vs. Intensity TOF vs. Intensity Online histogram DAQ-Middleware copy to File Server DAQ middleware + Manyo-lib

20. Difficulties of software collaboration between BLs • Differences between BLs • Different time schedule and manpower resources • Different experimental setup • Different scattering function • Different unit system • Different scientific objections

21. Feasible way of collaboration between BLs • Lower Level software (DAQ, Data reduction, etc.) can be standardized easily • Lower Level: Near level to hardware and/or raw data • DAQ middleware, Manyo-lib • Grouping of BLs • Similar BL can easily share software resources • Design of interface between common (framework) and beam line specific parts • Framework need to absorb these difference • Language & Library • Python, C++, STL • SWIG, wxPython, GSL • Project server • Wiki, SVN, doxygen

22. Grouping of Beam Lines

23. Maintenance of spectrometers Determination of condition and schedules … Device Control Sample setting Measurement Control of spectrometer condition and scheduling Data Reduction and Visualization Required Physical quantities Informative visualization From View point of users Decision making and optimization 実験条件の検討 Sequence control Crystal Alignment Data Reduction and Visualization

24. Visualization Element Container Viewer : sample pictures Controlled by commands on Manyo-lib Manyo Lib (python) + wxPython + matplotlib

25. 2D Plotting : sample picture Visualization Manyo Lib (python) + wxPython + matplotlib enough speed on laptop PC

26. Sequence Control

27. Users of Powder diffractometers We absolutely welcome all users! • In J-PARC, the group of diffractometers welcomes new users and beginners • who come from industrial companies and academic fields • who have no experience with crystal structure refinements

28. Integrated Analysis Environments for Powder diffractometer Intensity Database Data Processing Structure Search Suite Peak Search Suite Input Structure model to Rietveld Conventional Rietveld Method SDPD Suite Indexing Ab initio Structure Determination Evaluation (BVS, Chem. Bond, Crystallography) Constrains RMC/Rietveld, MEM/Rietveld, First Principal Calc. After Rietveld Suite Profile Analysis Suite (for engineering diffractions) Texture Analysis Suite (for engineering diffractions) Parametric Diffractometory Rietveld Plus + Visualization + GUI Suite

29. Rietveld refinement 3 different histograms were refined using a crystal model at same time. 155 deg 90 deg 30 deg

30. Data processing part GUI of the data processing part Designed based on the algorithms of ISAW. (http://www.pns.anl.gov/compution/isaw/index.shtml) blue Additional functions mainly for macromolecules which have large unit cells. green

31. Visualization 2D Plotting : Bragg peaks of single crystal x-y slice map TOF profile x-y slice map and TOF profile with predicted Bragg spots

32. summary • Software commissioning is on-going • Event data acquisition and on-line histogramming are available • Off-line analysis software are testing • Integration of software components, especially DAQ and device control will be done by March 2009 • Ver 1.0 • Prototype of Database of meta-data will be developed in 2009 • Advanced analysis (interpretation) components will be developed • May be under international collaboration