SNS/HFIR Software Developments

1. SNS/HFIR Software Developments Steve Miller Analysis Software August 15, 2006

2. Organizational Structure – October 1, 2006

3. SNS Recent Milestones Achieved • SNS Facility CD-4: First beam on Target – April 28 • Backscattering Spectrometer first data – May 19 • Reflectometers 4A and 4B first data – July 19

4. SNS Facility CD-4 Measurements Accelerator/Target Controls BL-7 Data

5. BSS Data Reduction – Diffraction Bank (Mica) Instrument Geometry Step 1: pathlength Step 2: angle Courtesy of Michael Reuter

6. BSS Inelastic Data 4-methyl pyridine N-oxide 5 kWatt, 3 hour, ¼ current detector/analyzer, T = 3 K Tunneling Peaks Visible! Courtesy of Ken Herwig and Eugene Mamontov Fit using DAVE Software

7. Reflectometer 4B Incident Beam Wavelength Spectrum Courtesy of John Ankner

8. First measurement on BL4A July 21, 2006 Sample: Ni 50 Å / Ti 50 Å multilayer SNS source power: 250 W Data collection time: 2.5 h Preparation time needed: 8 years TOF spectrum: integrated over whole detector Position spectrum: integrated over all TOF channels Same sample measured at IPNS: Bragg Peak Neutrons reflected from sample at 1 deg. grazing incidence Courtesy of Frank Klose

9. Magnetic Reflectometer Reflectivity • 50 A Ni / 50 A Ti * 20 Multilayer • 2 kw – low power

10. Instrument Commissioning Challenges • Low accelerator repetition rates: • Design point 60Hz, current rates between 1 to 10 Hz • Results in long time intervals between pulses on target • Keeping all neutrons with fast (~25 uSec) sample rate gives a large number of TOF channels – 5000 to 10000 typical • Reflectometry histogram files on the order of ~1.5GB each • Currently only able to perform linear histogramming • Low beam power • Measurements contend with background • Must acquire for longer periods of time • Must contend with occasional accelerator glitches

11. Creating, Processing, and Storing Data • Event Histogramming • Detector to Pixel mapping • Instrument Geometry • Metadata extraction • Create NeXus file • Catalog and Store • Reduce Data • All subsystems functional to some degree

12. Data Reduction Architecture Overview Three levels of reduction at SNS • Level 1: Driver is the overall mechanism that runs the data reduction process. It based on the requirements that are given by the instrument scientists. • Level 2: HLR is the representation of functions. It unifies calls to retrieve data and to call low level functions. Levels 1 and 2 python. • Level 3: (mostly C++) • DOM provides abstract layer for data manipulation. • SCL is a toolbox of reusable primitive functions necessary for data reduction process.

13. Visualizing Data via the Portal MCA Data ISAW Plot NeXus tags NeXus Files metadata

14. Metadata Search via ICAT Search String Optional Search Fields

15. Networking Diagram

16. Instrument Commissioning Lessons Learned • Detector and Instrument Definitions… • Non-trivial to determine • May need an iterative approach to derive calibration factors • Using NeXus… • Adds overhead for examining raw data • Question about how best to associate updated calibration data with existing NeXus files. Note that data reduction can take an optional calibration file along with NeXus data. • Create NeXus files during acquisition or “on-demand”? • Event Data • Had to adapt to provide histogramming tools sooner than originally anticipated • Detector Mapping • Needed to create and validate detector to pixel mapping files

17. Problems and Issues Encountered • Python memory management issue: • Deleted ints and floats still remain in heap memory, thus memory footprint can grow very large. 6GB not uncommon for us and “larger data” to come. • Causing us to rethink how best to utilize python for memory intensive applications • Memory de-allocation fix in python 2.5 does not address this problem • Does DANSE team have experience with this? • Need for tools to explore “pre-NeXus” data • Responded by creating prototype IDL tools • More tools needed, such as for producing rocking curves • Need more tools for working with event data

18. Pre-NeXus Data Examination Tools • Instrument Scientists need tools to explore data. • Currently producing prototype IDL tools to enable exploring and extracting data. • Considering producing tools derived from this which will integrate with the portal. • Using IDL VM to distribute applications Courtesy of Jean Bilheux

19. Instrument Commissioning Pattern Emerging • Analysis Software group provides: • the data “plumbing” • Data reduction • Visualization tools • Utilizing existing analysis software tools (DAVE, ReflPak, ISAW, GSAS, etc.) • The DANSE team can dramatically help SNS/HFIR: • Identify prototype instrument commissioning software tools • Identify and produce Instrument Day-1 analysis software • Keep an eye to the future developing portal-based advanced analysis software • Urge DANSE Science team leaders to continue to work closely with ORNL instrument teams and the Analysis SW group!

20. Instrument Schedule

21. Instrument Commissioning Schedule by year: • Ongoing: • Backscattering Spectrometer • Liquids Reflectometer • Magnetic Reflectometer • 2007 • ARCS • Powder Diffraction • CNCS • 2008 • SANS • SNAP • Sequoia • 2009 • Vulcan Engineering Diffraction • Spin Echo • TOPAZ SCD • Take Home Message: • We should anticipate the desire to bring instruments on-line sooner in order to broaden the scope of the user program. • Urge DANSE team members to continue to work closely with corresponding SNS and HFIR personnel (and vice versa). • With higher neutron flux in later years, the push will be for “first science”, not just “first data”. • Acting now can enable world class software to be available with first neutrons on instruments – our desire and a worthy goal.