1 / 19

Software for MAPS and MERLIN

This talk will provide an overview of the current MAPS/Het/MARI software and discuss the challenges faced in rewriting the software for MERLIN/ARCS. The aim of the meeting is to contribute to the plans for the direction of ARCS software development and explore cooperation in software development. Additionally, participants will gain experience working with an ARCS-like spectrometer and gain a fresh view of the problems of handling huge inelastic datasets.

arnoldp
Download Presentation

Software for MAPS and MERLIN

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software for MAPS and MERLIN T.G. Perring ISIS Facility, Rutherford Appleton Laboratory

  2. Plan of talk: Overview Brief description of the present MAPS/Het/MARI software Challenges of MERLIN/ARCS How we plan to re-write the software Our aims from this meeting: Contribute to the plans for direction of ARCS software development Explore cooperation in software development Experience + a working ARCS-like spectrometer Fresh view of the problems of handling huge inelastic datasets Outline

  3. MAPS spectrometer • Specification: • 20meV< EI< 2000meV • lmod-chop = 10m • lsam-det = 6m • low angle bank: 3-20 high angle bank:  60 • hbar/EI = 1- 5% (FWHH) • 40,000 detector elements • 2500 time channels • 108 pixels 0.4GB datasets Background chopper Monochromating chopper Sample position Position sensitive detector array

  4. Inelastic scattering from single crystals: Triple axis spectrometer: point-by-point serial operation Few kB of data Every group of users writes its own least-squares fitting algorithm from scratch Ad-hoc cooperation only MAPS/MERLIN/ARCS: Parallel operation: collecting on 3D surface in 4D S(Q,w)-space MAPS: raw data: 108 pixels = 0.4GB corrected data: 1-2 x 107 = 0.1-0.2GB x 10 datasets MERLIN, ARCS: 1-2 x 107 = 0.1-0.2GB x 100+ datasets Need ‘Rietveld refinement’ for inelastic scattering Groups cannot afford to write own software Pointless anyway: e.g. in diffraction GSAS, CCSL are trusted We should move to a similar model of trusted black boxes Overview

  5. (Rietveld refinement cont.) Use entire data set to extract a limited number of parameters in a model Magnetic coupling constants Force constants Full inversion of the data: if S(Q,w) in 4D Pragmatism: In magnetism S(Q,w) (or ''(Q,w)) is the quantity to test Qualitative features (are there antiferromagnetic fluctuations and if so, where ?) Unknown processes distorting measure of goodness of fit Rapid slicing and dicing, testing models on limited volumes of data, testing implications for other parts of the data volume, feeding back into operation of experiment seamless integration of simulation, visualisation and analysis programs More formal cooperation: Agreed data structures for instrument information, counts/ S(Q,w) Definitions for input/output of algorithms Capitalise on the investment we individually make in algorithms Overview

  6. Current software on MAPS, HET, MARI Planning experiment Tobyplot (reciprocal space viewing) Chop (resolution, flux) Monitoring experiment Genie-II OpenGenie Data reduction HOMER Genie-II Visualization MSLICE GUI driven Matlab Analysis Tobyfit (single crystals) [Ad-hoc programs in Multi-frills MFIT MSCATT]

  7. Tobyplot Ei=450 meV Psi=101.3° • Reciprocal space viewing • Mostly important for 3D systems • but very useful for 1D, 2D, to assess access in reciprocal space Fortran77 + PGPLOT VMS/Windows/Unix

  8. CHOP • Flux • Resolution: • At elastic position • As function of Test flux/resolution compromises Fortran77 + PGPLOT VMS/Windows/Unix

  9. Monitoring an Experiment GENIE-II I(t) for single spectrum rebinning, units conversion, integration Algebra on spectra [e.g. W1=(0.3W2 + W3)/W4] can call user-written FORTRAN algorithms Good for quick checks during and after experiments Still use today on MAPS VMS only OpenGENIE New generation Used on many ISIS instruments Windows, Unix, VMS Not the features of Matlab, IDL Is free, however

  10. Map file (detectors workspaces) DIAG ( list of bad detectors) MONO_VAN ( absolute units conversion) WHITE_VAN ( solid angle of detectors) Data reduction HOMER (au. Ray Osborn) I(detno, t) corrects Kf/ki, efficiency(kf) S(detno, ) ASCII output file (or VMS binary) • Encapsulates years of experience of the instruments • Scaled very well even to MAPS • $ homer/map=par:pix_981.map/mask=8900/van=8850 8900 100 -30 95 0.25

  11. Imaging single crystal data on HET, MARI, MAPS, and IRIS (Radu Coldea (ISIS / Oak Ridge now Oxford) MSLICE: 2D slices in (Q,) • GUI interface • run info. • sample parameters • 2D & 1D cuts 1D cuts in (Q,)

  12. Least squares fitting of resolution broadened cross-section models (Toby Perring, ISIS) TOBYFIT : Simultaneous fitting to many 2D or 1D data sets • Text-based interface for entering: • instrument • sample parameters • cross-section parameters

  13. MSLICE and TOBYFIT Integral part of the operation of the spectrometer - ‘Tertiary spectrometer’ MSLICE: Visualisation of 3D data in 2D slices, 1D cuts Can generate ‘backgrounds’ from selected parts of the data MATLAB as front end: GUI Graphics manipulating data structures, ad-hoc programming FORTRAN77 for speed of operation of algorithms PC with 1GB RAM, 500MHz+ necessary TOBYFIT: Fitting and simulationtest ideas by feeding S(detno, ) to MSLICE Hold an experiment in a parameter file Fortran + PGPLOT Runs on VMS, Windows, UNIX Communicate via ASCII files: 1 for data, 1 for detector parameters other sample & instrument parameters

  14. Challenges offered by future instrumentation • Physics a function of 4 variables: C(x1,x2,x3,x4) • Instrument gathers data on a 3D volume in that 4D space • Data gathered on a fine non-Cartesian grid • 0.1-0.2 Gbyte • MAPS: usually ~10 settings in an experiment • fine data on 3D surface, coarse in 4th dimension • ARCS/MERLIN: 100+ settings • fine in all 4 dimensions (scan Ei, or crystal orientation) • 10-20GByte complete data set • Data has low statistics - need techniques to pick out features in data • will always need real-time slicing and dicing of data • [too many ways of being led astray or being deceived] • will be doing this after going back to home institution

  15. Visualisation: a hierarchy of views • View 4D data: • ? How do that ? • Define integration interval along any one dimension, and then: • View 3D data: • isosurfaces + slider control for • intensity levels • rotation and viewpoint • binning along the three axes • smoothing, image processing control • Move a plane through the 3D volume: define integration interval along one of the remaining dimensions, and then: • View 2D data: • Contour plots, mountain plots, + slider controls for • contouring levels • interval scaling (linear, log, sqrt …) • binning along the two axes • Move a line across the plane, defining a thickness, and then • View 1D data: • overplotting, fine comparison • book-keeping of titles of the plots ... On raw counts, white beam files,… as well as S(Q,)

  16. Instrument resolution, modelling • Must be able to simulate results of experiments and view results in same way as data (number crunching) • Must perform on-line analysis (resolution-convoluted model fitting,multiple scattering) within framework of same package (even more number crunching) • flexibility: fit on limited volume of data, simulate for whole dataset, slice-and-dice in same way as data to try out ideas • User wants one-stop shop Compare, fit I(det,t) Sexp(Q,w) * inst S calc(Q,w) Convolve with instrument Visualisation, algebra on 1,2,3,4D + Tobyplot MkII

  17. Issues • Number crunching - more than the typical user institute will have • huge storage requirements (not just the raw data) • data management a real problem - we already create hundreds of cuts • thumbnails when click on file • database functions (select by date, temperature, field, scan of a parameter…) • history of analysis stored in file • well-defined data structures needed • ease interact seamlessly with other programs • deconvolution, modelling • user-written algorithms easy to write • define appropriate methods and algebras • (addition, subtraction, background generation, symmetrization…) • not just GUIs: • scripting must always be possible • maximises flexibility

  18. Our plans • Existing programs need to rewritten • F77, getting unmaintainable, monolithic,functionality insufficient, grown organically, written independently • Define NeXus files to hold all processed files • 1D,2D,3D,4D data + all relevant instrument information (detectors etc.) • sufficient information for MCSTAS simulation • include raw data files eventually • Mirror the data structures in Fortran95 + methods • Fortran95 for speedy algorithms • [number crunching for visualisation (MSLICE-2), fitting (Tobyfit-2)] • MATLAB for graphics, language for manipulation and scripting - the glue • [+access to all features of MATLAB for ad-hoc manipulations] • TGP, SMB + inst. Scientists + part effort from 2 post-docs (25-50% effort from each)

  19. (plans cont.) • About to start working with E-Science centre at Rutherford Laboratory • funding to implement grid based applications for science within the laboratory • demonstration projects: • data portal to distributed data stores • graphics processing and number-crunching • ISIS projects chosen to focus the development for real applications • aim to isolate user from where the work is done - no need for their own Beowulf cluster • user will have a front end - in our case we want MATLAB • but also web-based interface (oceanography, space science) • Ensures that one version is maintained • assumes high-speed networks • [GLOBUS toolkit to isolate user from location of resources] • ISIS: full effort of ISIS computing staff member + post-doc • effort of E-Science centre

More Related