1 / 23

Rich Client Platform for Synchrotron Science

Rich Client Platform for Synchrotron Science. An open source not for profit project On GitHub ‘ DawnScience ’ - Diamond Light Source Ltd. and the ESRF are largely publically funded research facilities. Disclaimer AKA - who says that?.

mimir
Download Presentation

Rich Client Platform for Synchrotron Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rich Client Platform for Synchrotron Science • An open source not for profit project • On GitHub ‘DawnScience’ • - Diamond Light Source Ltd. and the ESRF are largely publically funded research facilities

  2. DisclaimerAKA - who says that? • A Java Software Developer (not a Scientist) worked for 12 years with Java. • An Eclipse/RCP fan • I will attempt to explain a bit of the science for your enjoyment (hopefully not schadenfreude). • The rest of the talk will focus on software. • The eclipse features shown in this talk, such as the pipelining tools can be reused outside Science. • Matthew Gerring

  3. SynchrotronAKA – cool word, but what does it mean? syn·chro·tron/ˈsiNGkrəˌträn/ Noun:Acyclotron in which the magnetic field strength increases with the energy of the particles to keep their orbital radius constant. “They are machines which produce very strong light used for various scientific experiments and sometimes other things.”

  4. Where I sit now (User Facility) Where I used to sit (Experimental Facility)

  5. The Queen and Duke of Edinburgh at the official opening of DLS

  6. Scientists with some of the hardware used in their research Inside the ring (not star-trek conduit...) Video of Diamond...

  7. ResponsibilitiesAKA – what they want developers to do... • Software for controlling experiments • Motors, detectors, data collection scripts, configuration. • A high quality but flexible GUI. • Software for data • Ability to visually interact with n-dimensional data (i.e. graphs and slices). • Ability to write scripts to interact with data. • Custom user interface and forms for specific experiments. • Software for running analysis during experiments • Workflow and pipelines. Hard coded and/or user configurable options. • Real time visualization of analysed results.

  8. Integration ToolsAKA – how we are getting it done • EIDES – Eclipse IDE, Stupid - around 20 developers, 8 in scientific software • Eclipse RCP product built using Buckminster (previously PDE) • Usage of Jenkins for continuous integration. Specific releases still produced and tested, appearing as separate items in Jenkins. • Unit tests using Junitand Junitplugintests. • Almost no code walkthroughs and pair programming. Agile practices being used where otherwise possible. • We like Cheat Sheets and use them for tutorial and testing guides.

  9. Git / eGitAKA – no body expects the Spanish Inquisition

  10. ‘Shoulders of Giants’AKA – we didn’t think of everything • RCP many of the core features, editors, toolbars, views, projects • Ptolemy 2 (a version known as ‘Passerelle’) workflow and pipelining • GEF for visualization of pipeline graphs • Draw2D for 1D and 2D plotting (SWT XY Graph) • Pydev for python/jython scripting layer used by the scientists • HDF5 libraries for storing large data sets • SWT/Jface – lazy viewers being used extensively for large trees and tables • Apache, Eclipse-WST, springsource, JDK 6x, and many more...

  11. A quick look at lazy What it does Things to do in the source code tree = newTreeViewer(parent, SWT.MULTI | SWT.H_SCROLL | SWT.V_SCROLL | SWT.FULL_SELECTION | SWT.BORDER |SWT.VIRTUAL); tree.setUseHashlookup(true); publicclassFileContentProviderimplementsILazyTreeContentProvider { ... } tree.setContentProvider(new FileContentProvider()); tree.getTree().setItemCount(...); • Viewer reads contents as incrementally (normal, however...) • Items off screen not loaded • Not icon read, no label • Resources for the item unassigned • Allows large viewer contents to exist but not take up resources until used

  12. Demonstration - Lazy Trees Navigate the file system and open a file containing a large tree

  13. Slicing data • Cutting through N-dimensional data • With an XY plot • As an image • As a 3D iso-surface • Important to run everything concurrently • Use of Jobs • Use of ordinary threads • Use of blocking queues of various sizes • Third party native APIs using Bundle-NativeCode

  14. Demonstration – Slicing and dicing Example opening a tomography file and slicing it

  15. Lots of Tools • Wide range of numeric data tools (some specific to our space - others general purpose) • For images • Line, Box, Sector integration • Diffraction image interpretation, line profile for ‘D-spacing’ • Color mapping / Histogramming • Pixel Information and region control • For XY Graphs • Peak Fitting and Line Fitting • Derivative and other functions, including user defined • XAFS Analysis Tool • Specialist Tools • Hyper 3D • Spectroscopy perspective

  16. Demonstration – Tools and Hyper 3D Example showing slicing data using Hyper 3D on a human skull

  17. Pipelines / workflows based on Passerelle Ptolemy II Eclipse GEF (soon to be Graphiti) EMF / GMF replaced with Ptolemy II • Eclipse feature, EPL /Apache dependencies • Easy to create multi-threaded algorithms using branches • Balance over multi-CPU systems using JDK threading • Runs in separate process using Eclipse debug plugins • Easy to integrate new actors (extension points) RCP application

  18. Demonstration – Simple Matrix Maths Add, subtract – etc some images produced by an experiment...

  19. Pipelines continued • Concept of source actors, for example: • Monitor a file or directory • Monitor a web service • Iterate a large folder of data files • Ability to deploy as a web or rcp service • Run as external process • Communicate to and from controlling process during model run • Add actors using extension points

  20. Demonstration – Efficient Pipelines Pack a large number of files into a high density file (HDF5)

  21. Fridge Control Example • Controlling a fridge using Ptolemy II • Adding and running your own actors • The actors sleep instead of doing things Taken from examples on our github repo ‘DawnScience’

  22. Demonstration – Acme fridge control Example using actors which control a fridge, for instance

  23. Conclusion • Thanks to Eclipse for providing a great tool • RCP is fast and scalable too, using OSGI • SWT has ability to be configured for very large data • Ability to integrate native code in plugins if needed • Maybe we can support web application with RAP one day • Thanks to the Java community for its APIs • We will hopefully be an eclipse project one day... • Thanks for listening and please approach me at the conference if you want to talk about things Diamond Light Source Ltd. www.diamond.ac.uk ESRF www.esrf.fr Data Analysis Workbench, www.dawnsci.org

More Related