areaDetector: A module for EPICS area detector support

1. areaDetector: A module for EPICS area detector support Mark Rivers GeoSoilEnviroCARS, Advanced Photon Source University of Chicago

2. areaDetector Talk Outline • Motivation & goals for areaDetector module • Overview of architecture • Drivers for detectors & cameras • Plugins for real-time processing • Viewers and other clients • Demo with simDetector and Prosilica camera

3. areaDetector - Motivation • 2-D detectors are essential components of synchrotron beamlines • Sample viewing cameras, x-ray diffraction and scattering detectors, x-ray imaging, optical spectroscopy, etc. • EPICS is a very commonly used control system on beamlines, (APS, DLS, SLS, SLAC, NSLS-II, Shanghai, etc.) • Need to control the detectors from EPICS (useful even on non-EPICS beamlines, since other control systems like SPEC etc. can talk to EPICS) • Previously several packages available, each typically restricted to a small set of detectors (Flea, Pilatus, marCCD, etc.) • Clear advantages to an architecture that can be used on any detector, re-using many software components • Providing EPICS control allows any higher-level client to control the detector and access the data (CSS, SPEC, medm, Python scripts, IDL programs, etc)

4. areaDetector - Goals • Drivers for many detectors popular at synchrotron beamlines • Handle detectors ranging from >500 frames/second to <1 frame/second • Basic parameters for all detectors • E.g. exposure time, start acquisition, etc. • Allows generic clients to be used for many applications • Easy to implement new detector • Single device-driver C++ file to write. EPICS independent. • Easy to implement detector-specific features • Driver understands additional parameters beyond those in the basic set • EPICS-independent at lower layers. • Middle-level plug-ins to add capability like regions-of-interest calculation, file saving, etc. • Device independent, work with all drivers • Below the EPICS layer for highest performance

5. areaDetector – Data structures • NDArray • N-Dimensional array. • Everything is done in N-dimensions (up to 10), rather than 2. This is needed even for 2-D detectors to support color. • This is what plug-ins callbacks receive from device drivers. • NDAttribute • Each NDArray has a list of associated attributes (metadata) that travel with the array through the processing pileline. Attributes can come from driver parameters, any EPICS PV, or any user-written function. • e.g. can store motor positions, temperature, ring current, etc. with each frame. • NDArrayPool • Allocates NDArray objects from a freelist • Plugins access in readonly mode, increment reference count • Eliminates need to copy data when sending it to callbacks.

6. EPICS areaDetector Architecture Layer 6 EPICS CA clients Channel Access Clients (medm, IDL, ImageJ, SPEC, etc.) Layer 5 Standard EPICS records xxxDriver.template ADBase.template NDPluginBase.template NDPluginXXX.template Layer 4 EPICS device support Standard asyn device support(device-independent) C++ Base classes (NDArray, asynPortDriver, asynNDArrayDriver, ADDriver, NDPluginDriver) File (netCDF, TIFF, JPEG, HDF5) Layer 3 Plug-ins ROI StdArrays Process Layer 2 Device drivers Driver Channel access Record/device support Layer 1 Hardware API Vendor API asynInt32, Float64, Octet asynXXXArray asynGenericPointer (NDArray) Hardware C library calls

7. areaDetector – Data structures Look at NDArray.h Look at NDAttribute.h Look at an XML attribute file

8. areaDetector Organization (R2-x and later) areaDetector Top-level module RELEASE files, documentation, Makefile • Each box above is a separate git repository • Can be released independently • Hosted at http://github.com/areaDetector project • Each repository is a submodule under areaDetector/areaDetector ADCore Core module Base classes, plugins, simDetector, documentation ADBinaries Binary libraries for Windows (HDF5, GraphicsMagick) ADProsilica Prosilica driver ADPilatus Pilatus driver …

9. Source Code Organization on github • https://github.com/areaDetector is top-level project • Contains configure/ directory where paths and versions of supporting software are defined • Contain .gitmodules to define submodules that will be cloned with git clone –recursive • Contains documentation directory that builds and installs documentation • Contains a top-level Makefile to build all or selected submodules

10. Detector drivers • ADDriver (in ADCore) • Base C++ class from which detector drivers derive. Handles details of EPICS interfaces, and other common functions. • Simulation driver (in ADCore) • Produces calculated images up to very high rates. Implements nearly all basic parameters, including color. Useful as a model for real detector drivers, and to test plugins and clients. • Prosilica driver (ADProsilica) • Gigabit Ethernet cameras, mono and color • High resolution, high speed, e.g. 1360x1024 at 30 frames/second = 40MB/second. • Firewire (IEEE-1396 DCAM) (ADFireWireWin, firewireDCAM) • Vendor-independent Firewire camera drivers for Linux and Windows • Roper driver (ADRoper) • Princeton Instruments and Photometrics cameras controlled via WinView

11. Detector drivers (continued) • PVCAM driver (ADPvCam) • Princeton Instruments and Photometrics cameras controlled via PVCAM library • Pilatus driver (ADPilatus) • Pilatus pixel-array detectors. • marCCD driver (ADmarCCD) • Rayonix (MAR-USA) CCD x-ray detectors • ADSC driver (ADADSC) • ADSC CCD detectors • mar345 driver (ADmar345) • marResearch mar345 online image plate • Perkin-Elmer driver (ADPerkinElemer) • Perkin-Elmer amorphous silicon flat-panel detectors

12. Detector drivers (continued) • Bruker driver (ADBruker) • Bruker detectors controlled via their Bruker Instrument Server (BIS) • LightField driver (ADLightField) • Princeton Instruments detectors controlled via their LightField application using the Microsoft Common Language Runtime to automate it • PSL driver (ADPSL) • Photonic Sciences Limited detectors • URL driver (ADURL) • Driver to display images from any URL. Works with Web cameras, Axis video servers, static images, etc. • Andor driver (ADAndor) • Driver for Andor CCD cameras • Andor3 driver (ADAndor3) • Driver for AndorsCMOS cameras with V3 of their SDK

13. Detector drivers (continued) • Point Grey driver (ADPointGrey) • Driver for GigE, USB-3.0, USB-2.0, and Firewire cameras from Point Grey Research • Pixirad driver (ADPixirad) • Driver for CdTe pixel-array detectors from Pixirad • Generic GigE driver (aravisGigE) • Should work with any GigEVision compliant camera. From Tom Cobb at Diamond. Uses Aravis reverse-engineered GigEVision library • PVAccess (EPICS V4) driver • Receives NTNDArrays over PVAccess • Allows plugins to run on other processes or machines from the areaDetector driver

14. ADBase.adl – Generic control screen • Works with any detector • Normally write custom control for each detector type to hide unimplemented features and expose driver-specific features

15. Pilatus specific control screen

16. MAR-345 specific control screen

17. LightField driver

18. LightField driver

19. URL Driver • Driver that can read images from any URL. • Can be used with Web cameras and Axis video servers. • Uses GraphicsMagick to read the images, and can thus handle a large number of image formats (JPEG, TIFF, PNG, etc.).

20. Andor Driver • Supports USB and PCI CCD cameras from Andor. • Runs on 32-bit and 64-bit Linux and 32-bit and 64-bit Windows. • Original version by Matt Pearson from Diamond Light Source.

21. Perkin Elmer Flat Panel Driver

22. R2-0: Point Grey driver • New driver for all cameras from Point Grey using their FlyCap2 SDK. • Firewire, GigE and USB 3.0 • High performance, low cost

23. Point Grey GigE CameraBlackFly PGE-20E4C • e2v EV76C570 CMOS sensor • Global shutter • 29 x 29 x 30 mm • Power Over Ethernet • 4.5 micron pixels • 1600 x 1200 pixels, color (mono) • 47 frames/s • $595 • 5X cheaper than comparable Prosilica cameras we bought in the past

24. Point Grey USB-3.0 CameraGrasshopper3 GS3-U3-23S6M • 1920 x 1200 global shutter CMOS • Sony IMX174 1/1.2 • No smear •  Distortion-free • Dynamic range of 73 dB • Peak QE of 76% • Read noise of 7e- • 12-bit or 8-bit data • Max frame rate of 162 fps • ~356 MB/S, >3X faster than GigE • USB 3.0 interface • $1,295

25. Point Grey Driver

26. Point Grey Driver (Grasshopper3 camera)

27. Pink Beam, Mirror=2.0 mrad • Mirror angle=2.0 mrad (Beads_Pink_H) • 2 mm Al absorber • 8-bit data • 1 ms exposure time, 124 frames/s, 900 projections, 7.3 seconds total • Rotation axis orientation corrected for mirror angle Vertical slice Horizontal slice

28. Plugins • Designed to perform real-time processing of data, running in the EPICS IOC (not over EPICS Channel Access) • Receive NDArray data over callbacks from drivers or other plugins • Plug-ins can execute in their own threads (non-blocking) or in callback thread (blocking) • If non-blocking then NDArray data is queued • Can drop images if queue is full • If executing in callback thread, no queuing, but slows device driver • Allows • Enabling/disabling • Throttling rate (no more than 0.5 seconds, etc) • Changing data source for NDArray callbacks to another driver or plugin • Some plugins are also sources of NDArray callbacks, as well as consumers. • Allows creating a data processing pipeline running at very high speed, each in a different thread, and hence in multiple cores on modern CPUs.

29. Plugins (continued) • NDPlugInStdArrays • Receives arrays (images) from device drivers, converts to standard arrays, e.g. waveform records. • This plugin is what EPICS channel access viewers normally talk to. • NDPluginROI • Performs region-of-interest calculations • Select a subregion. Optionally bin, reverse in either direction, convert data type. • Divide the array by a scale factor, which is useful for avoiding overflow when binning. • NDPluginColorConvert • Convert from one color model to another (Mono, RGB1 (pixel), RGB2 (row) or RGB3 (planar) interleave) • Bayer conversion removed from this plugin, now part of Prosilica and Point Grey drivers. • NDPluginTransform • Performs geometric operations (rotate, mirror in X or Y, etc.)

30. Plugins (continued) • NDPluginStats • Calculates basic statistics on an array (min, max, sigma) • Optionally computes centroid centroid position, width and tilt. • Optionally Computes X and Y profiles, including average profiles, profiles at the centroid position, and profiles at a user-defined cursor position. • Optionally computes the image histogram and entropy • NDPluginROIStat • Multiple ROIs with simple statistics in a single plugin • More efficient when many ROIs are needed, e.g. for peaks in a 1-D energy spectrum • Min, max, total, net, mean • Time-series of each of these statistics

31. Plugins (continued) • NDPluginProcess • Does arithmetic processing on arrays • Background subtraction. • Flat field normalization. • Offset and scale. • Low and high clipping. • Recursive filtering in the time domain. • Conversion to a different output data type. • NDPluginOverlay • Adds graphic overlays to an image. • Can be used to display ROIs, multiple cursors, user-defined boxes, text, etc. • ffmpegServer • MJPEG server that allows viewing images in a Web browser. From DLS.

32. Plugins (continued) • NDPluginAttribute • Extracts NDAttributes from NDArrays and publishes their values as ai records • Can collect time-series arrays of the attribute values • NDPluginCircularBuff • Buffers NDArrays in a circular buffer • Computes a trigger expression using up to 2 NDAttribute values • When trigger condition is met then outputs NDArrays • User-specified number of pre-trigger and post-trigger arrays to output • NDPluginTimeSeries • Accepts 1-D NDArrays[NumSignals] or 2-D [NumSignals,NewTimePoints] and appends to time-series buffer • Operates in fixed length (stop when full) or circular buffer modes • Optional time-averaging of input data

33. Plugins (continued) • NDPluginFFT • Computes FFT of 1-D or 2-D NDArrays • Exports NDArrays containing the absolute value (power spectrum) of the FFT • Exports 1-D arrays of the FFT real, imaginary, absolute values, and time and frequency data. • NDPluginPVA • Converts NDArrays to EPICS V4 NTNDArrays • Exports the NtNDArrays over PVAccess with internal V4 server • Can be used to send structured data to EPICS V4 clients • When used with the PVAccess driver then areaDetector plugins can be run on different machine from the detector driver

34. commonPlugins.adl All plugins at a glance

35. NDStdArrays plugin

36. ROI plugin

37. Statistics plugin

38. Statistics plugin (continued)

39. Overlay plugin

40. Overlay plugin

41. Overlay plugin

42. Overlay plugin Centroid of laser pointer calculated by statistics plugin Cursor overlay X, Y position linked to centroid

43. Processing plugin

44. Processing plugin30 microsec exposure time N=100 recursive average filter No filtering

45. Transform plugin • R2-1 changes • Greatly simplified: just 8 operations including null operation • 13-85 times faster than previous releases depending on data type, color mode

46. Plugins: NDPluginFile • Saves NDArrays to disk • 3 modes: • Single array per disk file • Capture N arrays in memory, write to disk either multiple files or as a single large file (for file formats that support this.) • Stream arrays to a single large disk file • For file formats that support it, stores not just NDArray data but also NDAttributes

47. Plugins: NDPluginFile • File formats currently supported • NDFileTIFF • Supports any NDArray data type • Stores NDAttributes as ASCII user tags • NDFileJPEG • With compression control • NDFileNetCDF • Popular self-describing binary format, supported by Unidata at UCAR • NDFileHDF5 • Writes HDF5 files with the native HDF5 API, unlike the NeXus plugin which uses the NeXus API. Supports 3 types of compression. • Supports using an XML file to define the layout and placement of NDArrays and NDAttributes in the HDF5 file • R2-5 will support Single Writer Multiple Reader (SWMR). Only supported on local file systems, GPFS, and Lustre(not NFS or SMB)

48. Plugins: NDPluginFile • File formats currently supported • NDFileNeXus • Standard file format for neutron and x-ray communities, based on HDF5, which is another popular self-describing binary format; richer than netCDF • May be deprecated in a future release since NeXus files can now be produced with the NDFileHDF5 plugin using an appropriate XML layout file • NDFileMagick • Uses GraphicsMagick to write files, and can write in dozens of file formats, including JPEG, TIFF, PNG, PDF, etc. • NDFileNull • Used only to delete original driver files when no other file plugin is running

49. File saving with driver • In addition to file saving plugins, many vendor libraries also support saving files (e.g. marCCD, mar345, Pilatus, etc.) and this is supported at the driver level. • File saving plugin can be used instead of or in addition to vendor file saving • Can add additional metadata vendor does not support • Could write JPEGS for Web display every minute, etc.

50. NDPluginFile display: TIFF Example: saving 82 frames/second of 1024x1024 video to TIFF files, a few dropped frames.