1 / 15

areaDetector Data Processing Pipeline In EPICS V4

areaDetector Data Processing Pipeline In EPICS V4. Dave Hickin Diamond Light Source EPICS V4 Working Group Meeting SLAC Source 02/10/2013 . Objectives. Lossless high-performance transfer of detector data and camera images including metadata. Software infrastructure to support it.

eliora
Download Presentation

areaDetector Data Processing Pipeline In EPICS V4

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. areaDetector Data Processing Pipeline In EPICS V4 Dave Hickin Diamond Light Source EPICS V4 Working Group Meeting SLACSource 02/10/2013

  2. Objectives • Lossless high-performance transfer of detector data and camera images including metadata. • Software infrastructure to support it. • Framework for high performance scientific data services

  3. Why transfer detector data? • Transferring data between platforms(Usually from Windows to Linux) • Cameras often have Windows only support • Better support for HPC in Linux, e.g. HP file system • Linux toolchain • Preference for Linux (open source, reliability etc.) • More expertise on Linux • Distributing processing

  4. Case Study – I12 Beamline • I12 Beamline at Diamond • Joint Engineering, Environmental and Processing(JEEP) • Imaging, Tomography, X-ray diffraction, SAXS • PCO detector, Windows-only support • HDF5-writer, Lustre distributed files system • ~90 10MB images per second • 10 Gig Ethernet

  5. Practical considerations • Large amount of effort gone into developing areaDetector. New application would require significant resource • Large amount of effort in deploying areaDetector • Limited resources for development • Suggests incremental development and deployment based initially on areaDetector(“evolution not revolution”) • Metadata must be kept with image in structured data

  6. areaDetector overview • Provides a general-purpose interface for detectors and cameras in EPICS • Easily extensible • Supports wide variety of detectors and cameras • High-performance • Mechanism for device-independent real-time data analysis

  7. areaDetector overview (cont) • Camera drivers inherit from base class ADDriver • Drivers produces NDArrays Data; timestamp; size, offset, reverse, binning; unique ID Attributes (name, description, source(+type), value) • Run plugins (HDF writer, stats, ROI) • Plugins can be chained together • Plugins inherit from NDPluginDriver • Connect to asyn port on a driver (or plugin) • Consume NDArrays

  8. areaDetector and EPICS V4 areaDetector runs a plugin which is a pvAccess server Plugin translates NDArrays into EPICS V4 structured data (normative type). Closely maps to NDArray pvAccess used to transfer data V4 client implements ADDriver Translates V4 type back into NDArray Passes NDArrays to plugins

  9. Possible detector solution: V4 for distributed processing Server publishes V4 structured data PV containing the data Client plugins (remote or local) monitor PV Clients process data, publish output or republish Plugins can be chained through pvAccess V4 PVs replace asyn ports in areaDetector Server and clients could be implemented with or without areaDetector Copying minimised through sharing data

  10. Current State of Development • ADTransfer • ~Production quality • James Rowland, Ulrik Pedersen, Jon Thomson at Diamond • Ad hoc solution • Doesn’t use pvAccess network protocol or EPICS V4 core code • Uses pvAccess serialisation and data structure • Server pushes data to client • Runs on Windows • Optimised. Very efficient (95% usage of 10Gig Ethernet)

  11. Current State of Development (contd) • EPICS V4 Prototype • Early stages of development • Uses pvAccess protocol • Client monitors PV • Uses pvAccess and EPICS V4 core code • Not robust or high performance • Not based on libraries except core code • Linux only. No Windows support yet

  12. EPICS V4 Server and Client Prototype

  13. Prototype: Images

  14. Development Plan for areaDetector data processing • Use standard V4 libraries (local provider) • Optimise performance (sharing data, network optimisation) • Run on Windows • Reliability • Package • Other solutions (e.g. w/o AD) • Integration (e.g. standard types, broadcast)

  15. Associated development plan for EPICS V4 Base Development of EPICS V4 libraries - local channel provider for C++ (Marty Kramer) Efficient handling of large arrays to reduce copying (Michael Davidsaver) Windows support (MatejSekoranja, Peter Heesterman) Broadcast/Multicast (Matej Sekoranja) Any/Variant (MatejSekoranja)

More Related