1 / 34

Fast 2D Detector Control at ESRF beamlines

Fast 2D Detector Control at ESRF beamlines. David Fernández Alejandro Homs Emmanuel Papillon Manuel Pérez. Vicente Rey Cyril Guilloud V. Armando Sole Assen Kirov. Talk outline. Introduction ESRF FReLoN CCDs 2D control arquitecture Espia card based systems Espia card FReLoN 2k, Maxipix

gitel
Download Presentation

Fast 2D Detector Control at ESRF beamlines

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Fast 2D Detector Control at ESRF beamlines David FernándezAlejandro HomsEmmanuel PapillonManuel Pérez Vicente ReyCyril Guilloud V. Armando SoleAssen Kirov

  2. Talk outline • Introduction • ESRF FReLoN CCDs • 2D control arquitecture • Espia card based systems • Espia card • FReLoN 2k, Maxipix • Dalsa, Sarnoff, Aviex • Latest developments • Driver improvements • Data saving • Common CCD acq. platform

  3. FReLoN – Fast Readout & Low Noise FReLoN 1000 (1996): • 1024 x 1024 - 14 bit • 4 parallel readout channels • Readout time: 100 ms FReLoN 2000 (1998): • 2048 x 2048 - 14 bit • Readout time: 220 ms

  4. FReLoN 1000/2000 acquisition setup SPEC TACO Device server Gigabit Ethernet

  5. Other CCD-like devices supported • Princeton • Sensicam, PCO • Photometrics • Matrox Meteor II [H. Witsch, J. Meyer] • Trixell Pixium [L. Claustre] • Andor • ImagePro [A. Gotz] • Direct-Show • Mar CCD [L. Claustre] • IDS Falcon [D. Miraut, J. Meyer] • Firewire (IEEE 1394) [A. Gotz] • Dalsa Dalstar

  6. ADSC Q315r for MX beamlines Local Gigabit Gigabit Ethernet

  7. Espia card • Designed to interface the FReLoN 2k • Collaboration  ESRF + SECAD, S.A. • 2 Gbps fiber optic link (> 100 m) • PCI 64 bits / 66 MHz • ~180 MB/s maximum data rate • Bus master + Scatter Gather + IRQ • Driver source code available  rewritten! • Can write on 32-bit addresses only (4 GB)

  8. Control workstation (2005) Transtec (2005): • Dual Intel Xeon 2.6 GHz, HT • 2 GB ECC DDR-RAM • 3 x 73.4 GB Ultra320 SCSI 15Krpm • SuSE 8.2 + Linux 2.4.20-64GB-SMP • Remote NFS through 1 Gbps link 90 MB/s • 2 disks in software RAID 0  100 MB/s (8 MB files) • 4 disks in software RAID 0  150 MB/s (1 GB files) for ultra-fast tomography

  9. New FReLoN Generation FReLoN 2k (ATMEL): • No Mux – only one fiber optic! • Frame Transfer Mode • 2048 x 1024 @ 16 fps  66 MB/s FReLoN 4M (KODAK): • Only 4 channels – Full Frame Mode FReLoN HD (PSB-2): • 2048 x 1024 @ 32 fps  ~125 MB/s

  10. Camera Link: FOCLA • Connects to the Espia • 2 x Camera Link connectors: • Base configuration • Multiplexed • Multiple pixel packing formats • Test image generator @ ~180 MB/s

  11. Dalsa Pantera 1M60 • Frame transfer technology • 1024 x 1024 @ 60 fps  120 MB/s • 1 x Camera Link [Base] Connector • Compact! • Image not reconstructed • FOCLA does reconstruction

  12. Fast online display Dalsa setup Image reconstruction

  13. Sarnoff CAM512 • 512 x 512 @ 300 fps  150 MB/s • 512 x 128 @ 500 fps  125 MB/s • 2 x 8 ADCs – 12 bit • 2 x Camera Link [Base] Connectors • Separate RS-232 serial line

  14. Sarnoff setup RS-232 Camera Link Fiber optic

  15. Sarnoff results (2007) • 300 fps • 75 frames • 250 ms • 40x slowdown • No sound! • B&W • Flickering (room light) ID15: • Tomography in 0.5 sec @ 500 fps

  16. The real image Espia #1 Espia #2

  17. Image reconstruction in kernel Espia #1 Espia #2

  18. “Very High memory” support High frame rate  many image buffers Physical Memory Implementation in driver • Driver always has 32-bit aux. buffers • If next target frame has 64-bit pages: • Program each Espia to write on aux buffer • Frame end (all adapters finished) copy from aux. into target buffer • Inform waiting threads • Adapters are not synchronous • Memory copy with IRQ disabled  Kernel thread 5.8 GB Very high RAM 32 bit boundary 4 GB PCI 3.2 GB High RAM 800 MB Low RAM 0 MB

  19. Aviex PCCD-8448 • 2048 x 3584 @ 1.2 fps  18 MB/s • Binning 16 x 16  8 fps

  20. ADC ADC ADC ADC Software image reconstruction • Per frame operation • Warranted only once in a multi-frame (big) buffer • Callback thread safe

  21. Medipix2 / Maxipix • Pixel detector  photon counting • 256 x 256 – 13 bit • 0.3 ms readout time • 1000 fps  130 MB/s • Max: 1400 fps  180 MB/s • Chipboard: University of Geneve • PRIAM: ISG/ESRF • Interfaced to the Espia Espia

  22. Maxipix results (2007) • 1000 fps !!

  23. Multichip Maxipix • Medipix2 array • 5 x 1  1280 x 256 • 1000 fps  650 MB/s • PRIAM multiple FO links • Future configs: 2 x 2, 5 x 2

  24. New control workstation OPALE server from Ecrin Systems (FR): • Trenton Single Host Board and backplane • Dual Quad Core Intel Xeon @ 2.33 GHz • 8 GB of FB DDR2 • 3 x 140 GB SAS disks @ 15 Krpm • Chelsio T320 10 Gbps Ethernet controller

  25. Driver evolution • 32-bit application  3GB address space • 8GB RAM  Dynamic mapping of buffers • Move to 64-bit architecture • No so trivial • ESRF/Linux 2.1 [Red Hat Enterprise Linux 5] • Kernel version 2.6.18 • Copy to very-high-memory: 1.3 GB/s

  26. Sarnoff @ ID15 • 512 x 512 12 bits @ 300 fps  150 MB/s • Write to local disks: 300 MB/s • Write to remote disks: 300 MB/s NetApp FAS6070 10 Gbps Ethernet

  27. “La Bestia” • Dell PowerEdge 2900 • Similar to Ecrin/Trenton + 16 GB RAM • 4 + 6 = 10 disks  530 MB/s (800 MB/s raw) • Super Sarnoff: 2x FOCLA test image • Acquisition rate: ~360 MB/s Running

  28. Turbo Bestia • 4 + 6 + 8 = 18 disks • Single RAID 0 array  840 MB/s raw • 3 RAID 0 arrays + parallel write  1.5 GB/s raw 4x 6x 8x

  29. Espia Next Generation • PCI-Express 8x lanes • 4 fiber optic links @ 250 MB/s 1 GB/s • Full access to 64-bit memory • Maximum register compatibility

  30. Common CCD acq. Library • Reuse of common code  generic procedures + interfaces • Software “features” fallback if hardware has limited capabilities • High level Python implementation • Common TACO CCD interface • Interfaces: implicit  explicit • Image processing and data reduction as plugins (C++) • Extensible interface through user-defined “features” (Maxipix) • Generic TANGO interface • Collaboration framework (ALBA)

  31. Current status x 20 x 10 + 1[+1] (C++) x 1 (Python) x 5 + 1

  32. Conclusions • Significant number of supported 2D detectors • This number keeps raising … • New detectors demand high performance control • Experience in high speed acquisitions • Stable low level driver (Sarnoff failed only once…) • Functional high level interface • Never-ending project … (sub-projects) • Common image acquisition library will help!

  33. Acknowledgements SECAD, S.A. • F. Hereson Instrument Support Group: • J.C. Labiche • E. Collet • L. Siron • J.J. Thevenin • D. Pothin • C. Ponchut • J.M. Rigal • J. Clement • P. Fajardo Computing Services: • P. Makijarvi • B. Regad • P. Pinel • L. Garçon • F. Andrei • D. Gervaise • S. Ohlsson • B. Lebayle

  34. Thank you!

More Related