1 / 22

The Data Acquisition Card for the Large Pixel Detector at the European-XFEL

The Data Acquisition Card for the Large Pixel Detector at the European-XFEL John Coughlan, Sam Cook, Chris Day, Rob Halsall and Saeed Taghavi Science & Technology Facilities Council Rutherford Appleton Laboratory Oxfordshire, United Kingdom E-mail : john.coughlan@stfc.ac.uk. Contents.

xander
Download Presentation

The Data Acquisition Card for the Large Pixel Detector at the European-XFEL

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Data Acquisition Card • for the Large Pixel Detector • at the European-XFEL • John Coughlan, Sam Cook, Chris Day, Rob Halsall and Saeed Taghavi • Science & Technology Facilities Council • Rutherford Appleton Laboratory • Oxfordshire, United Kingdom • E-mail: john.coughlan@stfc.ac.uk Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  2. Contents • European-XFEL • Large Pixel Detector (LPD) • LPD DAQ Card (FEM) • FPGA Firmware 10G UDP/IP • Status Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  3. European-XFEL DESY Hamburg Experiments to start operation in 2015 Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  4. X-ray photons 220 ns 100 fs FEL process Data Sampling to ASIC Analogue Memory Digitize and Serial Transmit to DAQ card Eu-XFEL Bunch Structure Electron bunch trains; up to 2,700 bunches in 600 msec, repeated 10 times per second. Producing 100 fsec X-ray pulses (up to 27,000 bunches per second). 100 ms 100 ms 600 ms 99.4 ms XFEL ~ 27,000 bunches/s with 99.4 ms (%) emptiness Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  5. Large Pixel Detector • 1 Mega-pixel Detector • 0.5 m x 0.5 m • 16 SuperModules (SM) • 1 DAQ card per SM • 128 ASICs per SM • Delivery in 2013 • 16 sensor modules per SM Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  6. LPD ASIC • 130 nm IBM • 16 x 32 pixels • Large Dynamic range ~1 – 10**5 photons • 3 x Gains stored • Analogue Pipeline Memory • 512 samples deep • Triggered Operation • Digitise @12 bits and serial readout Gains x 100 x 10 x 1 Store in Pipeline during Bunch Train Readout all 3 Gain values during long 99 msec gap RAL Micro-Electronics Paper submitted to IEEE NSS Valencia Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  7. LPD Data Acquisition LPD DAQ Card FEM x 16 LPD Detector (STFC) Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  8. XFEL Data Acquisition LPD DAQ Card FEM x 16 Fast Timing Signals LPD Detector (STFC) GbE Controls • XFEL Clock = 99 MHz • Bunch Clock = 4.5 MHz • Train Rate = 10 Hz MicroTCA (Physics Ext) Clock & Controls and Veto Systems (UCL & DESY) See Poster Session Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  9. XFEL Data Acquisition • 2 MByte x 512 x 10 • ~10 GBytes/sec Train Builder System (STFC) Data Readout See Posters Session LPD DAQ Card FEM 10 Gbps Optical (DESY) x 16 Fast Timing Signals LPD Detector (STFC) GbE Controls AdvancedTCA • XFEL Clock = 99 MHz • Bunch Clock = 4.5 MHz • Train Rate = 10 Hz • Common Systems MicroTCA (Physics Ext) Clock & Controls and Veto Systems (UCL & DESY) See Posters Session Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  10. Train Builder FEM x 16 A B M N E F I J C D 2D Detector 1 Megapixels X-ray Images up to 512 per train K L O P G H XFEL Clock & Controls Clock Cmd Front End Electronics FEM B C A P Veto 10 Gbps optical links 30-100 m FEE Fragments 7 6 5 4 3 2 1 Train Nr in Buffer Data Flow Switch Train Builder System Advanced TCA Train Nr % N % 1 % 2 % 3 % 16 10 GBytes/sec Complete Trains 10 Gbps optical links PC Farm PC Layer see Poster Session “Train Builder Data Acquisition System for the European-XFEL” Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  11. FEM Functional Units FMC ANSI/VITA57 Train Builder PC Virtex 5 FX100T C&C Dual PPC PC 128 LPD ASICs • JTAG Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  12. FEM Clock Domains Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  13. LPD FEM Side 1 XFEL C&C SP3 I/O GbE Slow Controls Virtex 5 FX100T FPGA 2 x Samtec Backplane Connectors 240 way SP3 I/O SP3 CFG 128 LPD ASICs FLASH SRAM FEM side 1 Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  14. LPD FEM Side 1 XFEL C&C SP3 I/O GbE Slow Controls Virtex 5 FX100T FPGA 2 x Samtec Backplane Connectors 240 way SP3 I/O SP3 CFG LPD FLASH SRAM XFEL DAQ FEM side 1 • Dual 10 Gbps SFP+ • FPGA Mezzanine Card • FMC ANSI/VITA 57 • (DESY) • Train Builder Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  15. LPD FEM Side 2 LPD XFEL DAQ DDR2 SODIMM 1 GByte Memory JTAG cable Compact Flash configuration • PCB 16 layers (8 signal) RS232 cable FEM side 2 Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  16. FPGA Units Overview Virtex 5 FX100T FPGA (Dual PPC cores) Clock & Controls ASIC Fast Commands C&C Emulation Timing & Veto SP3 Top I/O (80%) SP3 I/O FPGAs ASIC Slow Ctrl GbE/ RS232 PC Processor Embedded ASIC Data Rx 128 ASICs Local Link 10GbE Link UDP/IP LPD Local Link XFEL DAQ DDR2 Memory Controller ASIC Simulated Data Virtex5 Main FPGA SP3 Confign FPGA DATA CTRLS Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  17. FPGA ASIC Unit ASIC Clock ‘99’ MHz PLL Master Clock Clock & Controls Timing System Interface ASIC Fast Serial Interface Clock State Machine VETOS Master Sync, Trigger, Readout, status… Master Cmd Strm Emulation ASIC Slow Serial Interface Computer System Interface CONFIG PLB ASIC Fast Data Interface Local Link 128 DATA OUTPUT ~ 640 MByte/s Hit List for Veto Mode 1 of 3 Gain Selection INPUT ~ 1.5 GBytes/s Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  18. Testing Bench with LPD ASIC PC with 10GbE NIC running MatLab software Power and Cooling Pseudo-Random Data Image Single ASIC module FPGA dev board with C&C test adapter • MatLab UDP • RS232 control Extender Card with Nat Inst cable FEM with DESY 10G FMC Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  19. FPGA 10G UDP/IP Firmware Module Xilinx “Local Link” Interfaces to Power PC DDR2 Memory Controller And ASIC Data Receiver Module 10G Test Bench • Chelsio T4 NIC • Quad 10G SFP+ • x8 PCIe Gen2 • UDP API “Direct Driver” to App Memory 10GbE UDP/IP Firmware Module Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  20. LPD Software • FPGA Embedded System Dual PPC Cores • Core #1 for DDR2 DMA Memory Controller • Core #2 Running Xilkernel / lightweight IP stack (LwIP) • Have TCP control protocol over TCP/IP • Control resources on board (GPIO, I2C, EEPROM) • FEM support library library & applications (C & Python) for rapid prototyping developed • GDA GUI based controls system as used on Diamond Light Source at Rutherford PC Software FPGA Embedded Software Xilinx EDK Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  21. FEM Status • 4 FEM cards assembled. All passed JTAG without error. • All components pass functional tests. • FPGA 10G UDP/IP working with DESY FMC to PC NIC. • Clock and Controls and Slow Controls GbE working. • FEM reading out images from LPD ASIC module. • Emulating Train Builder interface with FPGA dev boards. • Manufacturing another 20 FEMs for LPD (and Medipix) • Test FEM in LPD SuperModule in 2012 • Ready for LPD 1 Mega-pixel detector 2013 Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

  22. Thank you Acknowledgements: LPD Collaboration (STFC Rutherford Appleton Laboratory) M. Zimmer, I. Sheviakov (DESY) C. Youngman (XFEL) Posters Session: “Design of the Train Builder Data Acquisition System for the European-XFEL” John Coughlan et al. “Design and Development of Electronics for the EuXFEL Clock and Control System” Erdem Motuk et al. Tuesday 28th September 2011, TWEPP Vienna Presented by John Coughlan STFC Rutherford Appleton Laboratory

More Related