540 likes | 780 Views
Data Acquisition Systems. Ryan Rivera Fermilab Detector R&D Retreat May 5, 2011. Wikipedia. Data acquisition is the process of sampling signals that measure real world physical conditions and converting the resulting samples into digital numeric values that can be manipulated by a computer. .
E N D
Data Acquisition Systems Ryan Rivera Fermilab Detector R&D Retreat May 5, 2011
Wikipedia • Data acquisition • is the process of sampling signals that measure real world physical conditions and converting the resulting samples into digital numeric values that can be manipulated by a computer. Data Acquisition Systems - Ryan Rivera
This Talk • Current DAQ program • Future projections • Direction of Current Program • Direction of New R&D Data Acquisition Systems - Ryan Rivera
Current Program • Optical Links • CAPTAN • xTCA • Mu2e Data Acquisition Systems - Ryan Rivera
Current Program • Optical Links • CAPTAN • xTCA • Mu2e Data Acquisition Systems - Ryan Rivera
Optical Links • Versatile Link Common Project • Free Space Optical Transmission Data Acquisition Systems - Ryan Rivera
Optical Links • Versatile Link Common Project • Free Space Optical Transmission Data Acquisition Systems - Ryan Rivera
Versatile Link Common Project • A CERN-organized common project for ATLAS and CMS. • Charge: Develop general purpose optical link @ 5 Gbps data rates. • Fermilab has back-end component responsibilities. • Leading parallel optics investigations. Data Acquisition Systems - Ryan Rivera
Optical Transceiver Test Measurements Industry Standard Measurements and Apparatus • Eye Diagram Measurements: • Optical Modulation Amplitude • Extinction Ratio • Rise/Fall Times • Jitter Analysis: • Deterministic Jitter Decomposition • Random Jitter (Gaussian, Unbounded) • Eye Opening @ 10-12 BER Data Acquisition Systems - Ryan Rivera
CMS Pixel Optohybrid (POH) • Status: Current laser no longer available • Upgrades will require new devices • Requirements: • Rad hard • 640 Mbps • Next: FNAL responsible for testing and characterizing devices Data Acquisition Systems - Ryan Rivera
Parallel Optics – Technology Evolution Data Acquisition Systems - Ryan Rivera
Parallel Optics – Device Evaluation SNAP12 Transmitter (12 channels, 2.7 Gbps/channel) Parallel Optical Engine Transceiver (4 channels, 6.25 Gbps/channel) SFP+ Single Channel Transceiver (10 Gbps) Data Acquisition Systems - Ryan Rivera
Optical Links • Versatile Link Common Project • Free Space Optical Transmission Data Acquisition Systems - Ryan Rivera
Free Space Optical Transmission 10Gb/s Optical Receivers • Motivation • Reduce material budget • Work within rigid space constraints 10 Gbps Optical Transmitters at different wavelengths Silicon Detectors ~100-150 cm ~50-100 cm ~10- 50cm Beam Line Center Data Acquisition Systems - Ryan Rivera
Free Space Proof of Concept l1 = 1470 nm l2 = 1490 nm l3 = 1510 nm l4 = 1530 nm CWDM: Coarse Wavelength Division Multiplexing 21 cm CWDM DeMux CWDM Mux 8cm Lens Lens ~2 mm Si ~2 mm Si FPGA-Based BERT l1 l1 Rx Tx l2 l2 Rx Tx l3 l3 Rx Tx l4 l4 Rx Tx Rx LVDS Electrical Bit Streams Tx LVDS Electrical Bit Streams Next Step: Evolve the Optical Design for Detector Applications Data Acquisition Systems - Ryan Rivera
Free Space Proof of ConceptLab Test Operated error free for over 48 hours at 1 Gbps on all 4 four channels Data Acquisition Systems - Ryan Rivera 16
Optical Links Future • Parallel optics • Initiated US R&D request to recent DOE detector solicitation. Need KA15 money until DOE approval. • Working with vendors on emerging devices, and SMU/OSU on custom devices. • Versatile Link and CMS Phase 1 Optohybrid • Activities not funded by KA15, funded through CMS. • Free space optics • Exploring low mass/power options (modulators). Data Acquisition Systems - Ryan Rivera
Current Program • Optical Links • CAPTAN • xTCA • Mu2e Data Acquisition Systems - Ryan Rivera
CAPTAN What is the CAPTAN system? Data Acquisition Systems - Ryan Rivera
Compact And Programmable daTa Acquisition Node Data Acquisition Systems - Ryan Rivera
Attributes • It is simple. • It is 6” x 6” and for many systems, the only external connections are a 3.3V power supply and a standard Ethernet cable. Data Acquisition Systems - Ryan Rivera
Attributes • It is flexible. • The user can stack the foundation boards in different combinations to give unique functionality. Data Acquisition Systems - Ryan Rivera
Attributes • It is scalable. • In addition to the vertical stacking, the stacks can be repeated arbitrarily and connected with one or many PCs in an Ethernet network. Data Acquisition Systems - Ryan Rivera
COOLING CHANNEL OPTICAL BUS ELECTRONICS VERTICAL BUS LATERAL BUS MOUNTING HOLE User Template • The CAPTAN architecture consists of a few core boards but is intended to be augmented by custom boards designed and built by users. Data Acquisition Systems - Ryan Rivera
Core Boards “Green Board” NPCB – Node Processing and Control Board “Blue Board” DCB – Data Conversion Board “Red Board” PDB – Power Distribution Board. Data Acquisition Systems - Ryan Rivera
CAPTAN User Community • Fermilab • Brown • Purdue • Colorado • Milano • Lecce Data Acquisition Systems - Ryan Rivera
Applications • The CAPTAN system was designed to handle common data acquisition, control, and processing challenges within high energy physics. • Examples of such applications are tracker readout systems, R&D test stands, and parallel data processing. • As the CAPTAN system is a modular system it can be used for a wide range of applications, from very small to very large. Data Acquisition Systems - Ryan Rivera
FTBF Pixel Telescope Data Acquisition Systems - Ryan Rivera
PPD VIP Test Stand Data Acquisition Systems - Ryan Rivera
QIE Irradiation BER Testing Data Acquisition Systems - Ryan Rivera
IHEP Telescope - Beijing Data Acquisition Systems - Ryan Rivera
T980 Crystal Collimation Telescope Data Acquisition Systems - Ryan Rivera
CAPTAN Future • Port CAPTAN Software to Linux • Explore parallel processing power • Integrate with xTCA Data Acquisition Systems - Ryan Rivera
Current Program • Optical Links • CAPTAN • xTCA • Mu2e Data Acquisition Systems - Ryan Rivera
xTCA • ATCA (Advanced Telecommunications Computing Architecture) • Spec put forth by PICMG (PCI Industrial Computer Manufacturers Group: a consortium of over 250 companies) • AMC plugs into ATCA card • Or MicroTCA crate • ATCA + MicroTCA = xTCA • Successor to VME? 12U 14-slot ATCA shelf Data Acquisition Systems - Ryan Rivera
Advanced Mezzanine Card (AMC) Data Acquisition Systems - Ryan Rivera
xTCA and CAPTANIntegration Data Acquisition Systems - Ryan Rivera
VME vs. xTCA +Number of new developments decreasing, sales still constant + Many I/O modules available − Developed in 1981 − Bus technology has speed limitations − Wide busses create a lot of noise in analog channels − No standard management at crate level − No standard management at module level − One damaged bus line stops entire crate Data Acquisition Systems - Ryan Rivera
VME vs. xTCA + Scalable modern architecture • From 1 slot MicroTCA to full mesh ATCA + 1 Gbps serial communication links + Standard PCIe and Ethernet communication+ Well defined management + Hot-swappable + Safe against hardware/software failures + 99.999% availability is possible Data Acquisition Systems - Ryan Rivera
VME vs. xTCA − Many crates and modules from different manufacturers • Is there a 'standard' ? − Learning curve to software/hardware specifications − Need functional MMC/IPMI code for power delivery − Imposed system management Data Acquisition Systems - Ryan Rivera
xTCA Future • Large experiments are considering xTCA over VME. • CMS • ATLAS • LHCb • Experiments at DESY • Dream is to develop a readout system for strips/pixels • Evaluate for future FNAL Experiments (CMS upgrade, Mu2e) • SBIR submitted with industry partners. • Vince Pavlicek is on committee to develop the PhysMTCA or MTCA.4 (MicroTCA for Physics) spec. Data Acquisition Systems - Ryan Rivera
Current Program • Optical Links • CAPTAN • xTCA • Mu2e Data Acquisition Systems - Ryan Rivera
Mu2e • Tracker • 21,600 Straws • 216 Readout Controllers • ~65 GBps* • Calorimeter • 2112 Crystals • 44 Readout Controllers • ~15 GBps* • * zero-suppressed data • Additional data (< 1 GBps) from • Cosmic Ray Veto system and • extinction/target monitors Data Acquisition Systems - Ryan Rivera
Architecture Alternatives • Streaming (this is the selected architecture) +less front-end hardware (no concentrators or L1 Trigger) +simpler design + flexible trigger −higher cost • Triggered +less back-end hardware (networking and processing) + lower cost + allows higher digitization rate (e.g., waveform digitizers) −fixed L1 trigger Data Acquisition Systems - Ryan Rivera
Implementation Alternatives Servers Standalone (rack-mount) DTC NIC EVB Switch DTC Servers EVB Switch PCIe DTC DTC Servers EVB Switch PCIe DTC (event building in server) NIC DTC Data Acquisition Systems - Ryan Rivera
Comparing Technology • To meet the demands of a 100 GBps streaming DAQ, Mu2e will leverage… • 10 Gbps Networking • High Density FPGAs • High Performance Processors • Fiber-optic Communications (synergy with Optical Links) Historical Perspective Data Acquisition Systems - Ryan Rivera
Design Philosophy • Commercial Slow Control hardware and software is used for development • needed early, can’t wait for custom HW/SW • hopefully, utility extends into production and running • LabVIEW or EPICS • Embedded Processors • use commercial modules if possible to allow early software development • embedded Ethernet and HTTP with eye towards debugging ease Data Acquisition Systems - Ryan Rivera
Mu2e Future • One year from starting prototype phase. • ~6 years to installation complete. • Mu2e funded by Mu2e, but components of DAQ architecture may apply to future experiments (Project X and others). Data Acquisition Systems - Ryan Rivera
DAQ Discussion Points • Program Resources • National Picture Comparison • Best Future R&D Avenues Data Acquisition Systems - Ryan Rivera
DAQ Program Resource Issues • Limited manpower • Has hindered xTCA effort • xTCA development accessibility • Community MMC design effort? • FPGA power/expense/expertise Data Acquisition Systems - Ryan Rivera