1 / 38

The architecture of LUX Trigger Control Software November/06 2007

The architecture of LUX Trigger Control Software November/06 2007. Wojtek Skulski LUX Collaboration University of Rochester skulski _at_ pas . rochester . edu. The goals of this presentation.

Sharon_Dale
Download Presentation

The architecture of LUX Trigger Control Software November/06 2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The architecture of LUX Trigger Control Software November/06 2007 Wojtek Skulski LUX Collaboration University of Rochester skulski _at_ pas . rochester . edu LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  2. The goals of this presentation • I will present the architecture and software components which I used to develop several DAQ programs under BlackBox Component Builder. • The DAQ programs ran for thousands of hours in mission-critical experiments without either crashing or leaking a single byte of memory. • The architecture can be used to develop the LUX Trigger Control software running under BlackBox Component Builder. • In order to make this presentation self-contained, I am also explaining the hardware architecture of the LUX trigger boards. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  3. Outline • Development of BlackBox DAQ & control software: 2001-2007. • Duties of the LUX trigger DAQ and control system. • Data display and monitoring. • Board control. • Reconfiguration of FPGAs. • Architecture of the LUX trigger system. • Architecture of the BlackBox DAQ & control system. • Parts of the BlackBox DAQ & control system: • Object management POM. • Waveform graphics Gr. • Scientific Library Lib (graphics, matrices, special functions, digital filters, etc.) • DAQ script DataLogger. • Board control GUI panels : see my “without a sting” presentation • Low-level hardware interface libraries: see my “without a sting” presentation. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  4. DDC-x development system under BlackBox Analog signal reconstruction: digital FIR filter output Control & waveform display: BlackBox and Gr DDC-8 NIM pulser LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  5. Development of BlackBox DAQ & control software: 2001-2007 • I have been developing and refining the BlackBox-based DAQ and control since 2001 with my DDC-x waveforms digitizers. • Several students used and extended the DDC software. • I use the same DAQ software package at LLE on a daily basis with the following hardware: capacitance sensors (interfaced via the same USB controller as the DDC cards), VME-based distance-measuring interferometers, National Instruments A/D cards, sliding-scale sensors (interfaced via a serial port), humidity and temperature sensors (interfaced via a serial port). • All my experiments work smoothly. The BlackBox software never caused any significant problems or delays in my experimental work. • Software components described in this presentation were tested over thousands of hours in mission-critical experiments, some involving hundreds of thousand $ worth of equipment. • BlackBox is the best and the most robust development software I have ever used. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  6. HW architecture of the trigger The boards are described to give the context. Block diagrams of the trigger board connections. Block diagrams of the boards (the DDC and the EB). LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  7. Hardware architecture of the LUX trigger boards • LUX trigger is composed of two kinds of boards. • The front-end DDC-8 LUX is a waveform digitizer and signal processor. • The back-end Trigger Event Builder is a communication board. • Both boards use the same programmable chips: • Xilinx FPGAs perform real-time signal processing. • Cypress microcontroller manages the FPGA and provides the USB-2 data link with the PC. • PC is running BlackBox control software, which monitors and controls the trigger boards. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  8. Trigger system architecture Trigger consists of several DDC boards and one Event Builder board Up to 8 front-end digital pulse processors A single Level-2 decision board (Trigger Event Builder) NIM pulses to DAQ system (event yes/no) Data readout USB 2.0 50 MB/s to the PC Fast data links Up to 200 MB/s per link LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  9. Architecture of the front-end DDC boards The new DDC-LUX board will also have a fast data link, 200 MB/s Xilinx Cypress LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  10. Architecture of the Event Builder boards The Trigger Event Builder is a new board designed for the LUX trigger Clock IN * 1 NIM IN * 5 NIM OUT * 4 Eight fast data links connected to DDC Up to 200 MB/s per link Xilinx FPGA Cypress mcontroller USB 2.0 50 MB/s to the PC LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  11. Three development systems for LUX board programming PC Trigger boards FPGA configuration (VHDL) Xilinx ISE/WebPack 1 Xilinx FPGA mcontroller program (C) Cypress EZ-USB development SW 2 Cypress mcontroller DAQ and control SW Component Pascal running under BlackBox Component Builder Cypress mcontroller is running board control firmware 3 Board control datagrams LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  12. On-board firmware development for trigger boards • FPGA development software. • The FPGA is a massively parallel signal processor with 20 parallel multiply-accumulate units. It implements real-time signal processing and decision logic. • The original DDC-8 firmware was developed in 2004 and extensively tested. The current LUX firmware is built upon this original firmware. • Xilinx FPGAs type Spartan 3E-500 are programmed under Xilinx ISE WebPack. • Currently we use high-level language VHDL to implement the firmware. Other approaches (e.g., graphical System Generator) are possible, but not used yet. • Microcontroller development software. • Cypress mcontroller manages the FPGA and provides the USB-2 data link with the PC. Its processing power is relatively low (about 4 MIPS). • The mcontroller is programmed in C. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  13. LUX Trigger Control software architecture Running on a PC under BlackBox Data display and monitoring Board control Remote FPGA reconfiguration LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  14. Duties of the LUX Trigger Control software • Data acquisition and writing to disk (optionally). • Data display and monitoring. • One-dimensional waveforms (raw data from the ADC). • Transformed waveforms (e.g., smoothed or filtered with some sort of “matched filter”). • Two-dimensional correlations (e.g. time versus amplitude). • Scalers (e.g., numbers of rejected and accepted events). • Board control. • Thresholds, digital filter parameters, input DC offsets, etc. • Several boards have to be controlled. • Reconfiguration of the trigger system. • The trigger operation is almost entirely defined by the FPGA. The FPGA has to be “loaded” before the DDC boards can operate. • The FPGA configuration has to be performed remotely, without unplugging the DDC boards. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  15. Components of LUX Trigger Control Software • Graphics. • Integer-valued 1D waveform graphics Gr (raw ADC data). • Real-valued 1D vector graphics Lib (transformed data). • Real-valued 2D matrix graphics Lib (correlations, e.g., amplitude vs time). • Object management. • A package named Persistent Object Manager (POM) manages the above-mentioned graphical objects. • Numerical values such as scalers. • Numbers can be entered or shown in graphical panels. • Data-acquisition script. • A “script” for performing DAQ is compiled and runs at full speed. • USB-2 communication with trigger boards. • The component for USB communication is very well tested. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  16. Diagram of LUX Trigger Control Software Simplified structure of DAQ software running under BlackBox. Persistent Object Manager POM Manages graphical objects on screen. Saves histograms and matrices to disk. Data Logger acquisition “script” Compiled, runs at full speed. Management Histogramming Waveform graphics Gr Shows raw ADC data. Real-valued graphics Lib Math and graphics library. Shows transformed data and 2D matrices. Readout USB communication Receives data from the trigger boards. Sends control commands to the boards. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  17. Presentation of the software components which can be used in the Trigger Software Gr, POM, Lib, and the DAQ “script” LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  18. Waveform graphics Gr* Easy to use histogram and waveform display package. Designed for data acquisition (DAQ). Extensively used for DAQ display: DDC-1 and DDC-8 waveform digitizers, capacitance sensors, temperature and humidity monitoring, CAMAC, and student projects. *Developed by Wojtek Skulski. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  19. Why Gr was needed? Why needed: A typical DAQ program uses either histograms, waveforms, or time “histories” to record integer-valued ADC data. I found BlackBox Component Builder a tremendous tool to develop DAQ software. However, the waveform graphics was missing when I started using BlackBox circa 1999. Therefore, I decided to develop the histograming and display package for BlackBox. As a basis for development I used my previous package developed for Oberon System-3, because the two languages (Oberon and Component Pascal) are very similar to each other. I finished the development and released Gr to public domain in September 2001. To my own surprise, I have been using Gr in its almost original 2001 form ever since its release. I made only very few additions to the original version. It turned out that the design principle “make it as simple as possible, but not simpler” has yielded a software package which is useful, robust, and maintainable. The Gr package became my “secret weapon” which enabled me to develop robust DAQ software for my many projects. Students, who also used Gr, have found it useful for their own projects as well. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  20. When to use Gr • When to use Gr: • Integer-valued data, e.g., acquired from a digitizer. • One-dimensional data. E.g. temperature recordings, etc. • The data is not an image (which would require 2D display). • The display has to be “live” and interactive. • Convenient zooming and scaling is implemented. • When to use Lib rather than Gr: • Real-valued data (64-bit floating point numbers). • Two-dimensional display (e.g., time vs amplitude). • Lots of math, matrix algebra, or vector operations on data. • Special functions and/or digital filtering. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  21. Gr has a reasonable GUI and is very easy to use. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  22. Several DAQ examples and a tutorial are provided with the Gr package. Gr User Guide Example data acquisition panel. Detailed instructions how to use the Gr package. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  23. Persistent Object Manager POM* It manages and organizes any objects. Designed for organizing data acquisition display. Extensively used in mission-critical experiments: Tiled-grating assembly, capacitance sensors, temperature and humidity monitoring. *Developed by Daniel Miner (with some W.S. help). LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  24. Why POM was needed? Why needed: A typical DAQ program may use a large number of histograms, waveforms, matrices, or other data objects. Typically, the histograms are initialized and incremented under program control. Users want to display the content of these objects while the DAQ program is running. (The “post-mortem” examination is inconvenient, though often used.) This poses a problem of “dual ownership” of the data objects, which are “owned” by the DAQ program, but need to be displayed by the GUI. The Persistent Object Manager (POM) was developed to implement this feature. The DAQ program will be running, processing data, and incrementing the histograms, and yet you can display and save the histograms without stopping the DAQ. The histograms are managed as a “tree” (similar to a Windows Explorer). You can double-click on any histogram’s name, and you can examine its content. You can also save the entire tree. Not just histograms: any object whatsoever can be managed by POM when the wrapper for this object is available. We plan to write wrappers for several types of objects, such as vectors, matrices, etc., as needed by the LUX Trigger project. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  25. Graphical object management of Gr histograms POM organizes Gr histograms into a tree structure. We will implement management of other arbitrary objects, as needed. Application -specific panel Histos POM windows LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  26. Another POM screenshot Show another POM screenshot from the LLE experiment Point out that POM was used for thousands of hours and never crashed LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  27. POM “to do” list • Currently, POM can only manage Gr objects. • The “Gr wrapper” is implemented. • Other objects (e.g., Lib objects) need to be managed as well. • We need to develop the wrappers similar to the Gr wrapper, using the Gr wrapper as the template for development. • It is very straightforward to develop such wrappers. • Any object can be managed by POM when the wrapper is available. • The most immediate need is for Lib objects (real-valued vectors and matrices). • A text object (implementing a formatted BlackBox text) will be a very attractive addition. Such text objects can hold run-time comments and remarks, including graphics. Using the text objects, the experimenter’s comments will never be lost. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  28. DAQ DataLogger “script”* It receives and processes experimental data. Designed for organizing data acquisition process. Extensively used in mission-critical experiments: Tiled-grating assembly, capacitance sensors, temperature and humidity monitoring. *Developed by Wojtek Skulski. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  29. Why the “script” was needed? Why needed: Writing a DAQ program may seem straightforward. Just write a piece of code with whatever actions you want... However, the DAQ development will benefit from having a template program divided into typical sections, such as initialization of histograms, the DAQ “subroutine”, and possibly others. The Data Logger “script” was developed to help writing typical DAQ programs. There are sections to initialize histograms, a hook to perform the DAQ actions, and the interface to the BlackBox multitasker, which makes the DAQ run. And of course, the Data Logger uses POM to manage histograms (and other data objects in the near future). NB: The “script” is not a script. It is a compiled “module”, which means two things: (1) It runs at full speed. (2) The compiler performs full consistency checking with the rest of the DAQ system. I use the name “script” because functionally it is like a script, but its implementation is vastly superior to any kind of a scripting language (such as cint, I beg your pardon). The LUX Trigger software will be another example of this “script”. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  30. Example experiment using the Data Logger + POM This experiment was performed using POM and the Data Logger “script” Example data: interferometer history recordings (popped up by clicking POM) Data Logger panel (start/stop, etc.) Interferometer panel (application-specific) POM panel LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  31. Scientific and engineering library Lib* 1D and 2D display package. Matrix and vector calculus, digital filters, and special functions. Designed for data analysis, theoretical calculations, and modeling. Extensively used for mission-critical applications. Airborne antenna systems (BAE Systems), adaptive optics (UofR Laser Lab). *Developed by Robert Campbell, BAE Systems. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  32. Why Lib is needed? Why needed: Writing a numerical program may seem straightforward... Until you actually start doing it. Then you realize you need a library. Lib is a tremendous computational library with over 6000 items. It covers vector and matrix calculus, Fourier transforms, special functions, digital filtering, and many other areas. In addition, Lib provides a very decent plotting utility which can render both 1D and 2D data. The latter is of course of interest to us (e.g., time-amplitude correlations). Lib graphics and Gr graphics are complementary. I mostly use my own Gr graphics to display raw data. However, when it comes to showing processed data, I often use Lib. And of course, when I need to plot 2D data, I always use Lib because Gr only provides 1D display. NB: You may ask why there are both Gr and Lib graphics? Part of this is history. The two packages were developed independently. However, as I often use both, I appreciate that Gr is great for displaying raw data, while Lib has a more “official look”. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  33. When to use Lib libraries • When to use: • Real-valued data (64-bit floating point numbers). • Two-dimensional display (e.g., camera images, or time vs amplitude plots). • Lots of math, matrix algebra, or vector operations on data. • The list of subsystems: • Lib - Engineering & Scientific library, including 1D, 2D, and 3D plotting. • Algebra - Computational Algebra library. • Multi - Arbitrary ultra high precision arithmetic. • Filter - Digital and other filter design and analysis. • Nav - Navigation & Geodesic Coordinate transform modules and tools. • Wands - A collection of general purpose tools (Wand: a tool used by a magician). • Cpc and Ctls- several general-purpose tools and utilities. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  34. Lib: math and plotting facilities • Vectors & Matrices, including 0-length cases. • Polynomial forms of vectors, matrix determinant, inverse, singular-value decomposition, etc… • Complex numbers and complex functions. • Special Functions & curve fitting. • Random numbers: several r.n. generators. • Numerical methods. • Numerical integration, interpolation, root finding, and minimisation. • 1D, 2D, and 3D plotting. • Includes interface to OpenGL. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  35. Scientific plotting and computation library Lib LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  36. Scientific plotting and computation library Lib LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  37. A mission-critical application using Lib Adaptive optics control package, including hardware control. LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

  38. Conclusion: Tools to develop LUX Trigger Control Software The tools are ready: Gr, Lib, POM, Data Logger. Other tools are available from BlackBox community (e.g. web server). Several experiments were performed using these software tools. The experiments ran successfully for thousands of hours. Some development will be required (as is always the case). LUX Trigger Control Software. Wojtek Skulski University of Rochester, November 2007

More Related