130 likes | 139 Views
This paper discusses the use of System-on-Chip (SoC) technology, specifically the Xilinx Zynq-7000, for embedded control and interlocking in fast pulsed systems. Two projects are presented, one involving the integration of a fast interlock detection system within the CERN control framework using Embedded Linux, and the other implementing a lower-tier communication bridge for timing and analog control logic.
E N D
SoC technology for embedded control and interlocking within fast pulsed systems P. Van Trappen, N. Magnin, J. Schipper, M. Gauthier, E. Carlier, E. Oltedal System-on-Chip Workshop – CERN https://indico.cern.ch/event/799275/timetable/
Abstract Document reference • The control of CERN’s beam-transfer kicker magnet high-voltage pulse generators requires often the use of digital control in FPGAs to allow tight timing control (jitter better than one ns) and fast protection of the high-voltage thyratron and semiconductor switches. A FPGA is a perfect candidate for the digital logic, it is however limited in integration possibilities with the actual and future control systems. • TE-ABT has recently levered the market push for integrated devices, so called Systems on a Chip (SoC). This technology allows for a tightly coupled ARM (Cortex A9 in case of the Xilinx Zynq-7000) processing system and programmable logic in a single device thus avoiding difficult placement and routing on an electronic card. We present here two projects where the Xilinx Zynq-7000 has been used: • It is used in the fast interlock detection system to integrate it neatly within the CERN control framework, by using Embedded Linux and running a Snap7 server. • In the second project it implements a lower-tier communication bridge between a front-end computer, using serial communication in software and high-fan out multiplexing in the programmable logic for timing and analogue control logic.
Project 1 description Document reference Fast Interlocks Detection System (FIDS) • Single hardware replacement for a wide variety of old, often obsolete, ABT hardware that implements the fast-interlocking functionality, i.e. detect and act on faulty behavior of fast power-switches (thyratrons, GTO-stacks) for kicker installations that the CERN accelerator complex • Choice for a solution based on the Xilinx Zynq-7000 SoC on a custom card because: • Fast reaction times (<100 ns to retrigger) and variable comparator thresholds require a FPGA • VME-based solution would require development of a custom-card anyway; VME considered soon-to-be-obsolete (bandwidth sufficient); complex and costly architecture with separate CPU card and crate • No integration of a SBC to keep design and firmware 100% in our control (and free) • Use of integrated CPU allows: • Implementation of an embedded (SILECS) Snap7 server for operational communication • Reduced software development time because of peripheral device drivers for Embedded Linux and use of C / C++ / Python e.g. i2c driver for external path panel • Automated test-procedure running on the card itself; permanent diagnostics running on-board (Python) • Integrated digitizer & analysis functionality: streaming FPGA data to the CPU using DMA • Status: 35 boards produced, 10 being used >1 year without problems; LS2 operational installation started
Project 1 description Document reference Fast Interlocks Detection System (FIDS) Some more info on the SILECS VC (Virtual Controller): • SILECS = Software Infrastructure for Low-level Equipment Controllers (CERN) https://wikis.cern.ch/display/SIL/SILECS+Home • Set of libraries and a configuration tool for FEC <> PLC/hardware communication • Ensures a common memory map declaration, with checks on both sides • Virtual Controller = embedded stand-alone server process, supports ARMv7 compilation • Uses the Snap7 TCP/IP protocol, originally for native communication with Siemens S7 PLCs http://snap7.sourceforge.net/; also supports ModBUS-TCP. • Works well but requires polling • Possible improvement: use UPC-UA, industry-standard middleware, with a subscription model (issue with feature request at SILECS gitlab) • Newest PLCs support it by default • Open-source OPC-UA libraries exist for the VC • Improved security by encryption (S7 relies on ISO-TSAP which is old and possibly insecure - link) • Allows authentication (now anyone can create a socket and read/write to a Siemens S7 PLC)
Project 1 description Fast Interlocks Detection System (FIDS) – main controller board FASEC https://www.ohwr.org/project/fasec/wikis/home • Miscellaneous • UCD90120ARGC power controller (programmable over JTAG) to survey power rails and manage power-on and power-off sequence • Xilinx-style JTAG connector • 2x hexadecimal switches (for settings selection) • UART over a Micro-USB connector (FT232) • Front panel • 6x LEMO-00 analog input (+-10V range; 200 KSPS) • 1x SFP port (White Rabbit compatible) • 2x LEMO/SMC inputs (5V) • 2x LEMO/SMC outputs capable of driving 3.3V @ 50 ohm • 1x USB3.0 A connector for high-speed serial card-to-card GTP link (no USB functionality!) • 8x Programmable LED (4 by PL, 4 by PS) • POR Reset push button • Back panel • 2x 100 Mbit Ethernet RJ45 ports (1 interface, switched) • RJ11 connector for FIDS patch panel (isolated I2C)_ • D-Sub 15 connector with isolated 4x fail-safe NC contact channels and 2x 24V input channels • 10-layer PCB • Not reproduced outside of this project, however there’s a fork in the making at ESRF (European Synchrotron Radiation Facility, Grenoble) – a RFoWR project called CITY Document reference • XC7Z030 controller, SoC with Kintex-7 fabric (called PL, i.e. Programmable Logic) and Dual ARM Cortex-A9 MPCore at 1 GHz (called PS, i.e. Processing System) • Two Low-Pin Count FMC slots • FMC1 connectivity: Vadj fixed to 2.5V, 34 differential pairs, 1 GTP transceiver with clock, 2 clock pairs, JTAG, I2C • FMC2 connectivity: Vadj fixed to 1.8V, 34 differential pairs, JTAG, I2C • FPGA configuration • From QSPI flash, Ethernet (through U-Boot bootloader) or MicroSDcard • Clocking resources • 1x 33.33 MHz fixed oscillator, SoC main clock (clock distribution to PL possible) • 1x 125 MHz fixed oscillator for the FPGA fabric • 1x 25 MHz TCXO controlled by a DAC with SPI interface (AD5662, used by White Rabbit PTP core) • 1x 20 MHz VCXO controlled by a DAC with SPI interface (AD5662, used by White Rabbit PTP core) • 2x low-jitter frequency synthesizer/fanout (TI CDCM61004, fixed configuration, Fout=125 MHz, used by White Rabbit PTP core) • On-board memories • 2x 512 MByte (4 Gbit) DDR3L (MT41K256M16HA-125:E) • 2x 128 Mbit QSPI flash for FPGA bitstream and Linux kernel & root file system storage (S25FL128SAGMFIR01)
Project 1 description • Gateware & software development workflow: • Gateware with Vivado, using git submodules for HDL libraries • Using the typical Embedded Linux development workflow: • FSBL, U-Boot, kernel and rootfs cross-compiled on host (PetaLinux / Yocto OE) • Initramfs from SD card loaded to memory • Sometimes NSF rootfs during development Document reference Fast Interlocks Detection System (FIDS) Project gateware block diagram:
Project 1 description Document reference Fast Interlocks Detection System (FIDS)
Project 2 description Adapter board sbRIO Document reference G-64 eradication project Several ABT kicker installations still rely on G-64 hardware (bus & CPU) for which all hardware has become obsolete. No guarantee to keep the Motorola 6809 C-compiler working on modern hosts! No resources to consolidate the full control system, hence keeping all hardware but the PS/PO CPU card and adapters, replacing it by a NI sbRIO 9607 housing a Xilinx Zynq-7000 We designed a simple adapter board (EDA-03907) for power supplies, IO buffers and mechanical integration Motivation: RS-232 communication to FEC (CPU), multiplexing >60 outputs to the existing ABT electronic cards without changing crate and/or or hardware (FPGA) Also: project engineer with LabView expertise Future improvement: eradicate RS-232 and use Ethernet communication Status: prototype fully tested; 30 boards produced for LS2 operational installation
Project 2 description • Gateware & software development workflow: • Gateware with NI LabVIEW, not using any HDL (direct use of Vivado not supported for the sbRIO9607) • Not your typical Embedded Linux development workflow: • Using precompiled FSBL, U-Boot, kernel and rootfs • Mounting a host NSF directory for software development • Cross-compiling on CERN BE/CO VPC machines, Linaro cross-compiler at /acc/local/share/abt/ARM7/ Document reference G-64 eradication project Different gateware and software development workflow, because of National Instruments’ environment (‘NI Linux Real-Time’)
Environment Here are we today… Document reference • Support needed, preferably CERN-wide: • Development environment (BE/CO Makefile with ARM CPU target, common kernel, rootfs, uboot) • Boot solutions and server (TFTP, BOOTP) • Especially required for a quick and easy board swap during operation! • Centralized monitoring and syslog support (Kibana, BE/CO COSMOS) • Would happily use CERN CentOS 7 (CC7) if it is managed centrally, supports 32-bit, and a recent kernel is used to limit the drivers porting effort
Environment Document reference Today ABT has several FIDS boards connected to the GPN & TN, see cfe-86% (e as embedded): • running PetaLinux 2018.3 with a patched xlnx-kernel 4.14; • Poky rootfswith own kernel-space drivers and user applications; • limited effort done to reduce attack vector (ftpd and telnetd disabled); • boots from SD card; no common compiler, sources or boot-server! How to manage NFS mount authorization? Nmap scan, disabled busyboxftpd and telnetd: Starting Nmap 6.40 ( http://nmap.org ) at 2019-06-12 00:11 CEST Nmap scan report for cfe-867-mkbidisfids (172.18.192.199) Host is up (0.00049s latency). rDNS record for 172.18.192.199: cfe-867-mkbidisfids.cern.ch Not shown: 997 closed ports PORT STATE SERVICE VERSION 22/tcp open sshDropbearsshd 2017.75 (protocol 2.0) 53/tcp open domain dnsmasq 2.78 | dns-nsid: |_ bind.version: dnsmasq-2.78 102/tcp open iso-tsap 111/tcp open rpcbind | rpcinfo: | program version port/proto service | 100000 2,3,4 111/tcprpcbind | 100000 2,3,4 111/udprpcbind | 100024 1 42952/udp status |_ 100024 1 53763/tcpstatus Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel