210 likes | 294 Views
September 2.001. ENGINEERING ETHERNET BASED SOLUTIONS. By the Software Support Team (GTD St.Genis) and the Cryogenics Team (GTD BCN). This presentation is based in GTD experience integrating control and supervision architectures for Science, Space, Defense and Industrial applications.
E N D
ENGINEERING ETHERNET BASED SOLUTIONS By the Software Support Team (GTD St.Genis) and the Cryogenics Team (GTD BCN) This presentation is based in GTD experience integrating control and supervision architectures for Science, Space, Defense and Industrial applications. Projects with relevant GTD involvement: XMM-Newton Space Telescope, Eurofighter, Arianne V launcher, Spaceport Control-room and Accelerators (Grenoble and CERN)
ENGINEERING ETHERNET BASED SOLUTIONS By the Software Support Team (GTD St.Genis) and the Cryogenics Team (GTD BCN) • SOLUTIONS: … to a problem. This presentation discuss suitable problems for ETHERNET application. • 2) ENGINEERING: No general solution (at least for the most relevant problems). Criteria to create suitable solutions, identify major constraints and defined the proper integration strategy are also introduced.
1.1 PROBLEM OVERVIEW Displaying Large amount of data in humanreal-time scale (i.e.: seconds) Registering Non real-time large amount of data with high time-stamp accuracy … … … Requesting Reduced amount of data in human time scale (seconds) Configuring/Backing-up Large amount of data non real-time event-driven (i.e. start-up) … … … the Supervision • Processing/Control • Variable amount of data in process real-time** scale (1ms – 500ms)* • Data Supply for: • - Displaying • Registering • … … … the Control Commanding Variable amount of data in process real-time** scale (1ms – 500ms)* … … … the Process Actuation Variable amount of data in process real-time** scale (1ms – 500ms)* … … … • Acquisition • Variable amount of data in process real-time scale** (1ms – 500ms)* • Data Supply for: • Processing/Control • Displaying • Registering • … … … *) Ranges for industrial (PLC based) control systems in GTD **) REAL-TIME: known, accurate - i.e. 1ms - & controllable
1.2 PROBLEM OVERVIEW Synchronous / Time[s] status Asynchronous the Supervision records: events, trends, … Asynchronous Asynchronous requests data supply sub-sampling the Control commands data supply Synchronous / Time[ms] the Process
TIME 1.3 PROBLEM OVERVIEW Max Data Quantity synchronous data Time Schedule Bw1 Bw i Bw j Bw n Non Real-Time Real-Time Max Data Quantity asynchronous data Time Limits Application Domain of Ethernet as Fieldbus acquisition processing known & fixed T ± T
2.1 THE ENGINEERING APPROACH THE CONCEPT ETHERNET approach TOPOLOGY URD • For Processes affecting multiple devices: • Nature (synchronous/asynchronous) • Real Time Constraints • Bandwidth (Bw) PROBLEM … … FUNCTIONALITIES may block (partially or totally) Ethernet applicability validated 1 on 1 ADD + complexity + performance + new functionalities + data availability*/richness + flexibility ARCHITECTURE
2.2 THE ENGINEERING APPROACH THE CONCEPT Synchronous(1(typically Closed Control Loops signals – “feedback” and “output”) Real Time Requirements 0< Tx <10ms Tiny Bandwidth Limited (Field)bus participants Local Processing (eventually by distributing processing effort) 10ms< Tx <50ms Limited Bandwidth Limited (Filed)bus participants Deterministic Fieldbus Device’s efficiency for TCP/IP packaging is limited under 50ms 50ms < Tx Ethernet Applicable (Bw occupation 1:10) for “determinisms” New topologies (system redesign!) Greater Bw Packets oriented transmission for Bw optimization (Protocol “Overhead”) Fieldbus still Applicable Less Bw available Less flexible topologies (several distance constraints) 1) Synchronous: Deterministic time control on processes concurrently running at different devices.
2.3 THE ENGINEERING APPROACH THE CONCEPT Asynchronous Real Time Requirements 0< Tx <10ms [i.e. Open Loop Control – Alarm Handling, Time-stamping, …] Tiny Bandwidth Limited (Field)bus participants * Deterministic Fieldbus > Polling / Packages > Slow! Local Processing (eventually by distributing processing effort) 10ms< Tx <50ms[i.e. Open Loop Information refresh] * “Messages” better than “token” * Token > Polling / Packages limited efficiency (overhead) TCP/IP with PROTOCOL + Tx Policy 50ms < Tx [i.e. (very) large systems for info refresh, buffered data, configurations’ up/download, …] * fully package oriented Ethernet Best Choice (Bw occupation 1:10) for “confidence” New topologies (system redesign!) Greater Bw TCP/IP with PROTOCOL + Tx Policy
2.4 THE ENGINEERING APPROACH THE CONCEPT Recommendable … at the limit Not recommendable ETHERNET TCP/IP ASYNCHRONOUS suitability Deterministic FieldBus SYNCHRONOUS Real Time Requirements 0 10ms 50ms 100ms for Increasing Bandwidth
3.1 THE ENGINEERING APPROACH ENGINEERING A SOLUTION ETHERNET TCP/IP Industrial Control & I/O Devices 10MBits 100MBits • But … … … • What is the effective Bandwidth occupation? • What are the constraints? • How shall the information exchange be organized?
3.2 THE ENGINEERING APPROACH ENGINEERING A SOLUTION throughput More data managed (maybe not more knowledge!) 7 APPLICATION ENGINEERING: Superprotocol + BW management Information Organization Some constraints: somehow inherited structures High Level Protocols Application Layer Middleware Services 6 Presentation Some immaturity adding TCP/IP services to certain PLC’s Operating Systems. 5 Session TCP/IP 4 Transport 3 Network Some immaturity degrees in the firmware connected to proprietary architectures. (is it becoming a PLC weakness?!?) hardware Ethernet (IEEE 802.3) 2 Data Link 1 Physical
3.3 THE ENGINEERING APPROACH ENGINEERING A SOLUTION A case study ... the Control System for the LHC Cryogenics FIRST IDEA ... Derived literally from Specification Visualization Synchronous ~2s <50.000 Binaries <22.000 Floats ( 700.000 Binar.) ( 300.000 Floats) Events Asynchronous <acceptable delay from 700.000 Binaries Max 500 events/s ( 4.000 events/s) Asynchronous Manual Requests ~1sHuman Feeling < 3.500 Binaries ( 155.000 Binar.) supervision layer 3 Synchronous Process Ts=500ms (150ms) < 9.000 Binaries <15.000 Floats ( 275.000 Binar.) ( 105.000 Floats.) Synchronous Process Ts=500ms (100ms) < 3.500 Binaries < 3.500 Floats ( 155.000 Binar.) ( 160.000 Floats.) 1 2 control layer Synchronous Time Stamping 10ms <9.000 Binaries ( 275.000 Binar.) Bw~8MBits Bw~10,5MBits field layer
3.4 THE ENGINEERING APPROACH ENGINEERING A SOLUTION A case study ... the Control System for the LHC Cryogenics IMPLEMENTATION (as it is today!) * 3 X 8 (redundant) Visualization Asynchronous (worst case 2s) <50.000 Binaries <22.000 Floats ( 700.000 Binar.) ( 300.000 Floats) * Events Asynchronous <acceptable delay from 700.000 Binaries Max 500 events/s ( 4.000 events/s) Asynchronous Manual Requests ~1s Human Feeling < 3.500 Binaries ( 155.000 Binar.) 4 * 2 X 64 TIME STAMPING SAMPLING Process Asynchronous (worst case 150ms) < 9.000 Binaries < 5.000 Floats ( 275.000 Binar.) ( 105.000 Floats.) Process Asynchronous (worst case 100ms) < 3.000 Binaries < 2.000 Floats ( 155.000 Binar.) ( 160.000 Floats.) 2 x TCP/IP cards * 1 TIME STAMPING X 165
3.5 THE ENGINEERING APPROACH ENGINEERING A SOLUTION A case study ... the Control System for the LHC Cryogenics summarizing the chosen approach … • Distribution of the Time-stamping process. Time-stamping at the source eliminates synchronous communication. • >>> It is feasible to move to a full ETHERNET TCP/IP solution • >>> Requires specific Supervision system • Duplication of networks at the Control Layer to make synchronous exchanges “deterministic” and/or reduced collisions (moreover, dimensioning of the number of devices) • >>> Measure performances of the Hardware • Eliminate unnecessary throughputs by directly addressing the information item to the target (favored by the flexible Ethernet architecture) • 4) Bandwidth Management: Synchronous >>> Asynchronous! • (further explained)
3.6 THE ENGINEERING APPROACH ENGINEERING A SOLUTION A case study ... the Control System for the LHC Cryogenics HARDWARE evaluation (example … at the Control Layer) Performance (Bw)[Kbit/s] for n parallel managed communication’s ports/channels. Curves represent different PLC cycles (40ms .. 100ms) TCP/IP 100MBits Open Modbus CPS211 00 CPU534 14 NOE771 00 NOE771 00 Percentage of the PLC cycle time occupied by communications related tasks [%] for n parallel ports/channels and different PLC cycles.
3.7 THE ENGINEERING APPROACH ENGINEERING A SOLUTION A case study ... the Control System for the LHC Cryogenics HARDWARE evaluation (example … at the Control Layer) Good conditions found for 16 parallel managed ports/channels in a 50ms PLC cycle, thus resulting in approx. 256 Kbits/s of bandwidth and less than 20% PLC cycle occupation due to communication’s management (approx. 10ms). Duty Cycle / Band Width ratio for different number of parallel opened Ports/channels (MASTERS) Duty Cycle / Band Width ratio for different PLC Cycles.
3.8 THE ENGINEERING APPROACH ENGINEERING A SOLUTION A case study ... the Control System for the LHC Cryogenics PROTOCOL & BANDWIDTH MANAGEMENT Event Driven Protocols / Tx Policies: >>> Oriented to Asynchronous strategies a) People use to neglect “worst case” conditions b) Steady conditions shall be carefully understood >>> Event Driven use to be optimized for reduced pieces of INFO Worst conditions use to mean: large quantities! >>> Optimization is achieved by the probabilistic fact of “little changes” “worst case” and “close to worst case” are not negligible >>> Oriented to Synchronousstrategies It is feasible by: a) Dimensioning for the worst case b) Sampling to regularize data flow >>> What is the advantage? Bandwidth Management a) To support concurrent, less demanding, Asynchronous communication processes. b) To reduce collision probability in steady conditions Ts Ts Ti process Tk
3.9 THE ENGINEERING APPROACH ENGINEERING A SOLUTION A case study ... the Control System for the LHC Cryogenics IMPLEMENTATION (as it is today!) Example PCU: Quantum 534 14 + 2x NOE 771 00 Event Driven (by GTD) on Open Modbus Steady “worst conditions” (= Polling) Bw = 256 Kbit/s available - needed for Superv. 200KBit/s - needed for Process.160KBit/s Better than “worst conditions” - Bandwidth availability for true Asynchronous processes. 8 Redundant Data Servers (Bi-Pentium) Requests Events Visual Maximum 8 Control Units (PCUs) for each DS Events Visual Trends Requests TIME STAMPING SAMPLING Process Process Process Requests Process Events Visual Trends Maximum 4 Field Interfaces (FIs) for each PCU TIME STAMPING
ENGINEERING ETHERNET BASED SOLUTIONS conclusions 1. Synchronous-Asynchronous 2. Data Quantity 3. Time Frame Our recommendation: Try to use Ethernet for 1. Synchronous, over 75ms 2. Asynchronous, over 25ms Still difficulties: 1. Certain immaturities 2. Inherited standards 3. Large/Complex applications (more than necessary !?!) > Engineering Effort a) To adapt resources b) To create Tx policies c) To benefit from Ethernet added values. Case Study: Characterized by … 1. Application “size” 2. Information flow 3. Synchronous/Asynchronous Ethernet TCP/IP not well suited for synchronous, fast processes