Semiautomatic PLC Program Logic Verification, Validation and Debugging While Still Developing It

1. Semiautomatic PLC Program Logic Verification, Validation and Debugging While Still Developing It PLC Program Logic Error Hunter Product Presentation ITTECH Pty. Ltd. www.ittech-automation.com.au 2005 - 2012 Tony Simeonov

2. Targeted At PLC Program Developers • This is a tool aimed at designers of industrial applications using PLCs. It helps them debug, analyze and deliver, free of logical errors PLC application programs prior to installation and on site testing. Design and Development Phase Verification Phase Installation and Testing Development finished, go ahead and verify the program logic The PLC program development flow imposed by the tool. The idea for the verification method used in this tool is borrowed from electronic chip design where zero logic error tolerances are allowed in the design. Test engineers verify a PLC program, by just specifying logic conditions (properties of the logic) that need to be fulfilled by the program while it is executing. The tool will then automatically (by the push of a button) check whether the specified properties are valid and render verification results to the user (in the form of execution traces) for properties being violated by the program logic. Error found in verification, go back and correct the logic of the program 2005 ITTECH Pty Ltd

3. Intended To Be Used By PLC Program Maintainers • The product is also aimed at PLC application maintainers to help them analyze existing logic problems in an installed application, fix it and then run existing regression tests in order to ensure that the overall logic of the program is still in tact. Maintainers would also need the tool in order to modify and verify the logic of an existing application in case it needs to be changed to support additional equipment in an installation. Verification and Regression Testing Phase Run regression tests with the Logic Hunter Reinstallation & Testing Modification Phase 2005 ITTECH Pty Ltd

4. Metal Processing Silo Control Petrochemical & Rafinaries Water Treatment Paper Production Area of Product Applicability • The PLC program being developed, could be for any kind of industrial automation application one can think of (e.g. process control, machine control, automated production lines, HVAC, oil refineries, water supply/treatment etc.) • It is suitable for the verification of all kind of behavioral logical patterns (cyclic behavior or logic executing along a logical path in steps (state machine logic, sequences) or logic executing in parallel logical paths , S88 DCS type applications etc.) The PLC Program Error Hunter can be used in any kind of industrial control application 2005 ITTECH Pty Ltd

5. The Benefits From Using This Particular Tool • The PLC Program Error Hunter is a very cost-effective solution used for verification, validation, debugging and regression testing or in general for automated factory acceptance testing reducing the time to test and develop a PLC program and significantly increasing its reliability. • While verifying the logic in the development phase, you also create a regression test suite used all the way along in the rest of the lifecycle of the application. Maintenance Phase Verify logic of design and create regression test suite in a purely software environment Verify changes in the program logic with the regression test suite all the way along in integration, installation, validation and maintenance. 2005 ITTECH Pty Ltd

6. Typical faults • Wrong or missing interlocks • Incorrect program logic • Incorrect processes • Incorrect S/R of steps, incorrect jumps • Incorrect S/R of flags/signals • Multiple use of variables • Other Errors (typing errors) Overall Duration of project Starting up Control Program Approx 90% of time spent on electronics and control program Up to 70% of time spent on debugging program errors Approx 15% • Late system evaluation leads to large losses of time. • Only 20% of the time used during start up is used to gather information. The Benefits From Delivering Error Free Applications Before Installation • From a purely human prospective, it means significantly less headaches for PLC program developers. They can avoid situations where you need to deal with nasty surprises during commissioning. • Through the Logic Hunter, the developer gets familiar with the behavior of the program very early in the design phase and mitigates the development risks. • It saves the embarrassment and risk of damaging the automated system, because of undiscovered logic errors. • The end result is reduced overall costs of development, boost of productivity and dramatically increased reliability and quality of the delivered PLC program. If logic errors are found while the product is still under design and development you would spend only your own time, while discovering errors during commissioning waists the time of all the other parties involved in the project. • A vast number of investigations have shown that it is the software of control systems, which is responsible for the majority of errors in the phase of deployment as shown by the diagram below. 2005 ITTECH Pty Ltd

7. Disadvantages of Testing On The Installation • There is a reason for those numbers shown above: - Currently there is no reliable methodology on the market which allows to verify a PLC program before it is being delivered for installation. It is only possible to test and verify the program in real working conditions on the PLC itself and usually when the installation part of which is the PLC has been deployed to the customer. • Lets have a look at the disadvantages of testing on the installation in real conditions: - As mentioned in the previous slide, late discovery of logic errors is very costly because everyone else on the project needs to wait for you. - Can all possible paths in the logic of the program be tested in real conditions? It is hard to identify all possible test scenarios and then some tests may be destructive for the installation under test. As an example, imagine you want to test the functionality of a limit switch automatic motion freeze on a CNC machine. If the logic doesn’t work as expected you might damage the machine if testing it in real conditions. - Testing in real conditions also leaves the program vulnerable to undiscovered bugs which may expose themselves later on when the project has been deployed and the installation starts being used. Such kind if logical problems may cause major breakdowns in the installation leading to disastrous results. Don’t mess around. Don’t test on the facility. 2005 ITTECH Pty Ltd

8. The Disadvantages Of Testing With A Simulator • Well then, is there an easy way to verify that the logic of your PLC program is error free before deploying it. The only pre-deployment testing and logic verification method that could be used, is to simulate the program and the behavior of the equipment involved. • Then you need to identify test scenarios which prove that the logic of the program is error free. • You need to run the scenarios on a PLC program simulator by manually navigating the simulation with the test bench. • This is a tedious process and it does not guarantee that the PLC program logic will behave correctly in all cases after testing it. Simply it is impossible for a human to identify all scenarios where the program can go wrong. Because of this PLC simulators are mostly used for training purposes and not for verification. Simulators alone are not suitable for exhaustive validation and verification testing. Simulators are useful for training purposes and allow you to explore your design before deployment 2005 ITTECH Pty Ltd

9. Logic under test Test condition O1 O2 Error O1 I1 O2 A1 A1 A2 O2 O1 A2 I2 The Advantages Of The Verification Method Used By The Logic Hunter • The PLC Logic Error Hunter, being introduced here, guarantees the soundness of the logic of a PLC program, by verifying, that specified logic test conditions (most of which are auto-generated) are always met by the program in all possible execution scenarios. E.g. - Verify that outputs O1 and O2 are never “on” simultaneously in the logic below. PLC Logic in white is initial program logic. Initially the test condition would fail. After the verification we add the interlocks shown in orange. After we add the interlocks O1 and O2 the test would pass. 2005 ITTECH Pty Ltd

10. Advantages Brought To You By The Tool Continued • Up to 75% of all required tests needed in testing the logic of a PLC program can be auto generated by the Logic Hunter. • The auto-generation and verification are performed just at the push of a button as you will see later. No need to create virtual test benches, to identify test scenarios and run simulations manually for long hours. The verification runs automatically and is guaranteed to render results within seconds. • In the case where the test conditions are not fulfilled by the program’s logic as expected, the Logic Hunter offers powerful debugging techniques allowing to rapidly find the cause of unexpected program behavior as it will be demonstrated. • The testing results and tests code coverage results and reports are simple and easy to understand and can be used as sign off documents for PLC logic functional approval. 2005 ITTECH Pty Ltd

11. You start by selecting the PLC program that you want to verify or auto generate tests for. The Error Hunter Studio will display the drawing of the selected program on the screen. The PLC Logic Error Hunter Is Extremely Easy To Operate 2005 ITTECH Pty Ltd

12. Then you select the modules in the PLC program for which you want to auto generate tests. It can be as many modules as required. When you hit the auto test generate button, all required tests for the selected modules will be auto generated and a test generation report will be displayed in the right hand pane. Easy Of Operation Demo Continued 2005 ITTECH Pty Ltd

13. Easy Of Operation Demo Continued • After this, you can execute all auto generated tests as a single test suite (execute all tests as a bundle), either with or without test code coverage statistics generation. The particular demo execution here is without code coverage statistics. On the next slide we show test suite execution with code coverage statistics. 2005 ITTECH Pty Ltd

14. Easy Of Operation Demo Continued • Test suite execution with code coverage statistics displays. 2005 ITTECH Pty Ltd

15. Easy Of Operation Demo Continued • For debugging purposes you usually execute individual tests from the list of auto generated tests which have failed: You start by selecting the PLC program that you want to execute tests on. The Error Hunter will display the drawing of the selected program on the screen and then you can verify it by executing a particular test as demonstrated below: 2005 ITTECH Pty Ltd

16. Easy Of Operation Demo Continued • You hit the verification run button and the analyzer goes and runs your PLC program, generating automatically input for it and verifying the specified test condition on each cycle (scan) executed in the PLC program. • If the test condition becomes true at any time during PLC program execution we stop the run and generate a trace (counter example) of the execution leading to a PLC program state where the test condition (property) becomes true. • In a test script, the most important part is the specified logic test condition which gets checked whether it ever becomes true during PLC program execution (“p:” – specification). • Another type of specification in the script, are Input Signal Constraints (“c:” – specification). They “constrain” the value of an input signal to some limits (or a value) for a number of PLC program scan cycles (or all the time). • Another type of specification are SpecialInputs (“s:” - specification). They can promote internal PLC tags as input signals so the verification engine can drive them automatically for verification purposes. 2005 ITTECH Pty Ltd

17. Easy Of Operation Demo Continued • After running a verification on a test condition, you can trace execute the generated counter example step by step, trace it in slow motion mode and set breakpoints on logical events in it so that you can easily analyze the events leading to the test condition becoming true. 2005 ITTECH Pty Ltd

18. Analyzing Program Logic Problems With The Trace and Debug Facilities • You can stop the execution of a counter example trace run at an interesting place by setting breakpoints and break conditions: You can specify a breakpoint by simply putting a red dot next to the place where the trace run should stop (not implemented yet). The run will stop on every scan cycle there. You can specify a logical break condition for the program to stop when the condition becomes true. For PLC program debugging purposes (as opposed to regular computation programs) break conditions are much more useful than the usage of breakpoints. • The trace run can be executed in step mode, cycle step mode or slow motion (TBD) mode. In step mode, one rung in a ladder diagram is executed and then we stop again. Step and cycle steps can be executed at any time after the trace run has been stopped (e.g. after you stop on a breakpoint). • During a trace run the signals in the program become animated and lines are colour coded according to their state. You can also precisely monitor variable values of PLC registers by simply pointing over the symbol of a PLC program component (e.g. counter, timer count etc.) (TBD feature). 2005 ITTECH Pty Ltd

19. Trace and Debug Facilities Continued • In slow motion mode you fire up the input/output visualization window and let the program run in slow motion mode. Every change of a PLC input/output state is displayed and after 1 second the screen is updated with the next state change. You can stop the execution at any time. You can adjust the motion speed. You can provide your own picture for the animation. It can be the P&ID drawings of your installation. You just need to point out where the coordinates of PLC inputs/outputs are in the provided picture/diagram. 2005 ITTECH Pty Ltd

20. Trace and Debug Facilities Continued • From counter example trace mode you can switch to step or cycle step in plain simulation mode where you generate manually input for the program and then execute a plain simulation in step, cycle step or run mode with stopping on breakpoints and break conditions. You can arbitrarily switch between counter example trace and simulation mode. This is useful if you want to see how the program would behave after you modify its logic. In this case you generate a counter example for the event. Create a break condition for the logical condition, run the trace and after the trace stops on the condition you continue with manual simulation by forcing signals in the way you want them to behave in order to see what you are interested in. 2005 ITTECH Pty Ltd

21. Trace and Debug Facilities Continued • To create timing diagrams for signals (tags) in the PLC program with the waveform viewer, you execute a test script verification with the “trace” LEH option enabled. 2005 ITTECH Pty Ltd

22. Easy Of Operation Demo Continued • Once you are done with analyzing the results, if there was a problem in the logic, you modify the logic and rerun the verification on the same test condition to make sure that this time the logic behaves correctly. In case where the logic behaves correctly you run a verification on a test condition just once and then go ahead and specify another test condition and run a verification for it. This process continues until you exhaust all test conditions for a particular part of the program. • Note that all of the above (the run and counter example generation) can be automated if using the test suite (batch mode) to run the tests. In this mode you specify several test conditions which will be verified one after the other on a specified PLC program and a counter example for each one of them will be generated and recorded on hard disk for later analysis. Test suite (batch mode) verification is most suitable for regression testing to make sure that modifications to the program (when developing or maintaining it) haven’t introduced unexpected results in the behavior of the existing implementation of the program’s logic. • The analysis of the counter example (trace results) is always done interactively with the help of the trace and debug facilities. 2005 ITTECH Pty Ltd

23. The Internals Of The Logic Error Hunter • What are the “behind the curtains” techniques used to verify a specified by the user test condition and generate a counter example (repeatable execution trace): • Checking whether a logical test condition, becomes valid (true), in a PLC program is based on formal verification methods: The logic of the PLC program is represented as a number of executable time temporal logical equations (e.g. the ladder diagram from below is transformed in the equations 00001 = 10001; 00002 = 00040; 00040 = 00041; 00041 = 10002;) The verification engine generates automatically input for the PLC program (in the example below it drives inputs 10001 and 10002) and executes its logic in scan increments watching all the time whether the condition gets true. In the same time, it keeps track of the states of the logic (combination of the values for all tags in the PLC program). If all possible logic states (whole of the green area below) get exhausted and the condition doesn’t get true the verification run is terminated reporting to the user that the test condition never gets true. If the test condition gets true before the logic state space gets exhausted the verification run gets stopped and a counter example is generated which the user then can trace later on in order to understand how the condition becomes true. The execution of the logic is directed, so that it leads to the test condition becoming true in minimal time. • The important message here is, that it is guaranteed that if there is an error in the logic, when verifying a certain test condition, the error will always be found and reported to the user. “O1 && O2 = true” test condition verification internals. PLC logic state is expressed by the actual values of all signals involved in the PLC program. For this particular PLC program the PLC logic state is: (I2,I1,C1,C2,O1,O2) E.g. Logic state 101101 is a state where: I2=1; I1=0; C2=1; C1=1; O1=0; O2=1 The LEH would drive I1 and I2 in order to prove “O1 && O2=true” White rectangle area is all PLC program states (program state space). Green oval area is all possible PLC program states allowed to be reached by its logic (possible program state space). The state path that the verification engine traces while executing the test is shown with orange arrows. When we execute an verification run – we travel from initial state to error state. When we generate the counter example we travel in the opposite direction (from error state to initial state). Logic state 101101 Logic state: 000101 Logic state: 101000 Logic state: 100000 Logic state: 000000 Logic state: 111111 2005 ITTECH Pty Ltd Counter example states path

24. Internals Of The Logic Error Hunter Continued • It is worth mentioning that the Logic Error Hunter is aimed at verifying the logic of a PLC program (executes the PLC program as a chain of logical expressions (it is not a PLC automata). If there are timing issues in the PLC program (concurrency issues with signals) it is guaranteed that LEH will detect them (usually a concurrency problem is indicated by some of the tests not passing all the time) and the user can debug an rectify the problems. • The technology allows to debug and test PLC programs in semiautomatic manner. It is not fully automatic because the user still needs to identify and specify part of the tests and also analyze the cause of a failures if such exist. But it provides all the necessary facilities to easily track down problems. As mentioned before, the advantage compared to simulation is that the user does not have to worry about identifying test scenarios which is very time consuming. 2005 ITTECH Pty Ltd

25. Internals Of The Logic Error Hunter Continued • Can be used for unit & system testing. In unit testing you test a single PLC program. In system testing you run several PLC programs for different PLCs and specify a test condition for one of them. The programs may interact with each other by exchanging signals or messages. • In a counter example auxiliary verification conditions can be specified (TBD). They are called trace points (verification points used to confirm that the PLC program logic goes through a certain logical path in order to reach the test condition state). They are used to ensure that invariants in the PLC logic are preserved (A test condition becomes true always as a result of the same events ordered as anticipated by the user in time. E.g. Make sure that always when an emergency stop button is pressed all motors are brought to a stop and are stalled.) PLC 1 Program Unit Test PLC 2 Program Unit Test Message Exchange System Test both programs by doing a synchronous run with message exchange 2005 ITTECH Pty Ltd

26. The building blocks of the Product • There is a translator (compiler) which takes the code of a PLC program written in any mixture of the 5 languages specified by IEC61131 (IL, LD, SFC, FBD or ST) and for any target PLC brand supported by the product and translates it into a generic PLC program for the purposes of verification. It has a generic formal verification engine, a simulator and a debugger in it. It allows you to execute the test condition verification and counter example generation on it, run just plain simulation of the PLC program in “run”, “step” and “cycle step” mode, run the counter example trace and debug it. • There is the user front end (GUI) which presents the generic PLC program in a graphical form and allows the user to operate on it. The user can choose to show the program in any of the 5 standard PLC program forms. In the GUI you can edit the program, and there are all the command gizmos which allow to execute test condition verification, simulation and counter example debugging. • There is also a backwards translator (TBD) which converts a modified PLC program in generic form and converts it back to a normal PLC program targeted for a specific PLC brand. Product GUI PLC Program code to Generic Verification/Simulation Model TRANSLATOR Generic PLC Program To Drawing In (IL, ST,LD,SFC, FB) TRANSLATOR Drawing VIEWER Debugging/Search FACILITIES Verification and Simulation ENGINE TRANSLATOR From Generic PLC Program Back To PLC Code PLC Program EDITOR 2005 ITTECH Pty Ltd

27. Further Advantages Brought To You By The Error Hunter • As already mentioned, the Logic Hunter has the ability to create a regression test suite in order to verify the overall soundness of the logic every time it is being changed. • In batch mode regression testing, there is an automatic test coverage gathering facility allowing you to see which logical paths in the program have been covered in a regression test run. This helps with identifying additional test conditions to be added to the regression test suite in order to cover the whole logic of the program. • Also in batch verification mode there is a test result logging and report generation facility. It promotes standard program acceptance procedures being put in place for safety critical applications. • The Error Hunter Studio is indispensable in existing contiguous process control system upgrade procedures. In this case you verify the PLC control logic with the studio prior to downloading it in the PLC. Then you let the PLC take immediately control over the contiguous process without interrupting it for acceptance tests on the facility. • The verification is exhaustive. If there is a logic problem, it is guaranteed that it will be found and reported to the user. This is an important issue in enablement of mission critical application development. • Finally, there is a very short learning curve with the tool. You learn very fast how to operate it. 2005 ITTECH Pty Ltd