PROPANE An Environment for Examining the Propagation of Errors in Software

1. PROPANEAn Environment for Examining thePropagation of Errors in Software Martin Hiller, Arshad Jhumka, Neeraj Suri Chalmers University of Technology Göteborg, Sweden {hiller, arshad, suri}@ce.chalmers.se

2. Overall Objectives Software reliability can be provided/increased by adding mechanisms that detect and correct data errors  wrappers, assertions, etc. (especially for black-box software) A C E B D F Given a software system and limited resources one would like to concentrate work on the most vulnerable/exposed parts of the software, i.e. …  Where do upcoming errors propagate? PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

3. Outline • Error Propagation & Basic Approach • Examples of results obtained by using PROPANE • Aircraft arrestment system • Overview of PROPANE • Tool suite • Requirements & limitations • Conclusions & some future directions PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

4. Error Propagation Error = A system state which is different from the state in a ”correct” execution of the system (i.e, not mutations or software defects) A C E B D F Modules in a software system have different levels of exposure and different ”ability” to break error propagation (i.e., different levels of error containment) – examining error propagation gives the developer a picture/profile of these levels. PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

5. Basic Approach  Error Injection • Generate Golden Run, i.e., an error free reference run • Generate Injection Run, i.e., a run in which an error (i.e. erroneous system state) is injected • Compare Injection Run with Golden Run to see which parts of the system were corrupted by the injected error A C E B D F = injection location = probe PROPANE – Propagation Analysis Environment  By instrumenting the target software, PROPANE can, during execution, log individual variables and events and inject errors into individual variables. PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

6. Example: Aircraft Arrestment System Cable Tape drum (original) Tape drum (mirror) ms_slot_nbr i CLOCK CALC mscnt pulscnt slow_speed PACNT stopped DIST_S TIC1 TCNT Rotation sensor Pressure sensor Pressure valve Pressure valve SetValue ADC OutValue TOC2 PRES_S V_REG PRES_A IsValue Computer Target system overview Target software overview PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

7. Results Generated by PROPANE • From low level to high level information • Signal/Variable Plots • Propagation Signatures • Propagation Graphs • Propagation Summaries • Data compilations that can be used for further analysis PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

8. Signal/Variable Plots One-cycle single-bit upset at t = 1500 ms SetValue erroneous at t = 1539 ms PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

9. Propagation Signatures • Each injected error generates a Propagation Signatureshowing when and where the error propagated PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

10. Propagation Graphs The Propagation Graph is generated in three different formats • PROPANE native format • GML (Graph Markup Language) • dot (part of the graphviz tool suite from AT&T Research) Incoming errors from locations earlier in the propagation path error counttmin / tavg / tmax Probedlocation Outgoing errors to locations later in the propagation path PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

11. Propagation Graph for PACNT Starting point Propagation path Each arc carries information regarding number of propagated errors and propagation time Variables along the propagation trajectory PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

12. Propagation Summary for PACNT Probed location(variable) • For all incoming arcs to one node: • Total error count and error rate • Combined tmin / tavg / tmax Propagation Graphs and Propagation Summaries indicate which parts of the observed system state that are most vulnerable to the injected errors perhaps a good location for error detection/recovery mechanisms? PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

13. Error Propagation from PACNT ms_slot_nbr i Highest Error Rate CLOCK CALC mscnt pulscnt Lowest Error Rate slow_speed PACNT stopped DIST_S TIC1 TCNT SetValue OutValue PRES_S PRES_A ADC V_REG TOC2 IsValue PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

14. Overview PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

15. PLPROPANE Library • Static C Library providing the injection/logging API • propane_inject() /* Injects an error */ • propane_log_var() /* Variable probe */ • propane_log_event() /* Event probe */ • Target must be instrumented with these functions and linked with the library • Experiments are then set up using description files • Error types (what errors to inject, e.g., bit-flips) • Error triggers (when to inject errors, e.g., once/periodically) • Active probes (perhaps not all instrumented variables/events have to be logged) PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

16. Instrumentation Example Instrumented code Original code int spherical_volume( double radius ) { double volume; volume = 4.0 * (PI * pow(radius, 3.0)) / 3.0; return volume; } int spherical_volume( double radius ) { double volume; /* Injection location for radius */ propane_inject( IL_SPHERE_VOL, &radius, PROPANE_DOUBLE ); /* Probe the value of radius */ propane_log_var( P_RADIUS, &radius ); volume = 4.0 * (PI * pow(radius, 3.0)) / 3.0; /* Probe the value of volume */ propane_log_var( P_VOLUME, &volume ); return volume; } At this point, instrumentation is unfortunately still a manual task. However, automation is the next step in the development of PROPANE. PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

17. PSCPROPANE Setup Creator • Once the target is instrumented, experiments can be set up • Given information on injection locations, probes, error types and occurrences, PSC will create • description files for running experiments • description files for data extraction and analysis PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

18. PCDPROPANE Campaign Driver • After instrumentation and setup, the PCD runs the experiments • Invokes instrumented target executable containing the PL and all links to external modules, e.g., environment simulator • Provides continuous information on experiment status and remaining work • User control • Pause • Abort • Skip PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

19. PDEPROPANE Data Extractor • Extracts and analyses data from the raw readouts produced during experiments • Perform Golden Run Comparisons, i.e., compare an injection run with a ”clean” reference run to trace errors, generating a Propagation Signature • Collapse multiple propagation signatures into Propagation Graphs and Propagation Summaries • Prepare/compile data for further external analysis (using e.g. MatLab) PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

20. Requirements & Limitations • Programming language • The library itself is written in C  the language in the target system must be able to interface with C-libraries • Environment simulator • Stimuli to the target system must be provided by an environment simulator • PROPANE provides basic interfacing functionality such that control of the EnvSim can be made part of the experiment setup (~plug-in’s) • Target instrumentation • Difficulties with real-time applications that cannot be made to run in simulated time • Current implementation focused towards single-node SW • Distributed SW not considered at this point PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

21. Conclusions • PROPANE enables experimental evaluation of error propagation in SW • Propagation profiles indicate which parts of system state that are vulnerable  aid for equipping SW with error detection and recovery mechanisms • We envision that PROPANE can be used as a design stage aid for analysis of single node SW • Limitations apply for some apsects of real-time and distributed SW PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

22. Some Future Directions • Increased automation • E.g. automatic instrumentation of target software • Extended analysis capabilities • E.g. provide your own analysis plug-ins • Open source • Possibly in the near future PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002

23. Further Information(and future download area) www.ce.chalmers.se/LDC/DEEDS PROPANE: An Environment for Examining the Propagation of Errors in Software ISSTA 2002