1 / 42

Pattern-driven Analysis & Adaptation of High Assurance Software

This presentation discusses a cost-effective, automated approach to analyzing and adapting complex software for high assurance applications. It addresses a number of issues and aims to establish confidence and eliminate potential faults in flight. The technical approach involves modeling tools, HA models, software adaptation, and the use of pattern catalogs.

warrend
Download Presentation

Pattern-driven Analysis & Adaptation of High Assurance Software

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pattern driven analysis & adaptation of HA SW • Problem Description & Program Objectives • Technical Approach • Schedule • Milestones in next 6 months • Technology transition Slide 2

  2. Project web site This presentation and other information is available from: • http://dirac.ee.iastate.edu/sec/index.htm • User Name: sec • Password: skgd5103 Slide 3

  3. Problem Description* • Large body of complex software (Boeing OCP) • High assurance application (SEC) • Software is critical (faults may result in mission failure, loss of aircraft, equivalent to DO-178B level A) • Concern over a number of issues (83 in Issues List) • Complete manual review impractical for small team, esp. with respect to global issues • Testing is expensive, and does not cover all cases Slide 4

  4. Program Objectives* • Cost effective, automated approach (driven by static analysis) • Analyze/adapt OCP to establish confidence, eliminate problems, avoid potential faults in flight • Address the most important issues • Flexible with respect to definition of problems, specification of changes, ability to keep developer in the loop - based on user specified “Software Patterns” • Support for multiple modeling and programming languages (to amortize costs, address a broader user base) • Test cases generated from interface specs to meet MC/DC coverage criteria – for changed modules, "high risk” modules Slide 5

  5. Technical Approach Modeling tools (GME, Rose, etc.) HA Models Models KCS toolset (for translation, analysis, and adaptation) Adapted XML/XMI models XML/XMI models HA SW SW Adapted OCP, and other software source (in C++, Java, other languages.) OCP and other software source (in C++, Java, other languages) Patterns in XML/XMI Developer Review/edit T-VEC formal specs T-VEC Regression testing Issues List Catalog of patterns for high assurance software Slide 6

  6. Issues List: Complete • Compiled from a variety of sources • Identifies 83 issues related to software for high assurance systems • Includes both design level and language level issues • And issues related to verification and testing • List available from the Issues page of our web site http://dirac.ee.iastate.edu/sec/HA-SW-Issues.htm Slide 7

  7. Issues List: Sources* • Compiled from a wide variety of sources, including: • Handbook for Object-Oriented Technology in Aviation (OOTiA), Proceedings of FAA/NASA Workshops 1 & 2 on Object-Oriented Technology in Aviation, 2003 • Guide to the use of the Ada Programming Language in high integrity systems, ISO/IEC TR 15942, 2000 • John Barnes. High Integrity Ada: The SPARK Approach, Addison-Wesley, Harlow, England, ISBN 0-201-17517-7, 1997 • Use of the C++ Programming Language, Certification Authorities Software Team (CAST), position paper, CAST-8, January, 2002 • David W. Binkley. C++ in Safety Critical Systems, NIST, November 1995 • Boeing, BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801, 2002 Slide 8

  8. Issues List: Sources* • Bertrand Meyer. “Applying design by contract.” IEEE Computer 25(10):40-51, October 1992. • Barbara Liskov and Jeanette Wing. “A Behavioral Notion of Subtyping”, ACM Transactions on Programming Languages and Systems, 16(6): 1811-1841, November 1994. • Scott Meyers. Effective C++, 2nd edition, Addison-Wesley, Reading, MA, 1998 • Scott Meyers. More Effective C++, Addison-Wesley, Reading, MA, 1996 • Marshall P. Cline and Greg A. Lamow. C++ FAQs, Addison-Wesley, Reading, MA, 1995. • Jeff Offutt, et al. A Fault Model for Subtype Inheritance and Polymorphism, Twelfth IEEE International Symposium on Software Reliability Engineering (ISSEE ’01), 2001 • Robert V. Binder. Testing Object-Oriented Systems: Models, Patterns, and Tools, Addison-Wesley, Reading, MA, 2000. Slide 9

  9. XCIL: Complete* • eXtensible Common Intermediate Language (XCIL) • Target language independent XML representation • Based on UML, with actions defined by Microsoft Intermediate Language (MS-IL) and the Java Virtual Machine (JVM) • Extended to include full C++ • Precise semantics • Supports UML models and all .NET and JVM languages • Basis for integration with other tools • See http://dirac.ee.iastate.edu/sec/XCIL.htm Slide 10

  10. XPSL: Complete* • eXtensible Pattern Specification Language (XPSL) • Patterns described in GOF-like format • But formal (XPSL) – in order to drive underlying tools • An extension of Aspect-Oriented concepts • But at the level of actions, with support for transformations (in addition to insert/replace) • And semantically defined pointcuts identified by analysis • Anticipates OMG Query/View/Transformation standard, which we plan to use as a foundation • See http://dirac.ee.iastate.edu/sec/XPSL.htm Slide 11

  11. Pattern catalogs: Complete • 40 patterns (10 abstract, 30 concrete) • Addressing 26 of the 83 issues • Build on FAA/NASA, AVSI patterns defined by RC, Boeing, Honeywell, Goodrich and others • Few other high assurance software patterns available • Developed incrementally • Most important issues first • See http://dirac.ee.iastate.edu/sec/PatternCatalogs.htm Slide 12

  12. KCS toolset: To date • C++ to XCIL translator complete • Entire OCP (version 2.2) translated to XCIL • Pattern analysis/adaptation is also being developed incrementally • Initial focus on finding problems • Analysis support for 8 patterns, addressing 10 issues • Working on adaptation framework for two of the patterns • With more to follow, based on ranking of problems Slide 13

  13. Ranking of problems* • Hard run time error • A fault may occur when the erroneous code is executed • Lurking run time error • A hard run time error may result if additional correct code is introduced • Obstacle to analysis (potential masking of existing errors) • The code cannot be analyzed in some manner typically required for “certification” • Maintenance risk (potential source of future errors) • The code is correct, but subsequent changes are likely to introduce errors Slide 14

  14. Analysis results: To date* • For the 8 patterns and 10 issues addressed so far: • Matches on 9 of the 10 issues were found when the patterns were applied to the OCP • Typically several hundred matches per issue • Problems range from maintenance risks to lurking and potential run time errors • A few of these are discussed in the following slides • Summary given by OCP Resultslink on home page of web site http://dirac.ee.iastate.edu/sec/HomePage.htm • Detailed results (with mapping to source code) given on OCP Results page Slide 15

  15. Pattern catalog • Increment one • Development of eXtensible Pattern Specification Language (XPSL/XML) • Development of patterns for: Control flow, Data integrity, Pointer integrity, Unintended side effects, Multiple inheritance Slide 16

  16. Tool support • Increment one • Development of XCIL • Translation of C++ to XCIL, • Analysis framework (queries and views), • Analysis support for selected increment one patterns, • Focus on “local” patterns, affecting individual OCP components Slide 17

  17. Pattern: Safe Type Conversion • Addresses issue 4: Implicit type conversion • Implicit type conversion may involve a potential loss of data or precision, or semantics [OOTiA Handbook] • “Type conversions shall not be made from a longer to a shorter (number of bits) type. Type conversion shall not be made from an unsigned type to a signed type of the same size” - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801 • Matches • No implicit conversions between integer and float • 58,675 implicit conversions of a larger value to a smaller value • 352 implicit conversions of a signed value to an unsigned value • 87 implicit conversions of an unsigned value to a signed value • Matches represent potential run time errors • See OCP Results and Increment One Patterns for details Slide 18

  18. Adaptations • Subject to review • Need report on types converted from/to • Need report on writes to variables involved in conversions • Need to consider whether the actual range of values of the source type fits within the range of values permitted by the destination type • Need to consider whether the converted values are treated as numeric values (whether signed/unsigned matters) • Introduce user defined types or typedefs • Use bool instead of integer • Use single size, avoid conversions • Use explicit conversions • Introduce run-time range checks (not performed by C++) Slide 19

  19. Pattern: Compiler generated operations • Addresses issue 25: Compiler generated operations • Compiler generated operations have default meanings that may be inappropriate for a given class • Analysis results • 1,451 classes with compiler generated default constructors • 2,078 classes with compiler generated copy constructors • 2,100 classes with compiler generated “=” (assignment) operators • 2,218 classes with compiler generated “&” (address of) operators • Matches are subject to application of the pattern’s rules • Violations of these rules represent likely run time errors and lurking run time errors Slide 20

  20. Pattern: Compiler generated operations • Rules • Law of The Big Three: “If a class needs a destructor, or a copy constructor, or an assignment operator, it needs them all” … “During code reviews and debugging sessions, we have traced many core dumps back to violations of The Law” [Cline and Lomow, C++ FAQs] • “Declare a copy constructor and an assignment operator for classes with dynamically allocated memory” [Scott Meyers, Effective C++, 2nd edition, item 11] • Declare a constructor for all classes with an user defined invariant [OOTiA Handbook] • All fields without initial values must be set by a user defined constructor [OOTiA Handbook] • See OCP Results for details, associated adaptations Slide 21

  21. Adaptations • The user is responsible for introducing the necessary operations, while the tool is responsible for checking the user supplied code • In addition the tool may introduce run-times checks of constructor postconditions, to ensure the class invariant is established • Checks of user supplied constructor • Run time check of invariant as postcondition • Check that all fields are initialized • Checks of user supplied copy constructor • Run time check of invariant as postcondition • Check that all dynamically allocated components are replicated (deep copy) • Check of user supplied assignment operator • Check that all dynamically allocated components are replicated (deep copy) • Check of user supplied destructor • Check that all dynamically allocated components are deallocated Slide 22

  22. Pattern: No pointer arithmetic • Addresses issue 30: Use of pointer arithmetic • “Operational software shall not include pointer arithmetic. In general, there is little justification for going to the effort of designing code to do pointer arithmetic. The performance gained by doing pointer arithmetic is seldom worth the increased risk of error.” - Explicit pointer value calculation is prohibited for DO-178B software levels A through D - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801 • Matches • 470 instances of pointer arithmetic • Matches represent potential run time errors and obstacles to analysis • Rules • A pointer can be used only as a substitute for an opaque reference • See OCP Results and Increment One Patterns for details/adaptations Slide 23

  23. Adaptations • Replace each array element pointer with an array base pointer and index, performing all arithmetic on indices rather than pointer values • Selectively introduce run-time bounds checks (typically not performed by C++ compilers) • Restrict other uses of pointers in XCIL to the cases given (in which the use of a pointer is compatible with the use of an opaque reference), requiring the user to make associated changes Slide 24

  24. Pattern: Goto-free execution • Addresses issue 19: Use of goto’s • “The use of goto is regarded as both unnecessary, and undesirable compared to the use of other control structures. It should not appear in well written code.” – SPARK • The use of goto should be excluded because it complicates flow analysis (FA), symbolic analysis (SA), timing analysis (TA), and structural testing (ST). - Guide for the Use of the Ada Programming Language in High Integrity Systems, ISO/IEC TR 15942 • Matches • 1,744 gotos • Matches represent obstacles to analysis • See OCP Results and Increment One Patterns for details/adaptations Slide 25

  25. Adaptations • Subject to review • Unless used as a substitute for exceptions, refactor code to: • Replace goto with structured control flow (conditional, loop, break, continue, error return, other common forms) • If used as a substitute for exceptions, apply exception/error handling patterns: • Enforcing the disciplined use of exceptions (per Bertrand Meyer) • Ensuring all reported errors (thrown exceptions) have handlers – as in Java • Eliminating macros in favor of pattern substitution, allowing a variety of error reporting/error handling strategies Slide 26

  26. Pattern: Fall through switch case • Addresses issue 17: Break in switch • “Each case in a switch, which contains any executable code, shall be terminated with a break. This eliminates the dubious practice of falling through a case to use code in a succeeding case for common processing. It is too easy to lose track of the correct program flow in code maintenance.” Mandatory for DO-178B software levels A through D - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801 • Matches • 461 fall through cases • Matches represent maintenance risks • See OCP Results and Increment One Patterns for details/adaptations Slide 27

  27. Adaptations • Refactor code to either: • introduce a method called by both cases (when the shared code is cohesive) • introduce an inlined call (when the shared code is short and cohesive) • introduce a conditional (if statement or nested case) when the shared code is not cohesive • introduce replicated code (when the shared code is short but not cohesive) Slide 28

  28. Other increment one patterns • Non-discriminated union (Issue 47) • Implicit method return (Issue 56) • Loop termination (Issue 44) • Enforce invariant (Issue 13) • Closed argument list (Issue 15) • No dangling pointers to local variables • No assignment statement arguments (Issue 40) • No user thrown exceptions (Issue 49) • Top level handler for predefined exceptions (Issue 49) Slide 29

  29. Pattern catalog • Increment two • Development of patterns for: Contracts and subtyping, Coupling and cohesion, OO metrics, Dynamic resource allocation, Safety critical language subsets Slide 30

  30. Tool support • Increment two • Adaptation framework (transformations), • Adaptation support for selected increment one patterns, • Analysis support for selected increment two patterns, • Focus on “global” patterns, requiring an analysis of the entire OCP Slide 31

  31. Pattern: Formal interfaces* • Addresses Issue 1: Lack of precise requirements/interface specifications • “If we don’t state what a module should do, there is little likelihood that it will do it (The law of excluded miracles)” – Bertrand Meyer • Programming by contract: “The relationship between a class and its clients [can be] viewed as a formal agreement [contract]. Only through such a precise definition of every module’s claims and responsibilities can we hope to attain a significant degree of trust in large software systems” [Bertrand Meyer, Object-oriented Software Construction] • MC/DC test coverage criteria [DO-178B], Unit level subtyping tests • Matches • All class interfaces • See Increment Two Patterns for details, discussion of generation of pre/post from code Slide 32

  32. Pattern: Subtyping* • Addresses • Issue 6: Supertype/subtype compatibility • Issue 23: Use of dynamic dispatch • Issue 5: Dynamic dispatch during object construction • In particular • “The single largest source of design errors is improper inheritance” [C++ FAQs, Issue 119] • Errors may result from • Faulty intuition • Failure to implement to inherited specifications • Misuse of inheritance to share code/data • Programmer specified optimizations • Accidental overriding/hiding • Missing override Slide 33

  33. Pattern: Subtyping* • Analysis results • 938 subtyping relationships • 1,392 cases in which an operation is implemented by multiple methods (OMM pointcut) • 3,112 instances of “true dynamic dispatch” (TDD pointcut) • Review results (sample of ~40 subtyping relationships) • 24 violations of subtyping rules/LSP (including conflicting postconditions, less visible subclass operations, disabled methods, null method implementations, and weaker postconditions) - overlapping • Violations represent either lurking or hard run time errors • See Increment Two Patterns and OCP Subtyping Relationships for details/adaptations Slide 34

  34. Pattern: Multiple implementation inheritance* • Addresses issue 8: Ambiguity resulting from the use of multiple implementation inheritance • Multiple inheritance session of OOTiA workshop unattended because no one thought MII appropriate for high assurance applications • Review results (sample of ~40 subtyping relationships) • 4 instances of multiple implementation inheritance • Matches represent maintenance risks and potential run time errors • Rules • Replace with delegation for DO-178B levels A through C • See OCP Results and Increment One Patterns for details/adaptations Slide 35

  35. Other increment two patterns* • Safety critical language subsets (“Safe” C++?) (Issue 71) • Synchronization (Issue 3) • Multiple interface inheritance (Issue 7) • Replacement of multiple implementation inheritance with delegation (Issue 8) • No dynamic allocation after initialization (Issue 9) • Initialization before use (Issue 12) • Class coupling (Issues 13, 67, 68) • Dynamic dispatch test coverage • Method extension • Deep hierarchy Slide 36

  36. Regression/subtype testing* • Formal interface specifications (pre/post) are being developed for all classes we expect to change • And for all classes involved in subtype relationships that require LSP compliance testing • Test cases are generated from these formal interface specs by T-VEC • This approach helps us address issue 1: Lack of precise requirements/interface specifications • As well as providing a basis for regression testing • With MC/DC coverage Slide 37

  37. Lessons learned* • Tool support has been invaluable in identifying potential problems • Much more effective than reviews in terms of hours spent, accuracy, and counts of problems found • Valuable as a driver/assistant in the review process • Necessary due to size of OCP • Need to be able to interactively refine problem description, tune patterns Slide 38

  38. Schedule Slide 39

  39. Next Milestones* • Regression testing of 1st pass adapted software (pointer arithmetic, type conversion, etc.) • 6/27/2003 • Complete work on analysis and adaptation (strategy = most important first) • 7/31/2003 • Regression testing of 2nd pass adapted software • 8/29/2003 • Final Report • 12/19/2003 Slide 40

  40. Technology Transition* • Web site has latest versions of the Issues list, XCIL papers, XPSL papers, Pattern catalogs, and the OCP analysis results • Reports identifying problems found (by both reviews and the toolset) will be posted to the web site • Change log + T-VEC interface specs, test cases, and test results for changed modules will be provided • Revised OCP code will be posted when all regression testing is complete • The tools developed by ISU and the contents of the web site will be distributed on CD • The KCS tools will be made commercially available through Ensoft Corporation at ISU Research Park Slide 41

  41. Program Issues • Maximize benefit in time remaining • Extended subcontract with ISU to end of July • Space and time required to run KCS analysis and adaptations • Faster machine, parallelization, incremental • Use of the ISU toolset by others requires licensing of EDG parser (from Edison Software) • University agreement with Edison (similar to ISU) • Licensing from Ensoft (as part of XCIL translator for C++) Slide 42

More Related