The CONVERSE Project: Tough on Change, Tough on the Causes of Change.

1. The CONVERSE Project:Tough on Change, Tough on the Causes of Change. Improving Software in Engine ControllersUniversity of YorkJohn McDermid, John ClarkRA: Nigel TraceyRS: Zoe Stephenson

2. Improving Engine Controller Software Development • Aim to improve the process by which Full Authority Digital Engine Controllers (FADECs) are developed. • Two Part Strategy: • Family based approach – identifying and restricting change. • Handling the verification aspects of testing efficiently when change occurs. • Carried out a proof of concept derisking project in conjunction with the Rolls Royce University Technology Centre (UTC)

3. Family-based Engine Controllers I • Goal: reduce change estimation costs in a family-based process. • Build explicit feature models for systems and requirements (physical systems, interfaces, required functions) • Use other features and explicit and tacit domain information as context

4. Family-based Engine Controllers II • Record dependencies between features and context information • Model required variations from stakeholders (customers or market forces) • Combine variations and context information into a dependency graph

5. Family-based Engine Controllers III • perform impact analysis over the dependency graph to estimate change • accuracy depends on integrity of the context information • use accuracy data to validate and refine context dependencies

6. Testing: Properties of Interest • Functional properties. • Real-time properties. • Safety constraints. • Exception conditions. • Meeting reuse assumptions

7. Testing: things that help • Small modules • Restricted data types • Simple design structure These facilitate testing generally but also have a significant impact on the feasibility and efficiency of automated approaches to software testing.

8. A Framework for Automated Testing • Based on meta-heuristic search (genetic algorithms and simulated annealing) • All problems couched as optimisation problems. • Handles all indicated properties in a uniform manner.

9. Non-functional Black-box White-box Worst Case And Best Case Execution Times Constraint Solving SpecificationTesting StructuralCoverage Grey-box Exception Generation Safety Conditions Software Reuse Mutation The Framework

10. Testing Criterion SUT Search Techniques Extractor Test System SUT Information FitnessFunctionModule Fitness Function Test Data The Framework

11. Genetic Algorithm for Test Data Gen Mutate members of new population Initial population Use objective function to measure test data suitability Combine parents to formnew population Evaluate current population Run SUT with test data Select parents fromcurrent population Determine survivorsfor next population Stop Search? Test data

12. Some Publications • Integrating Safety Analysis with Automatic Test-Data Generation for Software Safety Verification. Nigel Tracey, John Clark, John McDermid and Keith Mander. In the Proceedings of 17th International System Safety Conference. August 1999. Pages 128-137. • Integrating Automated Testing with Exception Freeness Proofs for Safety Critical Systems. Nigel Tracey, John Clark, Keith Mander and John McDermid. In the Proceedings of 4th Australian Workshop on Safety Critical Systems and Software. Australian Computer Society. November 1999. • Assessing Test Set Adequacy for Object-Oriented Programs Using Class Mutation. Sun-Woo Kim, John Clark, and John McDermid. In the Proceedings of Symposium on Software Technology (SoST'99). Pages 72-83, September 1999. • The Rigorous Generation of Java Mutation Operators Using HAZOP. Sun-Woo Kim, John Clark, and John McDermid. In the Proceedings of the 12th International Conference on SOFTWARE & SYSTEMS ENGINEERING and their APPLICATIONS (ICSSEA'99). December 1999. • A Safety Change Oriented Process for Safety-Critical Systems. Nigel Tracey, Alan Stephenson, John Clark and John McDermid. In the Proceedings of Software Change and Evolution Workshop. IEEE International Conference on Software Engineering. May 1999. • Automated Program Flaw Finding using Simulated Annealing. Nigel Tracey, John Clark and Keith Mander. In Software Engineering Notes Issue 23 Number 2, the Proceedings of the ACM/SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). USA, March 1998. Pages 73-81. • The Way Forward for Unifying Dynamic Test-Case Generation: The Optimisation-Based Approach. Nigel Tracey, John Clark and Keith Mander. In the Proceedings of the IFIP International Workshop on Dependable Computing and Its Applications (DCIA). South Africa, January 1998. Pages 169-180. • An Automated Framework for Structural Test Data Generation. Nigel Tracey, John Clark, Keith Mander and John McDermid. ASE 1998, Honolulu. • Towards Industrially Applicable Formal Methods: Three Small Steps, and One Giant Leap. John McDermid, Andy Galloway, Simon Burton, John Clark, Ian Toyn, Nigel Tracey and Sam Valentine. In the Proceedings of IEEE ICFEM (International Conference on Formal Engineering Methods). December 1998. • Automated test-data generation for exception conditions. Nigel Tracey, John Clark, Keith Mander and John McDermid. Software Practice and Experience, January 2000. • CONVERSE: A Change-Oriented Process for Engine Controllers. Darren Buttle, John Clark, John McDermid, Alan Stephenson and Nigel Tracey IEE Software. Vol. 146(3)130-136, June 1999. • SEBPC New Directions. Chapters 4 and 12 • CONVERSE: A Family Oriented Software Development process for Engine Controllers. Simon Burton, Darren Buttle, Zoe Stephenson and John McDermid. • A Search-Based Automated Test Data Generation Framework for Safety-Critical systems. Nigel Tracey, John Clark, John McDermid and Keith Mander.

13. Overall • Successful project. • Has aroused considerable interest and attracted further investment from our industrial collaborator (Rolls Royce). • Invaluable help given by RR • (access to information, involvement in meetings, reviews, access to very specialised resources etc., and tailoring funded work to assist with CONVERSE) • 12 conference and journal publications • Over 10 seminars given. • Peer rated “Alpha 4”