Formal Methods and Testing: Possible Attributes for Success

1. Formal Methods and Testing:Possible Attributes for Success A. J. Cowling Department of Computer Science University of Sheffield

2. Rationale • The Number of Different Methods • Many formal and semi-formal methods exist • New ones are still being created • Their Usefulness • Determining which are useful requires empirical work • The number of methods would imply a lot of work • Therefore priorities need to be set • Technical Features • Some methods appear to be more successful than others • Possibly because of their technical features • These would be more appropriate candidates for study University of Sheffield Department of Computer Science

3. Requirements Model DevelopmentProcess Specification Model Formal steps Design Model Testing Implementation Model Required V & V Role of Models University of Sheffield Department of Computer Science

4. Testing and Models • Basic Testing Methods • Base the generation of test cases on one of these models • Black-box testing uses the specification model • White-box testing uses the implementation model • Hybrid Testing Methods • Combine the approaches – eg: • Black-box methods to generate the test sets • White-box methods to measure their coverage • May provide more effective testing than individual basic methods • At least, according to some papers • State-based Testing • Uses state-machine models for specification and implementation • Extended models (eg the X-machine) allow powerful results: • absence of faults up to some bounds, • under some assumptions, complete absence of faults University of Sheffield Department of Computer Science

5. Key Formal Methods • Model Checking • Requires state-based specification models • Shows whether required properties hold for the models • Can handle very large systems (10^20 states) • Machine Model Verification • Uses state-based specification and implementation models (eg B) • Can verify that implementation is consistent with specification • Refinement • Typically uses relational models (eg Z, VDM) • Refinement steps produce correct-by-construction implementations • Discontinuities in the models need to be accommodated • Retrenchment has been proposed for this University of Sheffield Department of Computer Science

6. Attributes for Success • Role of Models • Successful approaches appear to all be model-based • State machine models are particularly successful • Extended state-machine models even more so • Differences between Models • The different stages require different models for one system • Any form of V & V must accommodate these differences • ie must represent design transformations • currently an interest within model-driven architecture • It appears that successful methods: • explicitly handle multiple models, and • explicitly represent the differences between them. University of Sheffield Department of Computer Science