Independent Verification and Validation (IV&V) Techniques for Object Oriented Software Systems

1. Independent Verification and Validation (IV&V) Techniques for Object Oriented Software Systems SAS meeting July 2003

2. Outline • IV&V of OO artifacts • Identifying Risks • Proposed Techniques • GQM (Goal Question Metric) Application • BBN (Bayesian Belief Network) Application

3. Problem Statement • NASA has considerable experience applying IV&V to traditional function-based software systems • However the emerging use of OO brings unique challenges and risks • There is a need to develop IV&V techniques for addressing these risks • Verify these techniques

4. Objective • Understand risks unique to large OO systems • Establish a framework for identifying, evaluating, and implementing IV&V techniques for mitigating these risks

5. Multi -Year Goals • Identify Risks • Develop base set of IV&V Techniques • Apply & evaluate techniques • Focus additional research and match emerging techniques to prioritized needs

6. Approach • Phased Approach • Identify Risks, Develop Base set of Techniques, Evaluate Base set • Focused research to adapt techniques for a broader spectrum of OO projects • Reasoning • Level of maturity of IV&V’s understanding of the problem, developers processes, and V&V techniques specific to OO system development • Limited availability of defect data specific to OO development

7. Continuous research & evaluation of emerging techniques Focused research to map, adapt, or develop techniques to prioritized IV&V activities based on experienced and perceived needs for improvement in efficiency, accuracy, and capacity. 1) Identify OO Development Risks 2) Develop Base IV&V Techniques • Requirements • Analysis • Design 3) Evaluate Base IV&V Techniques Current Phase Next Phase

8. Activities to-date • Evaluation of current & emerging OO IV&V techniques • Identification of Risks with OO development • Investigating framework for OO IV&V • Developing base set of IV&V techniques

9. State of IV&V Practice • Rudimentary manual techniques for Use cases or OO analysis • Techniques limited to static analysis • No automated tools or metrics being used for requirements, analysis & design • Manual techniques ensure • Specifications use UML compliant notation • Design is adequately documented • Design specifications will meet requirements • Techniques do not address the quality of OO design objectives • Encapsulation, coupling, cohesion, polymorphism etc.

10. Emerging Techniques • Some examples of emerging techniques • Dynamic Analysis Techniques e.g. Yacoub et. al. "Dynamic Metrics for Object Oriented Designs" • Semantic Metrics e.g. Etzkorn et. al. “Towards a Semantic Metrics Suite for OOD” • Application of BBN e.g. Fenton et.el. “Bayesian Belief Network Model for the Safety Assessment of Nuclear Computer-Based Systems” • State of practice • These techniques offer exciting promise for improving IV&V products • Need further enhancement for practical IV&V application

11. Risks to OO Development • Risk to OO development fall into three major categories • Programmatic Risks • Project termination, software component integration, utilization of tools, and project failure • Technology Risks • Issues with technology used to build the product, such as: Operating system, Compilers, Libraries, and middleware • Methodology Risks • Type of OO methods being used, incorrect application of a method, and issues related to lifecycle phases

12. Programmatic Risks • Project termination • Sub par schedule and performance of product • Methods abandonment • Lack of progress or OO related benefits • High learning curves • Insufficient budget and schedule for training • Not meeting OO objectives • Autonomy between organizations, absence of short iterative development cycle • Dependencies on Tools • Cost and tool deployment requirements

13. Technology Risks • Object Model • Technology does not fully support the object model e.g. asynchronous communication between objects in relational databases • Persistence • Poor support for object persistence e.g. OODB, relational DB, ad-hoc flat files

14. Methodology Risks • Incorrect application of methodology, insufficient methodology, or no methodology • Design mixed with requirements/use cases , insufficient information in design models • Software performance • Language/hardware combination fails to supports performance • High defect level • System usability, reliability, and safety

15. GQM Approach • Identify software product metrics that can support analysis goals • For Use Case diagrams • GOALS- completeness, traceability, correctness • Questions-How to achieve the goals • Metrics-use case metrics such as by Marchesi • Later plans for application to Class diagrams

16. BBN Approach • For Use Case diagrams • Use of BBN to determine readiness for project transition into the next lifecycle phase • Actors, actions, communication association, as part of network topology • Work is on going for determining characteristics of nodes, e.g. weight, child and parent levels • CPT tables • Assumption: Use Cases completely expanded

17. Near term Goals • Develop base set of IV&V techniques for • Requirements phase products assuming Use Case Artifacts. • Analysis phase products assuming class and sequence diagrams. • Design phase products assuming class, sequence, and deployment diagrams and state charts. • Solidify feasibility of applying • GQM for Requirements and Analysis Phases • BBN to gauge readiness for lifecycle phase transition