Software Verification and Validation (V & V) by Roger U. Fujii and Dolores R. Wallace

1. See venus.cs.depaul.edu/se682/Presentations/VandV.ppt The slides we’ll cover are marked with a Software Verification and Validation (V & V)by Roger U. Fujii and Dolores R. Wallace Presented by Sanjay Patel

2. Agenda • Introduction • Overview Of Software Verification and Validation • Objectives of V & V • Responsibilities of V & V • Organizing a V & V Effort • Applying V & V to a Software Life Cycle • Effectiveness of V & V • Standards and Guidelines For Planning and Managing V & V • Summary • Questions

3. Introduction • Software V & V defined as a systems engineering methodology to ensure that quality is built into the software during development. • Analysis and test activities are performed by V & V in order to evaluate and assess the software products and development processes during each SDLC phase. • Software V & V is complementary to and supportive of quality assurance, project management, systems engineering, and development.

4. Introduction (Contd…) Relationship of V & V to the development team

5. Overview of Software Verification and Validation • Software Verification Process is a process for determining whether the software products of an activity fulfill the requirements or conditions imposed on them in the previous activities. • Software Validation Process is a process for determining whether the requirements and the final, as-built system or software product fulfills its specific intended use.

6. Refinement Design Review Requirement Review Code Inspection BlackBox Testing Requirement Specification Design Specification Implementation Validation: Requirement Against itself Verification: Implementation Against Design Verification: Design Against Requirement Verification and Validation

7. Objectives of V & V • Software V & V determines that the software performs its intended functions correctly. • Ensure that the software performs no unintended functions. • Measure and assess the quality and reliability of software. • As a systems engineering discipline, software V & V also assesses, analyzes, and tests the software on • how it interfaces with systems elements • Influences the performance, or reacts to stimuli from system elements

8. Responsibilities of V & V Versus Other Groups

9. Organizing a V & V Effort • Whom does the V & V reports? • Four methods of organizing a V& V effort are described: • Independent V & V • Embedded in the development system engineering group • Embedded in the development quality assurance group • Embedded in the user group.

10. Independent V & V • Independent of the development group and is called Independent V&V or IV&V • The IV& V group reports directly to the system program manager. • The V & V group establishes formal procedures for receiving software releases and documentation from the development team. • V & V tasks are oriented towards engineering analysis (algorithm analysis, control/data flow analysis), and comprehensive testing (such as simulation) • Main objective is to determine whether the software satisfies critical system requirements. • Sends all evaluation reports and discrepancy reports to both the program manager and development group.

11. Independent V & V (Contd..) • Advantages: • Detailed analysis and test of software requirements. • Early detection of high risk software and system errors. • Disadvantages: • Higher cost to the project • Additional development interfaces

12. V & V effort embedded in the Systems Engineering Group • Embedded in development’s systems engineering group • V & V tasks are to review the group’s engineering analyses (like algorithm development, sizing/timing), and testing (like test evaluation or review of the adequacy of the development test planning document) • V & V’s results are reviewed and monitored by systems engineering and quality assurance groups. • Advantages: • Minimize cost impact to the project • No system learning for the staff • No additional development interfaces • Disadvantages: • Loss of engineering analysis objectivity

13. V & V effort embedded in the development’s quality assurance group • V & V group is part of quality assurance and maintains its relationship with systems engineering and other development groups in the same manner as quality assurance. • Tasks include monitoring, auditing, reviewing content (for example, audit performance, audit support, test witnessing, walk-through support, documentation review) • Advantages: • Low cost to the project • Bringing V & V analysis capabilities into reviews, audits, and inspections • Disadvantages: • Loss of an independent software systems analysis and test capability

14. V & V effort embedded in the user group • Tasks consists of • configuration management support of development products • support of formal reviews • user documentation evaluation • test witnessing • test evaluation of the development test planning documents • user testing support • Advantages: • Strong systems engineering and user perspective that can be brought to bear on the software product during development • Disadvantages: • Loss of detailed analysis and test of incremental software products

15. Applying V & V to a Software Life Cycle • Minimum recommended V & V tasks that are required by the ANSI/IEEE Standard for Software Verification and Validation Plans (SVVP) for each development phase. • Considered effective and applicable to all types of software applications. • A V & V task specifies minimum input and output requirements for each V & V task • A V & V task may not begin without specific inputs, and is not complete until specific outputs are completed.

16. Applying V & V to a Software Life Cycle Minimum set of recommended V & V tasks

17. Applying V & V to a Software Life Cycle • Management of V & V • Concept Definition Evaluation • Requirements Analysis • Design evaluation • Implementation (Code) evaluation • Testing • Installation and Checkout activities • Operations and Maintenance evaluation and test.

18. Applying V & V to a Software Life CycleManagement of V & V • Management tasks for V & V span the entire life cycle. • Tasks include: • Plan the V & V process • Coordinate and interpret performance and quality of the V & V effort • Report discrepancies to the user or development group • Identify early problems and focus V & V activities on them • Provide a technical evaluation of the software performance and quality at each major software review • Assess the full impact of proposed software changes • Output of the V & V activities consists of: • Software Verification and Validation Plan (SVVP) • Tasks reports • Phase summary reports • Discrepancy reports • Final report

19. Applying V & V to a Software Life Cycle Concept Definition Evaluation • The main task is to evaluate the concept documentation to determine whether the defined concept satisfies user needs and project objectives (for example, statement of need, project initiation memo) in terms of system performance requirements, feasibility (for example, overestimation of hardware capabilities), completeness, and accuracy. • The evaluation assesses the criticality of each software item defined in the concept Conceptual Model

20. Applying V & V to a Software Life Cycle Requirements Analysis • V & V tasks verify the completeness of all the requirements • Common tasks during requirement analysis include control flow analysis, data flow analysis, algorithm analysis, and simulation. • V & V test planning begins in the requirements phase and spans almost full range of life cycle phases and it consists of : • Component testing • Integration testing • System testing • Acceptance testing

21. Applying V & V to a Software Life Cycle Design evaluation • Most commonly used tasks are algorithm analysis, database analysis, timing/sizing analysis, and simulation. • These V & V tasks during the design phase assures that: • Requirements are not misrepresented or incompletely implemented. • Unintended requirements are not designed into the solution • Requirements are not left out of the design

22. Applying V & V to a Software Life Cycle Implementation (Code) evaluation • Commonly used V & V tasks are control flow analysis, database analysis, regression analysis, and sizing/timing analysis. • Other tasks for code evaluation are walk-throughs, code inspection, and audits

23. Applying V & V to a Software Life CycleTesting • V & V test technique is to develop test cases that violate software requirements. • V & V testing is effective at probing for errors and weaknesses that reveal hidden faults. It often finds errors for which some functional, or black box, test cases can produce the correct output despite internal errors. • Optional tasks include regression analysis and test, simulation, and user document evaluation

24. Applying V & V to a Software Life Cycle Installation and Checkout activities • V & V validates that the software operates correctly with the operational hardware system and with other software, as specified in the interface specifications • Optional tasks include regression analysis and tests, simulation, and test certification

25. Applying V & V to a Software Life CycleOperations and Maintenance evaluation and test • Any changes that are made during the operations and maintenance phase will result in repeating all life cycle phase V & V activities in order to make sure that nothing is overlooked • V & V activities can be added or deleted depending on the type of changes made

26. Effectiveness Of V & V • Varies with project size, project complexity, and V & V staff experience • Some positive effects of V & V on a software project include: • Better quality • More rigorous development planning • Better adherence by the development organization to programming language, developments standards and configuration management practices. • Early error detection • Better schedule compliance and progress monitoring • Greater project management visibility into technical quality and progress • Better criteria and results for decision making at formal reviews and audits.

27. Effectiveness Of V & V • Negative effects of V & V on a software development project include: • Additional project cost of V & V (10% – 30 %) • Additional interface involving the development team, user, and V & V organization • Lower development staff productivity

28. Standards and Guidelines for Planning and Managing V & V Planning V & V with guidance from V & V documents

29. Summary • Software V & V is a proven systems engineering discipline for generating correct and quality software • Uncovers high risk errors early • Evaluates the correctness of products against system and software requirements. • Provides the user an incremental preview of system performance, with the chance to make early adjustments. • Provides decision criteria for whether or not to proceed to the next development phase • Software V & V has become a powerful risk management tool by providing the detailed technical insight into the “true” performance of the software

30. Questions ?????