CS 425/625 Software Engineering Verification and Validation

CS 425/625 Software EngineeringVerification and Validation Based on Chapter 19 of the textbook [SE-6] Ian Sommerville, Software Engineering, 6th Ed., Addison-Wesley, 2000 and on the Ch19 PowerPoint presentation available at the book’s web-site: www.comp.lancs.ac.uk/computing/resources/IanS/SE6/Slides/index.html November 10, 2003

Outline • Introduction • Testing and Debugging • Verification and Validation Planning • Software Inspections • Program Inspections (forms of Software Inspections) • Automatic Static Analysis

Introduction….. • Verification and validation (V&V): the checking and analysis processes that ensure the software satisfies its specification and meets the needs of the clients who are paying for it • Validation: “Are we building the right product?” • Verification: “Are we building the product right?” • Verification involves checking the software conforms with its specification while the more general process of validation ensures the software meets the needs of the clients • V&V is a whole life-cycle process, encompassing requirements reviews, design reviews, code inspections, and program testing

.Introduction.… • V & V techniques: • Software inspections • Software testing • Software inspections • Are static V&V techniques as they do not require the software to be executed • Consist of inspections, automated static analyses, and formal verifications of source code or system models • Can only check the correspondence between the software and its specification • Cannot demonstrate the system is operationally useful • Cannot check non-functional requirements of the software

..Introduction... • Software testing • It is a dynamic V&V technique as it needs an executable version of the software system • The system is executed with test data and its operational behaviour is assessed • Can reveal the presence of errors, not their absence • A successful test is a test which discovers one or more errors • The V&V technique for checking non-functional requirements and for verifying system integration • Should be used in conjunction with static techniques to provide full V&V coverage

…Introduction.. • Types of software testing • Defect testing • Intended to find inconsistencies between a program and its specification • Tests designed to discover program faults and defects • A successful defect test is one which reveals the presence of defects in a system • Statistical testing • Designed for software’s performance and reliability estimation • By running tests that reflect actual user inputs and their frequency, an estimate of operational reliability can be made

….Introduction. • Static and dynamic V&V [Fig 19.1, Somm00]

…..Introduction • Verification and validation should establish confidence that the software is fit for purpose • This does not mean the software is completely free of defects; rather, it must be good enough for its intended use • The required level of confidence depends on: • Software’s purpose: the level of confidence depends on how critical the software is to an organisation • User expectations: users may have lower expectations of certain types of software • Marketing environment: getting a product to market early may have higher priority than finding its defects

Testing and debugging. • Defect testing and debugging are distinct processes • V&V is concerned with establishing the existence of defects in a program • Debugging is concerned with locating and repairing these errors • Debugging involves formulating hypotheses about program behaviour then testing these hypotheses to find the errors

.Testing and debugging The debugging process [Fig. 19.2, Somm00]

V & V Planning.. • The planning of V&V should start early in the development process • The plan should balance static verification and testing, specify testing standards and procedures, establish checklists for inspections, and define the software test plan • Test planning breaks down V&V into a number of stages, often organized according to the V-model (shown on next page) • The focus is on setting standards and procedures for inspections and testing, not on describing product tests

The V-model of development [Fig. 19.3, Somm00] .V&V Planning.

..V&V Planning • The structure of a software test plan • The testing process • Requirements traceability • Tested items • Testing schedule • Test recording procedures • Hardware and software requirements • Constraints

Software Inspections. • Involve people examining software code or models (representations) with the aim of discovering defects • Do not require execution of a system thus can be used throughout the development process • May be applied to any representation of the system: requirements, design, test data, etc. • Very effective technique for discovering errors

.Software Inspections • In software inspections many different defects can be discovered in a single review of the source code or software model • In testing, one defect may mask another hence several executions are required • Software inspections reuse domain and programming knowledge so reviewers are likely to have seen the types of error that commonly occur • Software inspections and software testing are complementary, not competing techniques (see also slides 4 and 5)

Program Inspections……. • Program inspections are a type of software inspections • Consist of formal reviews conducted by teams and intended for program defect detection • Defects may be logical errors, anomalies in the code that might indicate an erroneous condition (e.g., an un-initialized variable), or non-compliances with standards

.Program Inspections…… • Inspection pre-conditions: • A precise specification must be available • Team members must be familiar with the organisation standards • Syntactically correct code must be available • An error checklist should be prepared • Management must accept that inspection will increase costs early in the software process • Management must not use inspections for staff appraisal

..Program Inspections….. The inspection process [Fig. 19.6, Somm00]

…Program Inspections…. • Program inspection procedure: • The inspection is planned by the moderator (or chairman) • The system overview is presented to the inspection team by the program author (or owner) • The code and associated documents are distributed to the inspectors (inspection team) in advance for individual preparation • The inspection meeting takes place and errors are noted (the inspection also involves a reader and, possibly, a scribe) • During re-work modifications are made by the author to repair discovered errors • Re-inspection may or may not be required, based on moderator’s decision

….Program Inspections… • Inspection teams are made up of at least 4 members: • Author of the code being inspected • Inspector who finds errors, omissions and inconsistencies • Reader who reads the code to the team • Moderator who chairs the meeting and notes discovered errors • Other roles are chief moderator and scribe

…..Program Inspections.. • Inspection checklist • A checklist of common errors should be used during the inspection • This error checklist is programming language dependent • The weaker the type-checking of the language, the larger the checklist • Examples of possible errors: initialisation, constant naming, loop termination, array bounds, etc.

Inspection checks (fault classes) [Fig. 19.7, Somm00]

…….Program Inspections • Inspection rates: • 500 statements/hour during overview • 125 source statement/hour during individual preparation • 90-125 statements/hour can be inspected • Inspection is therefore an expensive process • Inspecting 500 lines costs about 40 man/hours effort = £2800

Automated Static Analysis…. • Static analyzers are software tools for source code processing • They parse the program text to discover erroneous conditions • Very effective as an aid to inspections; supplement but cannot replace inspections • Particularly valuable for languages such as C that have weak typing (many errors can remain undetected by the compiler) • Less cost-effective for languages such as Java that have strong type checking (many errors can be detected during compilation)

.Automated Static Analysis… Automatic static analysis checks [Fig. 19.8, Somm00]

..Automated Static Analysis.. • Stages of static analysis: • Control flow analysis. Checks for loops with multiple exit or entry points, finds unreachable code, etc. • Data use analysis. Detects un-initialised variables, variables written twice without an intervening assignment, variables which are declared but never used, etc. • Interface analysis. Checks the consistency of routine and procedure declarations and their use

…Automated Static Analysis. • Stages of static analysis [cont’d]: • Information flow analysis. Identifies the dependencies of output variables. Does not detect anomalies itself but highlights information for code review (inspection) • Path analysis. Identifies paths through the program and sets out the statements executed in that path. Again, potentially useful in the review process

138% more lint_ex.c #include <stdio.h> printarray (Anarray) int Anarray; { printf(“%d”,Anarray); } main () { int Anarray[5]; int i; char c; printarray (Anarray, i, c); printarray (Anarray) ; } 139% cc lint_ex.c 140% lint lint_ex.c lint_ex.c(10): warning: c may be used before set lint_ex.c(10): warning: i may be used before set printarray: variable # of args. lint_ex.c(4) :: lint_ex.c(10) printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(10) printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(11) printf returns value which is always ignored LINT static analysis [Fig 19.9, Somm00]

CS 425/625 Software Engineering Verification and Validation