Download
1 / 29
290 likes | 413 Views
S/W Quality I. Unit and Regression Testing Lesson 5. Why Test ?. “Are you kidding me ?!?! I need to test my code ? Aren’t there people out there whose job it is to do testing ?”
E N D
S/W Quality I Unit and Regression Testing Lesson 5
Why Test ? • “Are you kidding me ?!?! I need to test my code ? Aren’t there people out there whose job it is to do testing ?” • At times throughout your career as a Software Engineer you may find yourself uttering these words … but when that happens just remember … • Who is better suited with the knowledge of the code than you to do some level of testing on your own code ? • The most important idea and lesson that you can take away in terms of maintaining a high level of Software Quality is to • Begin your testing plans as you begin your software design • Create, implement and execute your tests as you develop your software • Developing and Testing at the same time involves much less effort than waiting until everything is coded to begin • Remember, accept and embrace the idea that testing is not beneath you … • Please see the “Why Unit Test” Lesson Resources
Okay – so I have to test … • Now you’re wondering exactly what needs to be tested, when, how often, how, … • These are all very good questions – let’s take a look at the breakdown of the different families / groups of tests and their members and as well the factors that you should be testing for … • The 2 large families of tests include Structural and Functional types of tests • We will be exploring the intricacies of Structural and Functional tests next term in SQ-II ! • You’ll also notice that this breakdown of tests is examining the software and system for many different factors • Unit Testing could be viewed as the little brother to these groups • Unit testing often times is made up of a mixture of both Structural and Functional test types
Where to start … • Wow ! That’s quite a set of tests that needs to be completed … • Don’t despair however – you don’t need to complete all of these types of tests at once … • In the SEF, SAaD and your programming courses, you have learned about the 5 different levels of testing • Unit Testing, Integration Testing, System Testing, User-Acceptance Testing and Regression Testing • Each of levels of testing focuses on different aspects of the software / system being developed and tested and each of these levels of testing is generally performed by different project members and is performed for different purposes • The following table summarizes this information for you … • As for learning more about the different Structural and Functional types of tests – more on that in SQ-II next term …
Where to start … (cont’d) • As previously stated – the best way to ensure that quality is built into your system and software is to begin developing your Test Plan as soon as you begin gathering your requirements • Please note that the development of a Testing Strategy, Test Plan and the various Test Specifications are all parts of the formal testing process commonly used in most traditionally managed projects • We’ll be talking more about these documents, strategies and plans next term in SQ-II ! • However very similar ideas are followed and used in projects managed in an Agile way … • The goal is still risk reduction, feature/story completion and test-driven development (TDD)
What is a Unit ? • A Unit is the smallest component that can be treated as a “whole” within a system being developed • Can be as simple as a single programming statement, or as complex as a pair of connected procedures or methods • Generally a unit is viewed as : • Function in procedural languages (in C) • Method in object oriented languages (in C++, Java, C#)
Why Unit Test ? • To help programmers verify code modules • To find / correct issues as early as possible • test driven development (TDD) approaches (like XP, SCRUM) • To facilitate and enable Change Management • Regression Tests • To test for cohesiveness and coupling • Allows easier and more versatile integration • Provides a form of documentation about the code module being tested
Unit Testing Limitations ? • Unit Tests : • Do not assess system performance • Do not test all user requirements • Do not test for compliance (or any other Structural tests) • And so on… • In short, Unit Tests should not be the only tests conducted • But instead are a key , first set of tests completed within a System Development • They form the basis of the overall Testing Strategy
Who does it and when ? • Generally Unit Tests are performed by the developers themselves • Often times Unit Tests can and should be scripted (or automated) due to the number of test cases and the number of times they may be run • Unit Tests are most effective when executed as soon as a unit is completed • Unit Tests are also executed when changes are made that may affect a unit • these are called Regression Tests
Unit Testing How are they conducted ? How are they documented ?
How are they Conducted ? • Unit Tests are very careful to isolate a unit from other units • This can be done through the use of : • Stubs • Procedures/functions/subroutines/methods that are “place holders” • Really don’t do anything other than allow a call to take place • Test Harnesses • A small program that will test the unit • Setting up pre and post conditions • Remember when we discussed Use Case Descriptions ? • Sometimes it is necessary to setup a condition or state within the software being tested in order to fully and properly test the unit
Unit Test Description • It is important to realize that Unit Tests (and unit testing) are only a part of the overall Test Strategy • Test Strategies are documented in a Test Plan. • In its simplest form, the Test Plan contains a section on Unit Testing which has a Unit Test Description for each unit test used • A Unit Test Description contains: • Test identifier • A unique identifier - usually a number (or dotted number) • Short description of the test • Method of execution • Manual, automatic, sequence of operations, etc. • Input data (and/or input commands) • Expected outputs (data, messages, exceptions) • Observed outputs • Resulting actions if test fails
Special Note on Creating the Test Identifier • As I just mentioned, you need to somehow create a unique, meaningful and traceable unit test identifier • One approach would be to define legends which enumerate your sub-systems, each sub-system’s modules/classes, each sub-system module’s/class’ functions and methods and then use this as the test identifier • For example : if you have defined your Validation sub-system with identifier “V” and a class in that sub-system dealing with integer field validation as “I” and a method in that class dealing with acceptable range of values as method “1” – then a test identifier for this method might be V.I.1.1 (for the first test) • Another approach (and perhaps this approach proves more cohesive within the overall SDLC) is to remember all of the diagrams (traditional and UML) that you would have created about your system and its design by this point • Each one of those diagrams identified the sub-systems, modules/classes as well as the routines and methods – perhaps a good test identifier builds on those diagrams • For example remember in a DFD diagram, each sub-system is given its own number (e.g. Validation is sub-system 1), each processing step within that sub-system is given it’s own sub-number (integer validation is 1.1 in the DFD)and (if you diagram to this level) – each element of the sub-step is given its own sub-sub-number – so from this, your test identifier might be 1.1.1.1 • The nice thing about building upon previous project/system documentation is that it unifies the system overall
Automated Unit Tests • Set of functions or methods executed in order to invoke and to execute a unit of code, and confirm the results • Generally, each automated function or method tests one aspect of a unit • Analogous to the “description” in the previous slide • The results of the automated unit tests can be validated through an assertion • A function / method (or class of methods) that compares an actual output with a desired output • There are very good frameworks which help you automate your Unit Testing • But don’t forget that you yourself can write your own automated unit tests • Again, to build in quality it is highly recommended that unit tests are created as code is developed
NUnit • NUnit offers an automated unit-testing framework for all .NET languages and is open-source • It is possible to also integrate this framework into Visual Studio (2005, 2008 and 2010) • NUnit is a derivation of JUnit (developed for Java unit testing) and ultimately a variation of XUnit (a family name given to a testing framework developed for unit testing multiple development languages)
Visual Studio and Unit Testing … • The following link should be read in detail and the examples followed in order to understand how Visual Studio 2010 helps developers integrate their Unit Testing into their solution development • Click here for an Overview of VS2010 Unit Testing • Click here for an Overview of VS2012 Unit Testing • Click here for Details on VS2010 Unit Testing for C++ • Please note that VS2008 (and VS2010) do not easily or readily support unit testing of native C++ code • Here are some links to a good example that shows you how to setup, manage and execute unit tests in a C# program • Click here for VS2010 Unit Testing • Click here for VS2012 Unit Testing • Let’s work through the example … • The results of the unit tests are validated through an assertion • A special class of methods that compares an actual output with a desired output – with many possible variations
Regression Tests • “Yes , Virginia, there is a Santa Claus” • And Yes, Developer, there are Regression Tests ! • Good Traditional Project and System’s Development Objectives are to : • Always ensure that System Documentation is current • Always ensure that System Test Data and Test Conditions are current • Regression Testing is the type of testing that : • Confirms that previously tested functionality performs as required and expected after changes to the system are implemented • A project’s testing strategy would indicate when regression testing needs to be completed (i.e. after what types of changes, etc.)
Regression Tests (cont’d) • Should be performed when there is a high risk due to changes • In Development, could be done at points where a certain number of changes are made • In Maintenance, should be done when the potential risk is high due to the nature of the changes • Examples • Rerun previous tests • Note that a key to regression testing is that it is not necessarily the case that all Unit Tests (for example) need to be rerun to perform a regression test • It is more so the case that only a subset of Unit Tests are rerun in order to achieve a certain code coverage • Review previously prepared manual procedures
Test … Test … Everywhere a TEST • This brief lesson is meant to open your mind to the idea that testing plays a key and pivotal role in the development of quality software • This lesson is also meant to drive home the point that you (as the architect of a system, as the designer of a system, as the developer of a system) always need to be thinking about testing • It is your job to ensure that the code you develop does what you actually intended it to do … and how best to ensure that than through testing ?!?! • Next term in SQ-II we will be revisiting the idea of testing when I describe to you the whole enchilada !!
