1 / 105

CS 406 Software Testing Fall 98 Part II : Functional Testing

CS 406 Software Testing Fall 98 Part II : Functional Testing. Aditya P. Mathur Purdue University. Last update: July 19, 1998. Part II: Functional testing. Learning objectives- What is functional testing? How to perform functional testing?

jacob
Download Presentation

CS 406 Software Testing Fall 98 Part II : Functional Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 406 Software TestingFall 98Part II: Functional Testing Aditya P. Mathur Purdue University Last update: July 19, 1998

  2. Part II: Functional testing • Learning objectives- • What is functional testing? • How to perform functional testing? • What are clues, test requirements, and test specifications? • How to generate test inputs? • What are equivalence partitioning, boundary value testing, domain testing, state testing, and decision table testing? Functional testing

  3. What is functional testing? • When test inputs are generated using program specifications, we say that we are doing functional testing. • Functional testing tests how well a program meets the functionality requirements. Functional testing

  4. The methodology • The derivation of test inputs is based on program specifications. • Clues are obtained from the specifications. • Clues lead to test requirements. • Test requirements lead to test specifications. • Test specifications are then used to actually execute the program under test. Functional testing

  5. Test methodology Specifications Clues Expected behavior Program output is correct Test requirements Oracle or Test specifications Program has failed; make a note and proceed with testing or get into the debug mode. Test driver Until specs. Exhausted. Actual behavior Program Functional testing

  6. Specifications • Inputs and tasks: • Given inputs • Perform tasks Functional testing

  7. Specifications-continued • Input properties • Input • must satisfy • Function f is a pre-condition on input Functional testing

  8. Specifications-continued • Two types of pre-conditions are considered: • Validated: those that are required to be validated by the program under test and an error action is required to be performed if the condition is not true. • Assumed: those that are assumed to be true and not checked by the program under test. Functional testing

  9. Specification: example • For the sort program: • Inputs are: • N • pointer to a sequence of length N • pointer to an area in memory where the output sequence is to be placed. Functional testing

  10. Specification: example..continued • Tasks to be performed: • Sort the sequence in ascending order • Return the sorted sequence in an area provided. • Return 1 if sorting is successful, -1 otherwise. Functional testing

  11. Preconditions for sort • Validated: • N>0 • On failure return -1; sorting considered unsuccessful. • Assumed: • The input sequence contains N integers. • The output area has space for at least N integers. Functional testing

  12. Deriving pre-conditions • Pre-conditions result from properties of inputs. • Example: • alpha_sequence(name) alpha_sequence is the string obtained from name by removing all characters other then A-Z, and a-z. Thus, if name is “A12C” then alpha_name is “AC”. Functional testing

  13. Deriving pre-conditions-continued • This leads to the following pre-condition: • Validated: the string alpha_sequence(name) is shorter than name. • On failure: print “invalid name”. • This property could also lead to the pre-condition: • Assumed: the string alpha_ sequence(name) is shorter than name. Functional testing

  14. Post-conditions • A post-condition specifies a property of the output of a program. • The general format of a post-condition is: • if condition then effect-1 {else effect-2} • Example: • For the sort program a post-condition is: • ifN>0 then {the output sequence has the same elements as in the input sequence and in ascending order.} Functional testing

  15. Post-condition-continued • This could be stated more formally as: • ifN>0 then { and each is a member of the input sequence and sort returns 1. } else { the output sequence is undefined and sort returns -1. } Functional testing

  16. Post-condition-continued • Another example: • if (A=B) and (B=C) then return “equilateral”; • Can you complete the above post-condition for a program that is required to classify a triangle given the length of three sides? • Convention: We will not nest if-then-else statements while specifying a post-condition. Functional testing

  17. Incompleteness of specifications • Specifications may be incomplete or ambiguous. • Example post-condition: if user places cursor on the name field then read a string • This post-condition does not specify any limit on the length of the input string hence is incomplete. Functional testing

  18. Ambiguous specifications • It also does not make it clear as to • whether a string should be input only after the user has placed the cursor on the name field and clicked the mouse or simply placed the cursor on the name field. and hence is ambiguous. Functional testing

  19. Clues: summary • Clues are: • Pre-conditions • Post-conditions • Variables, e.g. A is a length implying thereby that its value cannot be negative. • Operations, e.g. “search a list of names” or “find the average of total scores” • Definitions, e.g. “filename(name) is a name is no spaces.” Functional testing

  20. Clues-continued • Ideally variables, operations and definitions should be a part of at least one pre- or post-condition. • However, this may not be the case as specifications are not always written formally. • Hence look out for variables, operations, and definitions within a specification! Functional testing

  21. Test requirements • A test requirement is a description of how to test the program that is under test. • Here is a sample test requirement for a program that classifies a triangle given the length of three sides. • A, B, C are non-zero and positive. • One of A, B, C is negative; error condition. • One of A, B, C is zero; error condition. Functional testing

  22. Test requirements-derivation • Test requirements are derived from clues. • For example, consider the following pre-conditions (clues): • Assumed: A, B, and C are lengths • Validated: A>0, B>0, C>0 • These pre-conditions on A, B, and C lead to the test requirement given above. Functional testing

  23. Test requirements-derivation • Note that we have clumped pre-condition for each input variable into one condition. This is being done only for inconvenience. • It is recommended that pre-conditions be separated for each variable. Functional testing

  24. Test requirements-derivation • Note also that each validated pre-condition results in at least two requirements: one for the validated part and the other for the failure part. • In our example above we did not list all requirements. For example, we are content with testing “one of A, B, C is negative; error condition.” Functional testing

  25. Test requirements-derivation • Post-conditions also lead to test requirements. • For example, the partial post-condition: • if (A=B) and (B=C) then return “equilateral” leads to the following test requirement: • A=B and B=C. Functional testing

  26. Compound validated pre-conditions • Compound pre-conditions are ones that use the and or or connectors. • Examples: validated compound pre-conditions: • Pre-condition: A and B • Pre-condition: user places the mouse over the name field and clicks it. Functional testing

  27. Compound validated pre-conditions • The first of the above pre-conditions leads to four requirements: • A true, B true (This is the validated part) • A false, B true (This and the rest are failures) • A true, B false • A false, B false • You may work out the requirements for compound pre-condition with the or connector. Functional testing

  28. Compound validated pre-conditions • Compound validated pre-conditions could become quite complex. • Example: (A and (B or C)) • Brute force method will lead to 8 test requirements. Functional testing

  29. Compound validated pre-conditions • In general this will lead to too many test requirements. • We can prune them by leaving out those requirements that are unlikely to reveal a program error. • For example, consider the validated pre-condition: A or B. Functional testing

  30. Pruning test requirements • There are four possible test requirements: • A true, B true • A false, B true • A true, B false • A false, B false • Consider a correct C implementation: if (!(A || B)) exit_with_error(“Error: A is %d, B is %d”, A, B); else.. {/* the validated code comes here.*/} Functional testing

  31. Possible errors • Programmer forgets to check for one of the two cases resulting in the code: • if (!A) exit_with_error(“Error: A is %d, B is %d”, A, B); or • if (!B) exit_with_error(“Error: A is %d, B is %d”, A, B); Functional testing

  32. Possible errors-continued • Or use a wrong logical operator as in: if (!(A && B)) exit_with_error(“Error: A is %d, B is %d”, A, B); • Let us analyze how the four different tests will perform in each of the four implementations: one correct, and three incorrect ones. Functional testing

  33. Truth table: or condition A B !(A || B) !(A&&B) !A !B T F F T F T F T F T T F F F T T T T T T F F F F Inputs Correct implementation Incorrect implementations Notice this one: will it help find any of the three possible errors? Functional testing

  34. Truth table analysis Case 1: • A test input with A=true and B=false will cause the correct program to evaluate the condition to false. • The two incorrect implementations, !(A&&B) and (!B) will evaluate the condition to true. Functional testing

  35. Truth table analysis-continued • Both incorrect implementations will print the error message. • The oracle will observe that the correct and the incorrect implementations behave differently. • It will therefore announce failure for each incorrect implementation thereby pointing to an error. End of Case 1. Functional testing

  36. Truth table analysis-continued Case 2: • Test input A=false and B=true will reveal the error in the two incorrect implementations, !(A&&B) and (!A). Case 3: • Test input A=false and B=false might find a fault in the then branch of the if condition. Functional testing

  37. Truth table analysis-continued Case 4: • Test input A=true and B=true might find a fault in the else branch of the if condition. • Thus, all four test inputs are likely to be useful. Functional testing

  38. Truth table analysis-continued • However, if we were to check for the correct implementation of the condition A or B, then only the first two inputs are necessary. • In this example, reducing the number of test specifications from 4 to 2 does not lead to any significant savings. When will the savings be significant? Functional testing

  39. Assumed pre-conditions • Each assumed pre-condition is likely to result in a test requirement. • Example: • Assumed: MODE is “on ground” or “flying” • This leads to two requirements: • MODE is “on ground” , MODE is not “flying” • MODE is not “on ground” , MODE is “flying” Functional testing

  40. Assumed pre-conditions • These can be simplified to: • MODE is “on ground” • MODE is “flying” Functional testing

  41. Test requirements checklist • Obtaining clues and deriving test requirements can become a tedious task. • To keep it from overwhelming us it is a good idea to make a checklist of clues. • This checklist is then transformed into a checklist of test requirements by going through each clue and deriving test requirements from it. Functional testing

  42. Test specifications • A test requirements indicates “how” to test a program. But it does not provide exact values of inputs. • A test requirement is used to derive test specification, which is the exact specification of values of input and environment variables. Functional testing

  43. Test specifications-continued • There may not be a one-to-one correspondence between test requirements and test specifications. • A test requirement checklist might contain 50 entries. These might result in only 22 test specifications. • The fewer the tests the better but only if these tests are of good quality! Functional testing

  44. Test specifications-continued • We will discuss test quality when discussing test assessment. • A test specification looks like this: • Test 2: • global variable all_files is initially false. • next_record is set to 1. • Upon return expect: • all_files to be true • next_record is last_record+1 Functional testing

  45. Test specifications-continued • Notice the format of a test specification: • Each test is given a number which serves as its identifier. • There is a set of input values. • There is a set of expected values upon return from execution. Any side effects on files or networks must also be specified here. In essence, all observable effects must be specified in the “Expect” part of a test specification. Functional testing

  46. Test specifications-continued • Any side effects on files or networks must also be specified. In essence, all observable effects must be specified in the “Expect” part of a test specification. • Similarly, values of all input variables, global or otherwise, must also be specified. Functional testing

  47. Test requirements to specifications • The test requirements checklist guides the process of deriving test specifications. • Initially all entries in the checklist are unmarked or set to 0. • Each time a test is generated from a requirement it is marked or the count incremented by 1. Functional testing

  48. Test requirements to specifications • Thus, at any point in time, one could assess the progress made towards the generation of test specifications. • One could also determine how many tests have been generated using any test requirement. Functional testing

  49. Test requirements to specifications • Once a test requirement has been marked or its count is more than 0 we say that it has been satisfied. • Some rules of thumb to use while designing tests: • Try to satisfy multiple requirements using only one test. • Satisfy all test requirements. Functional testing

  50. Test requirements to specifications • Avoid reuse of same values of a variable in different tests. Generating new tests by varying an existing one is likely to lead to tests that test the same part of the code as the previous one. In testing, variety helps! • Though we try to combine several test requirements to generate one test case, this is not advisable when considering error conditions. Functional testing

More Related