1 / 42

Testing (Continued)

Learn about Test-Driven Development, types of tests, differences between system tests and unit tests, test frameworks, testing tools, code coverage, and good coding style in software testing.

hurtm
Download Presentation

Testing (Continued)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Testing (Continued)

  2. Test-Driven Development • The tests are the detailed specification • And now can be executed, not just read More tests Fewer specification documents

  3. Types of Tests • System: End-to-end tests • User-like input to end-user output End to End Tests • Unit Tests • Test small pieces of system (single classes / functions) Unit Tests

  4. System Tests vs. Unit Tests • Strengths/Weaknesses of System Tests • Strengths/Weaknesses of Unit Tests

  5. System Tests vs. Unit Tests • Strengths/Weaknesses of System Tests • Test entire system  no gaps • Hard to debug:problem could be anywhere! • Fragile: often test exact user inputwould produce exact output text / file • Strengths/Weaknesses of Unit Tests • Easier to debug:start with 1 class / function • Less fragile: Don’t check user interface (I/O): check API / class does what it should with code (testfixture) • Don’t test whole system  gaps possible

  6. Types of Tests • Fewer (but not 0!) End to End Tests • Integration Tests • Bigger unit tests • But now test multiple classes / bigger functionality • Moderate difficulty to debug Integration Tests Unit Tests • More

  7. Unit Tests • Want: • To test individual classes / functions • How? • Carefully chosen user input? • Output ... No! structPoint { float x; float y; t_point& operator*=(float rhs); . . .

  8. Unit Tests • Write new code • To put class in right state • To directly send it some input / make some calls • To immediately check the responses • Test driver

  9. Unit Tests myCode.cpp struct Point { float x; float y; t_point& operator*=(float rhs); . . . tester.cpp int test1 () { Point testme (1, 2); testme *= 3; if (testme != Point (3, 6)) { cout << “uh – oh” << endl; return (1); } return (0); } test_main.cpp g++ myCode.cpp main.cpp –o prog.exe int main () { int error = 0; error += test1(); . . . // Lots more tests g++ myCode.cpp tester.cpp test_main.cpp –o test.exe

  10. Unit Test Frameworks • Lots of repetitive code to create test drivers • Set up the test • Check if the test passed • Run all the tests • Output appropriate messages • Collect statistics • Unit Test Frameworks • Useful macros (#defines) and functions to simplify coding • We are using UnitTest++ • Powerful, but easy to learn • ECE 297 Unit Test Quick Start Guide TEST(..) or TESTFIXTURE (…) CHECK(..) RunAllTests() Automatic Automatic

  11. “Test the Seams” • Overlapping tests good • Your code + partner’s code • Both unit tested • Make sure there’s an integration test • Therac-25

  12. Testing Tools • What tools? • Debugger • Use to debug when a test fails • Use to verify new code • Step through it and watch execution • Memory checker • Program seg faults? • Program behaving very strangely? • Maybe you are accessing memory you shouldn’t be! • Run debug_check configuration and/or valgrind

  13. Testing Tools 3. Code coverage • Tools that can track what lines of your programs have executed over all your tests MyCode.cpp int someFunc (int input) { if (input == 0) return (3); else return (7); } main.cpp int main () { int j = someFunc (8); // Wow I’m bad at testing! }

  14. Testing Tools 3. Code coverage • Tools that can track what lines of your programs have executed over all your tests MyCode.cpp int someFunc (int input) { if (input == 0) return (3); else return (7); } No test reaches this line. Code coverage: 6 out of 7 lines or 86% main.cpp int main () { int j = someFunc (8); // Wow I’m bad at testing! }

  15. Google: Testing so Important, Tutorials in the Bathroom

  16. “Testing on the Toilet”

  17. Automate Build & Tests • Make it easy to do the right thing • Ece297: • One command to build • One command to test • One command to submit • Google • Also easy to build, test • Easy to deploy • Send to the real web site and real users • Split into A/B test (e.g. 2% of web traffic gets new ad engine) • Easy to reverse deployment if a bad idea • Continuous deployment: requires good automated tests

  18. Good Coding Style

  19. 1. Use White Space int sumVec (int vec[], int nElem) { int i, result = 0; for (i = 0; i < nElem; i++) { result += vec[i]; } return (result); } void nextFunc (int i) { … White space: show code organization • Indent properly (3 or 4 spaces) per { }. • Leave blank lines between functions / key blocks int sumVec (int vec[], int nElem) { int i, result = 0; for (i = 0; i < nElem; i++) { result += vec[i]; } return (result); } void nextFunc (int i) {

  20. What does this do? float di (float a, float b) { float val, d, x, x2, y; d = 1.e-4; val = 0; for (x = a; x < b; x += d) { x2 = x + d; if (x2 > b) x2 = b; y = 0.5 * ((1. / x) + (1. / x2)); val += y * (x2 - x); } return (val); }

  21. What does this do? float definite_integral (float x_left, float x_right) { float integral, step_size, x1, x2, y_average; step_size = 1.e-4; integral = 0; for (x1 = x_left; x1 < x_right; x1 += step_size) { x2 = x1 + step_size; if (x2 > x_right) x2 = x_right; y_average = 0.5 * ((1. / x1) + (1. / x2)); integral += y_average * (x2 – x1); } return (integral); }

  22. 2. Descriptive Variable Names • Use descriptive names • Variables, functions, structs/types, … • get_file_name ( ); // Use _ to separate • getFileName (); // Or use upper case to mark words • Types: start with a capital letter • Variables: start with lowercase • class MyClass { … • MyClass oneVar;

  23. What does this do? float definite_integral (float x_left, float x_right) { float integral, step_size, x1, x2, y_average; step_size = 1.e-4; integral = 0; for (x1 = x_left; x1 < x_right; x1 += step_size) { x2 = x1 + step_size; if (x2 > x_right) x2 = x_right; y_average = 0.5 * ((1. / x1) + (1. / x2)); integral += y_average * (x2 – x1); } return (integral); }

  24. Comment what whole function does // Compute the definite integral of 1/x between x_left and x_right via the // trapezoidal method. Smaller values of step_size improve accuracy, but // increase computation time. float definite_integral_of_one_over_x (float x_left, float x_right) { float integral, step_size, x1, x2, y_average; step_size = 1e-4; integral = 0.; for (x1 = x_left; x1 < x_right; x1 += step_size) { x2 = x1 + step_size; if (x2 > x_right) // in case (x_right – x_left) is not a multiple of step_size x2 = x_right; y_average = 0.5 * ((1. / x1) + (1. / x2)); // average of y(x1) and y(x2) integral += y_average * (x2 – x1); } return (integral); } Comment any tricky bits of code

  25. asteroids.cpp Comments: Usefulness? /* This file implements an asteroid AI. It maintains * a current position and velocity for each asteroid and * the ship. It divides the screen into N bins, and * computes the desirability of moving the ship * to each bin by computing the time at which the ship * could it, and the bin’s asteroid density at that time. */ StudentRecord.h /* Main data structure used to store all information * about a U of T student. Linked list. */ structStudentRecord { intnClassesCurrent; // Number enrolled, current semester intnClassesComplete; // Numbercompleted& passed StudentRecord*next; // Pointer to next (linked list) record program.cpp // Compute the sum of the array, over all its elements int sum = 0; for (int i = 0; i < nElem; i++) sum += array[i];

  26. 3. “High-Level” Comments • Most important comments: give the big picture • Documentation should be in the comments • Not a separate document  will get out of date • Top of files // Functions to simulate resistor network. Proceeds in 6 stages … • Class / data structure definitions • Understand the data  can understand the program! • Start of functions • Tricky code Not useful: • Comments that translate C++ to English Most important Leastimportant

  27. Thoughts on This Code? int checkWeights (int weights[20]) { for (int i = 0; i < 20; i++) { if (weights[i] < 0) return (-1); if (weights[i] == 0) return (0); } return (1); } • Using a “magic number”: 20 • Change array size: must find and change all 20’s • Returning magic numbers: -1, 0, 1 • Must read code carefully to see what each means

  28. 4. Use Named Constants const int NUM_WEIGHTS = 20; // 1. Constant variable #define WEIGHT_ZERO 0 // 2. Pre-processor constant enumWtReturn {HAS_NEG = -1, HAS_ZERO = 0, ALL_POS = 1}; // 3. make an “enumeration” (list) of int constants int checkWeights (int weights[NUM_WEIGHTS]) { for (int i = 0; i < NUM_WEIGHTS; i++) { if (weights[i] < 0) return (HAS_NEG); if (weights[i] == 0) return (HAS_ZERO); } return (ALL_POS); } • Three ways to make constants  use any way you like • Name: ALL CAPITALS (convention)

  29. 5. Many Short Functions Short functions • Easier to re-use • Fix bugs in one place, not many • Make code easier to read: more abstraction • How long should functions be? • Should have many 5 to 10 line functions • Should very rarely have a function > 100 lines

  30. Thoughts on This Code? #include <math.h> void myFunc (float a[], float b[], int nvals) { float absAvg1 = 0; for (int i = 0; i < nvals; i++) absAvg1 += abs(a[i]); absAvg1 /= nvals; float absAvg2 = 0; for (int i = 0; i < nvals; i++) absAvg2 += abs(b[i]); absAvg2 /= nvals; ... } There is no honour in copy and paste coding! Refactor!

  31. Better Version? #include <math.h> float compAbsAvg (float array[], int nvals) { float absAvg = 0; for (int i = 0; i < nvals; i++) absAvg+= abs (array[i]); return (absAvg / nvals); } void myFunc (float a[], float b[], int nvals) { float absAvg1 = compAbsAvg (a, nvals); float absAvg2 = compAbsAvg (b, nvals); ... } 1. Not much shorter, but easier to read 2. Less chance of more code copying  future code will be shorter

  32. 6. Don’t Do Too Much in a Statement / Line #include <string> #include “StreetsDatabaseAPI.h” string name = getIntersectionName(getStreetSegmentInfo(getIntersectionStreetSegment(0,startInter)).to); // Hard to read! unsigned firstSeg= getIntersectionStreetSegment (0,startInter); unsigned destInterId = getStreetSegmentInfo(firstSeg).to; string destInterName = getIntersectionName (destInterId); // Show your work  divide into several steps on several lines // Use good variable names to show what intermediate results // are.

  33. 7. Group Related Data into Classes vector<int> fromIntersectionofSegment; vector<int> toIntersectionofSegment; vector<float> speedLimitofSegment; vector<double> lengthofSegment; class SegmentData{ public: intfromIntersectionIdx; inttoIntersectionIdx; bool oneWay; // If true, can only go from -> to float speedLimit; // In km/h double travelTime (); // Returns travelTime in s } vector<SegmentData> segmentInfo; // [0..numStreetSeg-1] 1. Groups related data and functions 2. Less name pollution

  34. 8. Defensive Coding • Use assertions to verify assumptions in your code void myFunc (int *ptr) { // Don’t ever call myFunc with a NULL ptr! if (*ptr == 1) { … #include <assert.h> void myFunc (int *ptr) { assert (ptr != NULL); if (*ptr == 1) { … • Exits program if ptr is NULL (condition not true) • Better than a comment: • Checked at runtime & gives useful error message > assert failed on line 208 of file myFunc.cpp: ptr != NULL

  35. What If I Need That Last Bit of Speed? #define NDEBUG // Just turned off assertion checking // Make sure this line is in front of // #include <assert.h> #include <assert.h> void myFunc (int *ptr) { ptr = NULL; assert (ptr != NULL); // Not checked  won’t fire. … • Can turn off assertion checking in release build • Avoids any speed overhead • Leave on for debug build for extra checking • And maybe leave on in release too if your program not very time critical

  36. 8. Defensive Coding B. Set deleted / invalid pointers to NULL delete (ptr); ptr = NULL; // Now we’ll crash if we try to use it  good! … ptr->value = 1; // Will seg fault immediately C. Self-checking code (advanced technique) program_ok = validate_key_data_structure (my_struct);

  37. Should have used assert or validity checkers!

  38. Find the Bug! const int NUM_WEIGHTS = 20; // 1. Constant variable #define WEIGHT_ZERO 0 // 2. Pre-processor constant enumWtReturn {HAS_NEG = -1, HAS_ZERO = 0, ALL_POS = 1}; // 3. make an “enumeration” (list) of int constants int checkWeights (int weights[NUM_WEIGHTS]) { for (int i = 0; i < NUM_WEIGHTS; i++) { if (weights[i] = 0) return (HAS_ZERO); if (weights[i] < 0) return (HAS_NEG); } return (ALL_POS); } Test program, find a failing case … • weights = {-1, 2, 3, 4, 5, 6, … 20} • checkWeights returns ALL_POS

  39. 9. Use Compiler Warnings > g++ -Wall weights.cpp > weights.cpp: In function ‘int checkWeights(int*)’: > weights.cpp:11: warning: suggest parentheses around assignment used as truth value int checkWeights (int weights[NUM_WEIGHTS]) { for (int i = 0; i < NUM_WEIGHTS; i++) { if (weights[i] = 0) return (HAS_ZERO); if (weights[i] < 0) return (HAS_NEG); } return (ALL_POS); } Line 11

  40. 9. No Warnings Don’t have any warnings in your code • Warnings flag potentially problematic code • If you leave some warnings in, hard to see new ones • Fix right away: stay at 0 warnings! • Tell compiler to generate all useful warnings (more than default) • Command line: g++ –Wall <…> • We turn on all warnings we consider useful in your makefile Warnings in your code  lower style mark

  41. Summary: One Code Base to Rule Them All • Code will be read more than written • By you • By others • Make it readable! • Code will be modified many times • Make small functions & avoid repeated code • Keep the tests with the code • Automated, so you can re-run often • Keep documentation in the code -- comments!

  42. Code Reviews • If you do one thing for code quality code reviews • Altera/Intel: periodic review • 4 reviewers read code written by one team member • Significant amount: ~400 – 1000 lines • Write down thoughts • Then meet to discuss  readable, clear, efficient? • Google: every commit reviewed and approved • Integrated into source code control system • Change not visible to others until reviewed & approved Read code as a team & share feedback

More Related