Coverage analysis using gcc (or g ++)

1. Coverage analysis using gcc (or g++) myprogram.exe myprogram.gcno myprogram.gcov text report myprogram.cpp myprogram.gcda source code executable run results coverage gcc –ftest-coverage –fprofile-arcs myprogram.exe gcov –c –f –b myprogram • Compile the program with additional options • creates the control flow graph (static analysis) • instruments the code to save statistics on executed statements • Run the program with your desired (test) input • saves the covered statements and other statistics • Analyze the results of the run for coverage • merges the control flow graph with collected statistics • saves results in human-readable form

2. Example Actual code (ispow3.cpp) Control flow graphs (ispow3.gcno) Covered statements (ispow3.gcda) A simple program to compute ifa number is a power of 3

3. gcov – First impression • gcov ispow3 Simplest output: shows, per file • number of lines executed out of total number of executable lines • Example (for ispow3 run with the input ‘100’) • File '/opt/local/include/gcc44/c++/bits/ios_base.h' • Lines executed:0.00% of 2 • /opt/local/include/gcc44/c++/bits/ios_base.h:creating 'ios_base.h.gcov' • File '/opt/local/include/gcc44/c++/ostream' • Lines executed:0.00% of 13 • /opt/local/include/gcc44/c++/ostream:creating 'ostream.gcov' • File '/opt/local/include/gcc44/c++/bits/basic_ios.h' • Lines executed:0.00% of 10 • /opt/local/include/gcc44/c++/bits/basic_ios.h:creating 'basic_ios.h.gcov' • File 'ispow3.cpp' • Lines executed:90.91% of 11 • ispow3.cpp:creating 'ispow3.cpp.gcov' • File '/opt/local/include/gcc44/c++/iostream' • Lines executed:100.00% of 1 • /opt/local/include/gcc44/c++/iostream:creating 'iostream.gcov' • File '/opt/local/include/gcc44/c++/bits/char_traits.h' • Lines executed:0.00% of 2 • /opt/local/include/gcc44/c++/bits/char_traits.h:creating 'char_traits.h.gcov' • File '/opt/local/include/gcc44/c++/bits/locale_facets.h' • Lines executed:0.00% of 5 • /opt/local/include/gcc44/c++/bits/locale_facets.h:creating 'locale_facets.h.gcov’ • Comments • the program covers one source file and 6 headers • for each file, a .gcov detailed report is generated (we’ll examine these next) Read this as: 90.91% of 11 executable lines in ispow3.cpp were covered

4. gcov – Simple per-file report File 'ispow3.cpp' Lines executed:90.91% of 11 ispow3.cpp:creating 'ispow3.cpp.gcov' So which are these ‘11 executable lines’ in ispow3.cpp? • Column 1: line execution status • ‘-’ not an executable line • ‘3’ line execution count • #### line not covered, i.e. not executed Column 2: line nr. in source file Column 3: source code Indeed, this line is not executed for input x == 100 Why is the closing brace marked as ‘executable’??

5. gcov – Detailed per-file report Generate with gcov–cispow3 • We now also see function call reports • function name (mangled, use c++filt to demangle) • number of times the function is called • number of times the function returned • whether all the control-flow-graph blocks in the function were executed • after each line containing branches (e.g. the for loop) • how many times each branch was taken • after each line containing function calls (e.g. << operator) • how many times each call returned Indeed, for the run x==100, we take only branch x>=0 (hence, the else block below is never executed) • There are 2 function calls counted for the • ‘closing brace’ line • these are part of the finalization code (execution of global cleanup operations) • this is the ‘missing’ 11th executable line…

6. Typical gcov usage • build your code base with the gcov options (-ftest-coverage –fprofile-arcs) • run all your testsuite as usual • collect all *.gcov files and gcov text output • compute statistics from this output to • assess the effectiveness of your testuite (how much was covered) • improve the testsuite • see what was not covered (enough) • design new tests to cover those cases • Strong points of gcov • works out of the box for any program compilable with gcc/g++ • no complex set-up or configuration files needed • can embed analysis output easily in any build/test process • you can (in theory) read the *.gcno and *.gcda files to add your own analyses • several other options, not explained here (see man page) • more statistics (per-block, per-function, unconditional branches, …) • it is free and supported on many platforms

7. Weak(er) points of gcov • output needs some postprocessingto be easily read • file-based output is not always easy to read • OK for source files • can be confusing for header files (think e.g. of multiple inclusion) • must be aware of low(er) level details • hidden code (e.g. initialization and destruction) • name mangling • optimization (best to switch it off completely) • control-flow-graph statistics do not always map well to line-oriented statistics • not every line is a graph node (or block of nodes) • problems with • multiple instructions per line • macros • gcov does only • statement coverage • function coverage • function call coverage • branch coverage • gcov does not do • loop coverage, (multiple) condition coverage, path coverage