Development/CodeCoverage

Code coverage is a measurement method which can measure how much of your code was executed. It helps test case development: if you think you are ready writing test cases and all of your code is tested, you can check if the test cases really covers all of your code.

Be careful, however, because if you reach 100% code coverage with the test cases and all the test cases passed, it does not mean that your code is perfect. The code coverage cannot detect if you left out some checks.

Code Coverage Measurement of Wireshark

For code coverage measurement you will need GCC. GCC will instrument the your code which means that counters are placed at the important points of the code. The GCC 4.1 and later has the option "--coverage" which supports the instrumentation. (For GCC 3.4 and before the options "-fprofile-arcs -ftest-coverage" has to be used.)

You will also need a recent version of libtool as previous versions do not pass the "--coverage" option to gcc. libtool version 1.5.24 is known to be working while version 1.5.22 is known to be broken.

The steps are the same as described at page Development except you also have to specify the "CFLAGS=--coverage" configure option:

~/wireshark$ ./autogen.sh
~/wireshark$ ./configure CFLAGS=--coverage [options]
~/wireshark$ make

When the build process finished *.gcno files are created additionally to the *.o object files under .libs/ directories. Now you have to start wireshark and run the test cases, i.e. open a test file. After you exit Wireshark, the code coverage results will be saved to several *.gcda files.

~/wireshark$ ./wireshark

Let's assume that we would like to see how much of the epan/dissectors/packet-usb.c was executed. To check this we have to use gcov and specify the object and the source file:

~/wireshark$ cd epan/dissectors
~/wireshark/epan/dissectors$ gcov -o .libs/libcleandissectors_la-packet-usb.o  packet-usb.c
File '/usr/include/glib-2.0/glib/gutils.h'
Lines executed:0.00% of 41
/usr/include/glib-2.0/glib/gutils.h:creating 'gutils.h.gcov'

File '/usr/include/glib-2.0/glib/gthread.h'
Lines executed:0.00% of 4
/usr/include/glib-2.0/glib/gthread.h:creating 'gthread.h.gcov'

File '/usr/include/glib-2.0/glib/gstring.h'
Lines executed:0.00% of 6
/usr/include/glib-2.0/glib/gstring.h:creating 'gstring.h.gcov'

File 'packet-usb.c'
Lines executed:2.43% of 494
packet-usb.c:creating 'packet-usb.c.gcov'

Summary was shown, the detailed result can be found in the *.gcov files. The part will show that all the lines of proto_reg_handoff_usb() function were executed once:

~/wireshark/epan/dissectors$ tail packet-usb.c.gcov 
        -: 1583:
        -: 1584:void
        -: 1585:proto_reg_handoff_usb(void)
        1: 1586:{
        -: 1587:    dissector_handle_t linux_usb_handle;
        -: 1588:
        1: 1589:    linux_usb_handle = create_dissector_handle(dissect_linux_usb, proto_usb);
        -: 1590:
        1: 1591:    dissector_add("wtap_encap", WTAP_ENCAP_USB_LINUX, linux_usb_handle);
        1: 1592:}

Note that it is possible to run wireshark several times: the counters will be added to the exsisting *.gcda files. If you would like to clear the counters, just delete the *.gcda files:

~/wireshark$ for i in $(find -iname '*\.gcda'); do rm $i; done

Generating HTML Report with lcov

When the *.gcda files are ready, the lcov can be used to generate a nice HTML report. (Under a Debian install it with "apt-get install lcov".)

The first step is to convert the *.gcda files to *.info files which lcov can understand. (As of SVN revision 24015 there is a problem that not all source files can be found which are referenced from #line 1 "..." type lines. The workaround is to disable the line checksum generation with "--no-checksum" option.)

~/wireshark$ geninfo --compat-libtool --no-checksum .
Found gcov version: 4.2.3
Scanning . for .gcda files ...
Found 1080 data files in .
Processing ./gtk/gsm_map_stat.gcda
Processing ./gtk/capture_if_dlg.gcda
Processing ./gtk/hostlist_tr.gcda
[...]
Processing ./file.gcda
Processing ./capture-pcap-util-unix.gcda
Processing ./tap-dcerpcstat.gcda
Finished .info-file creation

Now the genhtml can be used to create the HTML output. (As of SVN revision 24015 of Wireshark a little patch is needed for genhtml to skip the nonexsistent source files, see later.)

~/wireshark$ mkdir lcov-output
~/wireshark$ genhtml -o lcov-output/ $(find -iname '*\.gcda\.info')
Reading data file ./gtk/range_utils.gcda.info
Reading data file ./gtk/uat_gui.gcda.info
Reading data file ./gtk/gui_utils.gcda.info
[...]
Reading data file ./tap-stats_tree.gcda.info
Found 1267 entries.
Found common filename prefix "/home/nmarci/src/wireshark/wireshark"
Writing .css and .png files.
Generating output.
Processing file capture-pcap-util.c
Processing file tap-bootpstat.c
Processing file g711.c
[...]
Processing file /usr/include/glib-2.0/glib/gthread.h
Writing directory view page.
Overall coverage rate: 24634 of 394629 lines (6.2%)

Known problems

If your build process end with the following error message, you will have to upgrade your libtool. Version 1.5.24 is known to be working. Be careful: upgrading libtool will not modify ./libtool (which is probably still the old version), you have to run "make distclean" and "./autogen.sh" again after libtool upgrade.

/usr/bin/ld: .libs/wireshark: hidden symbol `__gcov_init' in /usr/lib/gcc/i486-linux-gnu/4.1.3/libgcov.a(_gcov.o) is referenced by DSO
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status
make[2]: *** [wireshark] Error 1
make[2]: Leaving directory `/home/foo/wireshark'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/foo/wireshark'
make: *** [all] Error 2

If there is a missing source file genhtml will die. The workaround is the following patch:

--- /usr/bin/genhtml    2007-09-09 17:14:20.000000000 +0200
+++ genhtml.modified    2008-01-05 13:47:00.000000000 +0100
@@ -3165,6 +3165,15 @@
                %count_data = %{$_[2]};
        }
 
+        if (!-f $source_filename) {
+            write_source_prolog(*HTML_HANDLE);
+            push (@result,
+                     write_source_line(HTML_HANDLE, 0,
+                                       "(missing file)", 0, 0));
+            write_source_epilog(*HTML_HANDLE);
+            return (@result);
+        }
+
        open(SOURCE_HANDLE, "<".$source_filename)
                or die("ERROR: cannot open $source_filename for reading!\n");

Imported from https://wiki.wireshark.org/Development/CodeCoverage on 2020-08-11 23:12:41 UTC