kcov can't get correct coverage and write files in tests

[AmarOk1412]

Hei!

I'd just tried to use kcov to do my coverage. The structure of my project is:

-> src/ # contains all .rs useful for my soft
-> tests/ # contains all .rs dedicated to unit-tests

I installed kcov with:

wget https://github.com/SimonKagstrom/kcov/archive/master.tar.gz &&
  tar xzf master.tar.gz &&
  cd kcov-master &&
  mkdir build &&
  cd build &&
  cmake .. &&
  make &&
  make install DESTDIR=../../kcov-build &&
  cd ../.. &&
  rm -rf kcov-master

My tests are executed via RUST_TEST_THREADS=1 RUST_BACKTRACE=1 cargo test -- --nocapture

After that, I run kcov with
for file in target/debug/test_*[^\.d]; do mkdir -p "target/cov/$(basename $file)"; ./kcov-build/usr/local/bin/kcov --exclude-pattern=/.cargo,/usr/lib --verify --coveralls-id=$TRAVIS_JOB_ID "target/cov/$(basename $file)" "$file"; done. And I see that I got 0% in coverage.

So I tried to launch this on my computer:

  test-coverage  ./kcov-build/usr/local/bin/kcov --exclude-pattern=/.cargo, --verify target/cov/ target/debug/test_api-3de2d4096dcca822
kcov: 213 invalid breakpoints skipped in /home/AmarOk/Projets/rori/target/debug/test_api-3de2d4096dcca822

running 1 test
test tests_api::api_get_name ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

(test-api is one of my file). The test succeed. So I opened: firefox target/cov/index.html:

Command: | test_api-3de2d4096dcca822
-- | --
Date: | 2018-04-10 12:56:30 |   | Instrumented lines: | 0
Code covered: | NaN% |   | Executed lines: | 0

I think I missed something but I don't know what.

Another thing (But I'll dig later). Another test is writing a sqlite database on the system. When I run this test with kcov, I've got errors which looks like that a user don't have the permission to manipulate files.

./kcov-build/usr/local/bin/kcov --exclude-pattern=/.cargo, --verify target/cov/ target/debug/test_server-e02ddec4199eb2a5
kcov: 213 invalid breakpoints skipped in /home/AmarOk/Projets/rori/target/debug/test_server-e02ddec4199eb2a5

running 11 tests
test tests_server::server_register_other_device ... FAILED
test tests_server::server_load_devices ... FAILED
test tests_server::server_register ... FAILED
test tests_server::server_register_device ... FAILED
test tests_server::server_link_device ... FAILED
test tests_server::server_get_ring_id ... FAILED
test tests_server::server_add_new_anonymous_device ... FAILED
test tests_server::server_remove_device ... FAILED
test tests_server::server_add_unknown_from_interaction ... FAILED
test tests_server::server_remove_other_device ... FAILED
test tests_server::server_unregister ... FAILED

failures:

---- tests_server::server_register_other_device stdout ----
	thread 'tests_server::server_register_other_device' panicked at 'called `Result::unwrap()` on an `Err` value: SqliteFailure(Error { code: ReadOnly, extended_code: 1032 }, Some("attempt to write a readonly database"))', /checkout/src/libcore/result.rs:916:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Have a nice day!

[SimonKagstrom]

Sounds like two separate issues. Anyway, can you check that your binary is built with debug info? If you use readelf -S <prg> you should see some .debug_str etc sections.

Also, you can run kcov with debugging, kcov --debug=15 <other args> to see if it appears to find the source files as it should.

If the sources are not where kcov (and the debug info!) expects them to be, there is the --replace-src-path=path argument as well.

[AmarOk1412]

for the first question:

readelf -S target/debug/test_api-57d82527abd0a87b
 [36] .debug_str        PROGBITS         0000000000000000  0240e828
       0000000000b04265  0000000000000001  MS       0     0     1

So yes.

This is kcov --debug=15 output: https://opportunis.me/paste/?21376ff20cece565#UOZsGgPSGxl17+y1ED3Lfz+Vmx9KaAKw/xrQvCu/4wo=

Don't know if kcov found source files. But I don't think all PT error for 18653: -1 are ok.

Also tried to force the src-path:

./kcov-build/usr/local/bin/kcov --replace-src-path src --exclude-pattern=/.cargo, --verify target/cov/ target/debug/test_api-57d82527abd0a87b 
kcov: error: src is formatted incorrectly

[SimonKagstrom]

The source-path fix should look something like --replace-src-path=src:/path/to/source/where/it/is. It has debug info, so that's good.

Something is wrong with the source lookup though, but I can't quite see what it is

[AmarOk1412]

Ok I'll try. If you want more details, the repo is here https://github.com/AmarOk1412/rori_core

[SimonKagstrom]

I've now built your project, but can't reproduce the problem.

I built with make tests and then ran kcov on one of the tests:

kcov --include-pattern=rori_core --verify /tmp/kcov target/debug/test_database-e4a7fd49bff8e6c5

which does get me coverage like this (I've excluded system code):

Filename | Coverage percent | Covered lines | Uncovered lines | Executable lines
/tmp/rori_core/src/rori/database.rs | 13.8% | 11 | 69 | 80
/tmp/rori_core/tests/test_database.rs | 14.1% | 22 | 134 | 156

I don't know rust, so this on a "fresh" system where I just installed rust via the shell script installer.

[AmarOk1412]

Oh! It works if I add --include-pattern=rori_core indeed. I missed that point.

Thanks for all!

[SimonKagstrom]

Sorry this took so long, I should have suspected filtering to start with. I can't really say why it didn't work for starters, --exclude-pattern=.cargo should only remove your source code if it happens to be placed in a path which contains .cargo.

Anyway, I typically run kcov without options first, i.e.,

kcov /tmp/kcov target/debug/test_database-e4a7fd49bff8e6c5

and then see what filtering needs to be done. It's often quite obvious which parts are interesting and not.

[AmarOk1412]

Np. Everybody has something else to do :)