QMakeTestRunner assists in setting library paths when running unit tests in products built with QMake. It makes make test work out of the box right after makefiles have been generated. It is Free Software, published on Github, and small and easy to integrate into projects as a Git submodule.
QMake as a build system is simple and complete for building code. A repeated problem we run into when using QMake is that it does not implement features for executing parts of the project (like tests). Others ran into this issue as well, as for example this discussion of specifying run configurations from QMake on Qt DevNet indicates. Multi-part projects often consist of dynamic libraries that contain the features, and applications and tests that link this library. Amongst other things, this approach allows tests to link objects which contain the application's code. Now when the tests are executed, the dynamic linker needs to be able to pick up the freshly built library and link the test with it at execution time. Projects usually run test cases with make test, and ideally the tests should execute out of the box after the sources have been configured and built (out of the box meaning that no tweaking of the environment variables should be needed between make and make test). After all, it should be as easy as possible for developers to execute tests. To solve this problem, paths that are created in the build directory need to be added to the dynamic linker library path before the tests can be executed. Especially with shadow builds, this would require manual setup, and make test would not work out of the box after QMake has been run.
This is a problem in IDEs like Qt Creator, but also others - for every build configuration (having separate shadow build directories), the developer needs to configure the library path before being able to debug the application. This second problem needs to be fixed in the IDEs. For the first one - configuring test runs from the QMake .pro file - we have developed QMakeTestRunner.
As soon as any target in a QMake project has the testcase attribute assigned to CONFIG, QMake automatically generates a target called check that runs this test. Unfortunately, make check does not allow the manipulation of the linker paths before running the tests either. QMakeTestRunner contains boilerplate code that wraps QMake's make check target with a new one called make test, and through variables specified in the QMake .pro file, allows to specify paths to be added to the linker path.
QMakeTestRunner is intended to be used as a git submodule to the main project. It should not require any modifications to be used. It requires Python to be in the path, checks for it, and the QMake run will fail if Python cannot be detected. When using other version control systems than Git, QMakeTestRunner should be small enough to simply be copied into the project as a subdirectory.
The following example adds the directory libs/ in the project build directory to the linker path. It assumes the QMakeTestRunner repository is located under 3rdparty/QMakeTestRunner. The path where the dynamic library is generated in is located below the output (build) directory, which is why it is prefixed with OUT_PWD:
TEST_LIB_PATHS += $$OUT_PWD/libs
include( 3rdparty/QMakeTestRunner/testtarget.pri )
To trigger extra diagnostic output of the test runner, add
TEST_VERBOSE = 1before including testtarget.pri.
Once testtarget.pri has been included, a test target is defined in the makefiles. Simply run make test to execute the original make check with the necessary paths.
Frameworks on OSX are a different kind of library. The framework path is passed in to the dynamic linker using a different environment variable. To specify a framework path, set the TEST_FRAMEWORK_PATHS variable before including testtarget.pri.
TEST_FRAMEWORK_PATHS += $$OUT_PWD/frameworksAdditional variables can be exported to the test environment. The following example adds variable FOO with value bar to the unit test environement.
TEST_CUSTOM_ENV += VARIABLE=valueThe following example shows how to export useful paths to reference files under the source and the build directory.
TEST_CUSTOM_ENV += \\\"top_srcdir="$$top_srcdir"\\\"
TEST_CUSTOM_ENV += \\\"top_builddir="$$top_builddir"\\\"
Variables top_srcdir and top_builddir should be defined before being used. For more information, refer to QMake top level srcdir and builddir.
###Setting up QMakeTestRunner as a Git submodule
Using a Git submodule has the benefit that the code for the runner does not need to be duplicated into the repository of the project that is supposed to use it. It also makes it easy to pull updates to the test runner by simply updating the submodule to a new revision. To add the submodule as 3rdparty/QMakeTestRunner in your project, follow these steps:
> mkdir -p 3rdparty
> git submodule add git://github.com/AgileWorkersSoftware/QMakeTestRunner.git 3rdparty/QMakeTestRunner
> git submodule initThe directory 3rdparty/QMakeTestRunner should now contain the current revision of the test runner scripts.
QMakeTestRunner is Free Software licensed under the GPL, version 3. Contributions to it are welcome, please propose them as Github pull requests. To submit a bug report or feature wish, please use the Github issue tracker for the project.
Commercial support for QMaketestRunner or general issues with QMake is provided by Agile Workers Software, we are happy to help.