Skip to content

Releases: catchorg/Catch2

v3.3.1

29 Jan 22:24
v3.3.1
2ab20a0
Compare
Choose a tag to compare

Improvements

  • Reduced allocations and improved performance
    • The exact improvements are dependent on your usage of Catch2.
    • For example running Catch2's SelfTest binary performs 8k less allocations.
    • The main improvement comes from smarter handling of SECTIONs, especially sibling SECTIONs

v3.3.0

22 Jan 19:57
v3.3.0
c837cb4
Compare
Choose a tag to compare

Improvements

  • Added MessageMatches exception matcher (#2570)
  • Added RangeEquals and UnorderedRangeEquals generic range matchers (#2377)
  • Added SKIP macro for skipping tests from within the test body (#2360)
    • All built-in reporters have been extended to handle it properly, whether your custom reporter needs changes depends on how it was written
    • skipTest reporter event is unrelated to this, and has been deprecated since it has practically no uses
  • Restored support for PPC Macs in the break-into-debugger functionality (#2619)
  • Made our warning suppression compatible with CUDA toolkit pre 11.5 (#2626)
  • Cleaned out some static analysis complaints

Fixes

  • Fixed macro redefinition warning when NVCC was reporting as MSVC (#2603)
  • Fixed throws in generator constructor causing the whole binary to abort (#2615)
    • Now it just fails the test
  • Fixed missing transitive include with libstdc++13 (#2611)

Miscellaneous

  • Improved support for dynamic library build with non-MSVC compilers on Windows (#2630)
  • When used as a subproject, Catch2 keeps its generated header in a separate directory from the main project (#2604)

v3.2.1

09 Dec 22:18
v3.2.1
ab6c737
Compare
Choose a tag to compare

Improvements

  • Fix the reworked decomposer to work with older (pre 9) GCC versions (#2571)
    • This required more significant changes to properly support C++20, there might be bugs.

v3.2.0

16 Nov 19:24
v3.2.0
65cc7fd
Compare
Choose a tag to compare

3.2.0

Improvements

  • Catch2 now compiles on PlayStation (#2562)
  • Added CATCH_CONFIG_GETENV compile-time toggle (#2562)
    • This toggle guards whether Catch2 calls std::getenv when reading env variables
  • Added support for more Bazel test environment variables
    • TESTBRIDGE_TEST_ONLY is now supported (#2490)
    • Sharding variables, TEST_SHARD_INDEX, TEST_TOTAL_SHARDS, TEST_SHARD_STATUS_FILE, are now all supported (#2491)
  • Bunch of small tweaks and improvements in reporters
    • The TAP and SonarQube reporters output the used test filters
    • The XML reporter now also reports the version of its output format
    • The compact reporter now uses the same summary output as the console reporter (#878, #2554)
  • Added support for asserting on types that can only be compared with literal 0 (#2555)
    • A canonical example is C++20's std::*_ordering types, which cannot be compared with an int variable, only 0
    • The support extends to any type with this property, not just the ones in stdlib
    • This change imposes 2-3% slowdown on compiling files that are heavy on REQUIRE and friends
    • This required significant rewrite of decomposition, there might be bugs
  • Simplified internals of matcher related macros
    • This provides about ~2% speed up compiling files that are heavy on REQUIRE_THAT and friends

Fixes

  • Cleaned out some warnings and static analysis issues
    • Suppressed -Wcomma warning rarely occuring in templated test cases (#2543)
    • Constified implementation details in INFO (#2564)
    • Made MatcherGenericBase copy constructor const (#2566)
  • Fixed serialization of test filters so the output roundtrips
    • This means that e.g. ./tests/SelfTest "aaa bbb", [approx] outputs Filters: "aaa bbb",[approx]

Miscellaneous

  • Catch2's build no longer leaks -ffile-prefix-map setting to dependees (#2533)

v3.1.1

19 Oct 19:23
v3.1.1
5df88da
Compare
Choose a tag to compare

Improvements

  • Added Catch::getSeed function that user code can call to retrieve current rng-seed
  • Better detection of compiler support for -ffile-prefix-map (#2517)
  • Catch2's shared libraries now have SOVERSION set (#2516)
  • catch2/catch_all.hpp convenience header no longer transitively includes windows.h (#2432, #2526)

Fixes

  • Fixed compilation on Universal Windows Platform
  • Fixed compilation on VxWorks (#2515)
  • Fixed compilation on Cygwin (#2540)
  • Remove unused variable in reporter registration (#2538)
  • Fixed some symbol visibility issues with dynamic library on Windows (#2527)
  • Suppressed -Wuseless-cast warnings in REQUIRE_THROWS* macros (#2520, #2521)
    • This was triggered when the potentially throwing expression evaluates to void
  • Fixed "warning: storage class is not first" with nvc++ (#2533)
  • Fixed handling of DL_PATHS argument to catch_discover_tests on MacOS (#2483)
  • Suppressed *-avoid-c-arrays clang-tidy warning in TEMPLATE_TEST_CASE (#2095, #2536)

Miscellaneous

  • Fixed CMake install step for Catch2 build as dynamic library (#2485)
  • Raised minimum CMake version to 3.10 (#2523)
    • Expect the minimum CMake version to increase once more in next few releases.
  • Whole bunch of doc updates and fixes
  • Added support for building Catch2 with Meson (#2530, #2539)

v2.13.10

16 Oct 09:07
v2.13.10
182c910
Compare
Choose a tag to compare

Fixes

  • Fixed issue with catch_discover_tests when there is multiple of 256 tests (#2401, #2503)
  • Catch2-provided main and wmain are explicitly marked as __cdecl when compiled with MSVC (#2486, #2487)
  • Improved break-into-debugger behaviour for ARM Macs. It should now be possible to step execution after the break (#2422)
  • Replaced deprecated std::aligned_storage (#2419, #2420)

v3.1.0

17 Jul 19:24
v3.1.0
97c48e0
Compare
Choose a tag to compare

Improvements

  • Improved suppression of -Wparentheses for older GCCs
    • Turns out that even GCC 9 does not properly handle _Pragmas in the C++ frontend.
  • Added type constraints onto random generator (#2433)
    • These constraints copy what the standard says for the underlying std::uniform_int_distribution
  • Suppressed -Wunused-variable from nvcc (#2306, #2427)
  • Suppressed -Wunused-variable from MinGW (#2132)
  • Added All/Any/NoneTrue range matchers (#2319)
    • These check that all/any/none of boolean values in a range are true.
  • The JUnit reporter now normalizes classnames from C++ namespaces to Java-like namespaces (#2468)
    • This provides better support for other JUnit based tools.
  • The Bazel support now understands BAZEL_TEST environment variable (#2459)
    • The CATCH_CONFIG_BAZEL_SUPPORT configuration option is also still supported.
  • Returned support for compiling Catch2 with GCC 5 (#2448)
    • This required removing inherited constructors from Catch2's internals.
    • I recommend updating to a newer GCC anyway.
  • catch_discover_tests now has a new options for setting library load path(s) when running the Catch2 binary (#2467)

Fixes

  • Fixed crash when listing listeners without any registered listeners (#2442)
  • Fixed nvcc compilation error in constructor benchmarking helper (#2477)
  • Catch2's CMakeList supports pre-3.12 CMake again (#2428)
    • The gain from requiring CMake 3.12 was very minor, but y'all should really update to newer CMake

Miscellaneous

  • Fixed SelfTest build on MinGW (#2447)
  • The in-repo conan recipe exports the CMake helper (#2460)
  • Added experimental CMake script to showcase using test case sharding together with CTest
    • Compared to catch_discover_tests, it supports very limited number of options and customization
  • Added documentation page on best practices when running Catch2 tests
  • Catch2 can be built as a dynamic library (#2397, #2398)
    • Note that Catch2 does not have visibility annotations, and you are responsible for ensuring correct visibility built into the resulting library.

v3.0.1

17 May 21:35
v3.0.1
605a347
Compare
Choose a tag to compare

Catch2 now uses statically compiled library as its distribution model.
This also means that to get all of Catch2's functionality in a test file,
you have to include multiple headers.

You probably want to look into the migration docs,
which were written to help people coming from v2.x.x versions to the
v3 releases.

FAQ

  • Why is Catch2 moving to separate headers?
    • The short answer is future extensibility and scalability. The long answer is complex and can be found on my blog, but at the most basic level, it is that providing single-header distribution is at odds with providing variety of useful features. When Catch2 was distributed in a single header, adding a new Matcher would cause overhead for everyone, but was useful only to a subset of users. This meant that the barrier to entry for new Matchers/Generators/etc is high in single header model, but much smaller in the new model.
  • Will Catch2 again distribute single-header version in the future?
    • No. But we do provide sqlite-style amalgamated distribution option. This means that you can download just 1 .cpp file and 1 header and place them next to your own sources. However, doing this has downsides similar to using the catch_all.hpp header.
  • Why the big breaking change caused by replacing catch.hpp with catch_all.hpp?
    • The convenience header catch_all.hpp exists for two reasons. One of them is to provide a way for quick migration from Catch2, the second one is to provide a simple way to test things with Catch2. Using it for migration has one drawback in that it is big. This means that including it will cause significant compile time drag, and so using it to migrate should be a conscious decision by the user, not something they can just stumble into unknowingly.

(Potentially) Breaking changes

  • Catch2 now uses statically compiled library as its distribution model
    • Including catch.hpp no longer works
  • Catch2 now uses C++14 as the minimum support language version
  • ANON_TEST_CASE has been removed, use TEST_CASE with no arguments instead (#1220)
  • --list* commands no longer have non-zero return code (#1410)
  • --list-test-names-only has been removed (#1190)
    • You should use verbosity-modifiers for --list-tests instead
  • --list* commands are now piped through the reporters
    • The top-level reporter interface provides default implementation that works just as the old one
    • XmlReporter outputs a machine-parseable XML
  • TEST_CASE description support has been removed
    • If the second argument has text outside tags, the text will be ignored.
  • Hidden test cases are no longer included just because they don't match an exclusion tag
    • Previously, a TEST_CASE("A", "[.foo]") would be included by asking for ~[bar].
  • PredicateMatcher is no longer type erased.
    • This means that the type of the provided predicate is part of the PredicateMatcher's type
  • SectionInfo no longer contains section description as a member (#1319)
    • You can still write SECTION("ShortName", "Long and wordy description"), but the description is thrown away
    • The description type now must be a const char* or be implicitly convertible to it
  • The [!hide] tag has been removed.
    • Use [.] or [.foo] instead.
  • Lvalues of composed matchers cannot be composed further
  • Uses of REGISTER_TEST_CASE macro need to be followed by a semicolon
    • This does not change TEST_CASE and friends in any way
  • IStreamingReporter::IsMulti member function was removed
    • This is very unlikely to actually affect anyone, as it was default-implemented in the interface, and only used internally
  • Various classes not designed for user-extension have been made final
    • ListeningReporter is now final
    • Concrete Matchers (e.g. UnorderedEquals vector matcher) are now final
    • All Generators are now final
  • Matcher namespacing has been redone
    • Matcher types are no longer in deeply nested namespaces
    • Matcher factory functions are no longer brought into Catch namespace
    • This means that all public-facing matcher-related functionality is now in Catch::Matchers namespace
  • Defining CATCH_CONFIG_MAIN will no longer create main in that TU.
    • Link with libCatch2Main.a, or the proper CMake/pkg-config target
    • If you want to write custom main, include catch2/catch_session.hpp
  • CATCH_CONFIG_EXTERNAL_INTERFACES has been removed.
    • You should instead include the appropriate headers as needed.
  • CATCH_CONFIG_IMPL has been removed.
    • The implementation is now compiled into a static library.
  • Event Listener interface has changed
    • TestEventListenerBase was renamed to EventListenerBase
    • EventListenerBase now directly derives from IStreamingReporter, instead of deriving from StreamingReporterBase
  • GENERATE decays its arguments (#2012, #2040)
    • This means that str in auto str = GENERATE("aa", "bb", "cc"); is inferred to char const* rather than const char[2].
  • --list-* flags write their output to file specified by the -o flag
  • Many changes to reporter interfaces
    • With the exception of the XmlReporter, the outputs of first party reporters should remain the same
    • New pair of events were added
    • One obsolete event was removed
    • The base class has been renamed
    • The built-in reporter class hierarchy has been redone
  • Catch2 generates a random seed if one hasn't been specified by the user
  • The short flag for --list-tests, -l, has been removed.
    • This is not a commonly used flag and does not need to use up valuable single-letter space.
  • The short flag for --list-tags, -t, has been removed.
    • This is not a commonly used flag and does not need to use up valuable single-letter space.
  • The --colour option has been replaced with --colour-mode option

Improvements

  • Matchers have been extended with the ability to use different signatures of match (#1307, #1553, #1554, #1843)
    • This includes having templated match member function
    • See the rewritten Matchers documentation for details
    • Catch2 currently provides some generic matchers, but there should be more before final release of v3
      • IsEmpty, SizeIs which check that the range has specific properties
      • Contains, which checks whether a range contains a specific element
      • AllMatch, AnyMatch, NoneMatch range matchers, which apply matchers over a range of elements
  • Significant compilation time improvements
    • including catch_test_macros.hpp is 80% cheaper than including catch.hpp
  • Some runtime performance optimizations
    • In all tested cases the v3 branch was faster, so the table below shows the speedup of v3 to v2 at the same task
task debug build release build
Run 1M REQUIRE(true) 1.10 ± 0.01 1.02 ± 0.06
Run 100 tests, 3^3 sections, 1 REQUIRE each 1.27 ± 0.01 1.04 ± 0.01
Run 3k tests, no names, no tags 1.29 ± 0.01 1.05 ± 0.01
Run 3k tests, names, tags 1.49 ± 0.01 1.22 ± 0.01
Run 1 out of 3k tests no names, no tags 1.68 ± 0.02 1.19 ± 0.22
Run 1 out of 3k tests, names, tags 1.79 ± 0.02 2.06 ± 0.23
  • POSIX platforms use gmtime_r, rather than gmtime when constructing a date string (#2008, #2165)
  • --list-* flags write their output to file specified by the -o flag (#2061, #2163)
  • Approx::operator() is now properly const
  • Catch2's internal helper variables no longer use reserved identifiers (#578)
  • --rng-seed now accepts string "random-device" to generate random seed using std::random_device
  • Catch2 now supports test sharding (#2257)
    • You can ask for the tests to be split into N groups and only run one of them.
    • This greatly simplifies parallelization of tests in a binary through external runner.
  • The embedded CLI parser now supports repeatedly callable lambdas
    • A lambda-based option parser can opt into being repeatedly specifiable.
  • Added STATIC_CHECK macro, similar to STATIC_REQUIRE (#2318)
    • When deferred tu runtime, it behaves like CHECK, and not like REQUIRE.
  • You can have multiple tests with the same name, as long as other parts of the test identity differ (#1915, #1999, #2175)
    • Test identity includes test's name, test's tags and and test's class name if applicable.
  • Added new warning, UnmatchedTestSpec, to error on test specs with no matching tests
  • The -w, --warn warning flags can now be provided multiple times to enable multiple warnings
  • The case-insensitive handling of tags is now more reliable and takes up less memory
  • Test case and assertion counting can no longer reasonably overflow on 32 bit systems
    • The count is now kept in uint64_t on all platforms, instead of using size_t type.
  • The -o, --out output destination specifiers recognize - as stdout
    • You have to provide it as --out=- to avoid CLI error about missing option
    • The new reporter specification also recognizes - as stdout
  • Multiple reporters can now run at the same time and write to different files (#1712, #2183)
    • To support this, the -r, --reporter flag now also accepts optional output destination
    • For full overview of the semantics of using multiple reporters, look into the reporter documentation
    • To enable the new syntax, reporter names can no longer contain ::.
  • Console colour support has been rewritten and significantly improved
    • The colour implementation based on ANSI colour codes is always available
    • Colour implementations respect their associated stream
      • previously e.g. Win32 impl wou...
Read more

v3.0.0-preview5

20 Apr 21:59
v3.0.0-preview5
f526ff0
Compare
Choose a tag to compare
v3.0.0-preview5 Pre-release
Pre-release

This is intended to be the last preview before full release. All API-breaking changes should be accounted for.


Catch2 now uses statically compiled library as its distribution model.
This also means that to get all of Catch2's functionality in a test file,
you have to include multiple headers.

You probably want to look into the migration docs,
which were written to help people coming from v2.x.x versions to the
v3 releases.

FAQ

  • Why is Catch2 moving to separate headers?
    • The short answer is future extensibility and scalability. The long answer is complex and can be found on my blog, but at the most basic level, it is that providing single-header distribution is at odds with providing variety of useful features. When Catch2 was distributed in a single header, adding a new Matcher would cause overhead for everyone, but was useful only to a subset of users. This meant that the barrier to entry for new Matchers/Generators/etc is high in single header model, but much smaller in the new model.
  • Will Catch2 again distribute single-header version in the future?
    • No. But we do provide sqlite-style amalgamated distribution option. This means that you can download just 1 .cpp file and 1 header and place them next to your own sources. However, doing this has downsides similar to using the catch_all.hpp header.
  • Why the big breaking change caused by replacing catch.hpp with catch_all.hpp?
    • The convenience header catch_all.hpp exists for two reasons. One of them is to provide a way for quick migration from Catch2, the second one is to provide a simple way to test things with Catch2. Using it for migration has one drawback in that it is big. This means that including it will cause significant compile time drag, and so using it to migrate should be a conscious decision by the user, not something they can just stumble into unknowingly.

(Potentially) Breaking changes

  • Catch2 now uses statically compiled library as its distribution model
    • Including catch.hpp no longer works
  • Catch2 now uses C++14 as the minimum support language version
  • ANON_TEST_CASE has been removed, use TEST_CASE with no arguments instead (#1220)
  • --list* commands no longer have non-zero return code (#1410)
  • --list-test-names-only has been removed (#1190)
    • You should use verbosity-modifiers for --list-tests instead
  • --list* commands are now piped through the reporters
    • The top-level reporter interface provides default implementation that works just as the old one
    • XmlReporter outputs a machine-parseable XML
  • TEST_CASE description support has been removed
    • If the second argument has text outside tags, the text will be ignored.
  • Hidden test cases are no longer included just because they don't match an exclusion tag
    • Previously, a TEST_CASE("A", "[.foo]") would be included by asking for ~[bar].
  • PredicateMatcher is no longer type erased.
    • This means that the type of the provided predicate is part of the PredicateMatcher's type
  • SectionInfo no longer contains section description as a member (#1319)
    • You can still write SECTION("ShortName", "Long and wordy description"), but the description is thrown away
    • The description type now must be a const char* or be implicitly convertible to it
  • The [!hide] tag has been removed.
    • Use [.] or [.foo] instead.
  • Lvalues of composed matchers cannot be composed further
  • Uses of REGISTER_TEST_CASE macro need to be followed by a semicolon
    • This does not change TEST_CASE and friends in any way
  • IStreamingReporter::IsMulti member function was removed
    • This is very unlikely to actually affect anyone, as it was default-implemented in the interface, and only used internally
  • Various classes not designed for user-extension have been made final
    • ListeningReporter is now final
    • Concrete Matchers (e.g. UnorderedEquals vector matcher) are now final
    • All Generators are now final
  • Matcher namespacing has been redone
    • Matcher types are no longer in deeply nested namespaces
    • Matcher factory functions are no longer brought into Catch namespace
    • This means that all public-facing matcher-related functionality is now in Catch::Matchers namespace
  • Defining CATCH_CONFIG_MAIN will no longer create main in that TU.
    • Link with libCatch2Main.a, or the proper CMake/pkg-config target
    • If you want to write custom main, include catch2/catch_session.hpp
  • CATCH_CONFIG_EXTERNAL_INTERFACES has been removed.
    • You should instead include the appropriate headers as needed.
  • CATCH_CONFIG_IMPL has been removed.
    • The implementation is now compiled into a static library.
  • Event Listener interface has changed
    • TestEventListenerBase was renamed to EventListenerBase
    • EventListenerBase now directly derives from IStreamingReporter, instead of deriving from StreamingReporterBase
  • GENERATE decays its arguments (#2012, #2040)
    • This means that str in auto str = GENERATE("aa", "bb", "cc"); is inferred to char const* rather than const char[2].
  • --list-* flags write their output to file specified by the -o flag
  • Many changes to reporter interfaces
    • With the exception of the XmlReporter, the outputs of first party reporters should remain the same
    • New pair of events were added
    • One obsolete event was removed
    • The base class has been renamed
    • The built-in reporter class hierarchy has been redone
  • Catch2 generates a random seed if one hasn't been specified by the user
  • The short flag for --list-tests, -l, has been removed.
    • This is not a commonly used flag and does not need to use up valuable single-letter space.
  • The short flag for --list-tags, -t, has been removed.
    • This is not a commonly used flag and does not need to use up valuable single-letter space.
  • The --colour option has been replaced with --colour-mode option

Improvements

  • Matchers have been extended with the ability to use different signatures of match (#1307, #1553, #1554, #1843)
    • This includes having templated match member function
    • See the rewritten Matchers documentation for details
    • Catch2 currently provides some generic matchers, but there should be more before final release of v3
      • IsEmpty, SizeIs which check that the range has specific properties
      • Contains, which checks whether a range contains a specific element
      • AllMatch, AnyMatch, NoneMatch range matchers, which apply matchers over a range of elements
  • Significant compilation time improvements
    • including catch_test_macros.hpp is 80% cheaper than including catch.hpp
  • Some runtime performance optimizations
    • In all tested cases the v3 branch was faster, so the table below shows the speedup of v3 to v2 at the same task
task debug build release build
Run 1M REQUIRE(true) 1.10 ± 0.01 1.02 ± 0.06
Run 100 tests, 3^3 sections, 1 REQUIRE each 1.27 ± 0.01 1.04 ± 0.01
Run 3k tests, no names, no tags 1.29 ± 0.01 1.05 ± 0.01
Run 3k tests, names, tags 1.49 ± 0.01 1.22 ± 0.01
Run 1 out of 3k tests no names, no tags 1.68 ± 0.02 1.19 ± 0.22
Run 1 out of 3k tests, names, tags 1.79 ± 0.02 2.06 ± 0.23
  • POSIX platforms use gmtime_r, rather than gmtime when constructing a date string (#2008, #2165)
  • --list-* flags write their output to file specified by the -o flag (#2061, #2163)
  • Approx::operator() is now properly const
  • Catch2's internal helper variables no longer use reserved identifiers (#578)
  • --rng-seed now accepts string "random-device" to generate random seed using std::random_device
  • Catch2 now supports test sharding (#2257)
    • You can ask for the tests to be split into N groups and only run one of them.
    • This greatly simplifies parallelization of tests in a binary through external runner.
  • The embedded CLI parser now supports repeatedly callable lambdas
    • A lambda-based option parser can opt into being repeatedly specifiable.
  • Added STATIC_CHECK macro, similar to STATIC_REQUIRE (#2318)
    • When deferred tu runtime, it behaves like CHECK, and not like REQUIRE.
  • You can have multiple tests with the same name, as long as other parts of the test identity differ (#1915, #1999, #2175)
    • Test identity includes test's name, test's tags and and test's class name if applicable.
  • Added new warning, UnmatchedTestSpec, to error on test specs with no matching tests
  • The -w, --warn warning flags can now be provided multiple times to enable multiple warnings
  • The case-insensitive handling of tags is now more reliable and takes up less memory
  • Test case and assertion counting can no longer reasonably overflow on 32 bit systems
    • The count is now kept in uint64_t on all platforms, instead of using size_t type.
  • The -o, --out output destination specifiers recognize - as stdout
    • You have to provide it as --out=- to avoid CLI error about missing option
    • The new reporter specification also recognizes - as stdout
  • Multiple reporters can now run at the same time and write to different files (#1712, #2183)
    • To support this, the -r, --reporter flag now also accepts optional output destination
    • For full overview of the semantics of using multiple reporters, look into the reporter documentation
    • To enable the new syntax, reporter names can no longer contain ::.
  • Console colour support has been rewritten and significantly improved
    • The colour implementation ...
Read more

v2.13.9

12 Apr 20:40
v2.13.9
62fd660
Compare
Choose a tag to compare

Fixes

  • Fixed issue with -# (filename-as-tag) flag when __FILE__ expands into filename without directories (#2328, #2393)
  • Fixed CAPTURE macro not being variadic when disabled through CATCH_CONFIG_DISABLE (#2316, #2378)