Fix ctests so each restart test has a seed unique to the first test #1832
Conversation
|
Ran a partial ctest suite to make sure each test run has a unique seed: [dyoungworth@ceta-fedora28 FDBA3]$ ctest -R 'RandomReadWriteTest|SwizzledLargeApiCorrectness|restarting' [dyoungworth@ceta-fedora28 FDBA3]$ grep RandomSeed test_runs/2019_07_11__09_48_41/test_/.json | awk '{print $12} {print $13} {print $25}' |
cmake/AddFdbTest.cmake
Outdated
| @@ -91,6 +91,9 @@ function(add_fdb_test) | |||
| # if the value was undefined before, it will still be | |||
| # undefined. | |||
| math(EXPR assigned_id "${test_idx} - 1") | |||
| math(EXPR test_file_idx "${CURRENT_TEST_FILE_INDEX} + ${NUM_TEST_FILES}") | |||
There was a problem hiding this comment.
the only thing you should change here is the math-expression for test_idx
There was a problem hiding this comment.
Will do. I kept the old test_idx just so when printing tests you get a contiguous count, but that doesn't happen when tests are skipped anyway
cmake/AddFdbTest.cmake
Outdated
| @@ -91,6 +91,9 @@ function(add_fdb_test) | |||
| # if the value was undefined before, it will still be | |||
| # undefined. | |||
| math(EXPR assigned_id "${test_idx} - 1") | |||
| math(EXPR test_file_idx "${CURRENT_TEST_FILE_INDEX} + ${NUM_TEST_FILES}") | |||
| set(CURRENT_TEST_FILE_INDEX "${test_file_idx}" PARENT_SCOPE) | |||
| math(EXPR assigned_test_number "${test_file_idx} - ${NUM_TEST_FILES}") | |||
There was a problem hiding this comment.
Basically same as above: instead of introducing a new variable, just change the math for assigned_id
Makes sure that the ctest infastructure generates a unique seed for every test run, including the first and second tests of each restart test.
Also protect against potential bug where first restart test's logs could be overwritten by the second. (Only happens if logs files do not generate a unique id)