Occasionally failing tests #191

msimberg · 2022-04-12T13:18:24Z

This issue is meant to collect the names of failing tests together with links to the failures for investigation. This is a long-running issue and will likely not be closed by one or only a few PRs.

condition_variable2

https://app.circleci.com/pipelines/github/pika-org/pika/6661/workflows/def23137-e33b-4958-a8c1-d67ac9b63934/jobs/60858

648/755 Test #640: tests.unit.modules.threading.condition_variable2 ........................................................***Failed   14.95 sec
/home/circleci/project/pika/libs/pika/threading/tests/unit/condition_variable2.cpp(344): test '!st.stop_requested()' failed in function 'auto test_timed_cv(bool, bool, std::chrono::duration<long, std::ratio<1, 1000> >)::(anonymous class)::operator()(pika::stop_token) const'
/home/circleci/project/pika/libs/pika/threading/tests/unit/condition_variable2.cpp(344): test '!st.stop_requested()' failed in function 'auto test_timed_cv(bool, bool, std::chrono::duration<long, std::ratio<1, 1000> >)::(anonymous class)::operator()(pika::stop_token) const'
/home/circleci/project/pika/libs/pika/threading/tests/unit/condition_variable2.cpp(344): test '!st.stop_requested()' failed in function 'auto test_timed_cv(bool, bool, std::chrono::duration<long, std::ratio<1, 1000> >)::(anonymous class)::operator()(pika::stop_token) const'
Tests failed. 0/0 sanity checks and 3/123 tests failed.

disabled in arm64 CI in Replace bors with GitHub merge queues #741

condition_variable3
- https://cdash.cscs.ch/test/63767909
jthread2
- https://cdash.cscs.ch/test/35872109
shared_mutex1:
- https://cdash.cscs.ch/test/69190832
std_thread_scheduler (Attempt to wait for detached thread to finish in std_thread_scheduler_test #250)
- https://cdash.cscs.ch/test/12732612
- https://app.circleci.com/pipelines/github/pika-org/pika/1889/workflows/63918cb6-267a-436a-9eb1-a7718ff70f00/jobs/18999/steps?invite=true#step-103-500
stop_token_race
thread_pool_scheduler (timeout)
- https://app.circleci.com/pipelines/github/pika-org/pika/3923/workflows/18bff009-a572-4402-a343-02917e21f379/jobs/37313
tss
- https://app.circleci.com/pipelines/github/pika-org/pika/1245/workflows/08a5ce76-c3dc-4fb2-b557-aa1ce3a96b40/jobs/13394
ensure_started
- See Fix potential use-after-free in when_all_vector #966 (comment).
thread_rescheduling
- timeout on CCE release build on Jenkins
split_continuation_clear
- Apparently only with GCC 14 + stdexec configuration? Probably Data race when split sender is cancelled NVIDIA/stdexec#1426
- https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/479009878135925/5304355110917878/-/jobs/7723123972#L165
- https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/479009878135925/5304355110917878/-/jobs/7680430741#L134

The text was updated successfully, but these errors were encountered:

msimberg · 2022-06-03T15:07:10Z

I will attempt to look at the std_thread_scheduler failure for 0.6.0.

250: Attempt to wait for detached thread to finish in `std_thread_scheduler_test` r=msimberg a=msimberg I think this fixes part of #191 (the `std_thread_scheduler_test` failures). The sender is still discarded immediately so that part is still somewhat tested, but we attempt to wait for the detached `std::thread` to avoid having it possibly start after exiting main. Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>

395: Re-enable release builds on Jenkins r=msimberg a=msimberg Half-depends on #67 (i.e. should go in afterwards). Fixes #382. This may end up increasing the failure rate of #191 to something that is no longer sustainable. With this we also have to be a lot more mindful not to spam `bors try/merge` since the used node hours will at least double. Draft because of the above reasons. Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>

579: Disable `std_thread_scheduler_test` on macOS CI r=msimberg a=msimberg It fails frequently and is listed on #191. Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>

579: Disable `std_thread_scheduler_test` on macOS CI and reduce parallelism on arm64 CircleCI configuration r=msimberg a=msimberg `std_thread_scheduler_test` fails frequently and is listed on #191. I'm attempting to reduce parallelism for tests on the `arm64` configuration to see if it reduces timeouts on the `condition_variable2` test. One of the timed tests fails, which may indicate oversubscription (since we haven't seen it fail on other configurations). Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>

579: Disable `std_thread_scheduler_test` on macOS CI and reduce parallelism on arm64 CircleCI configuration r=msimberg a=msimberg `std_thread_scheduler_test` fails frequently and is listed on #191. I'm attempting to reduce parallelism for tests on the `arm64` configuration to see if it reduces timeouts on the `condition_variable2` test. One of the timed tests fails, which may indicate oversubscription (since we haven't seen it fail on other configurations). 580: Update atomics feature test to correctly use fallback test with libatomic r=msimberg a=msimberg Attempts to fix #576. The regular atomics test already had a fallback to use `libatomic`, but because the `REQUIRED` flag was set the default test fails immediately and never lets the fallback test run. This adds another flag `NOT_REQUIRED` which overrides `REQUIRED` and is used in the default atomics tests (the fallback test still uses plain `REQUIRED` if requested by the caller). Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>

579: Disable `std_thread_scheduler_test` on macOS CI and reduce parallelism on arm64 CircleCI configuration r=msimberg a=msimberg `std_thread_scheduler_test` fails frequently and is listed on #191. I'm attempting to reduce parallelism for tests on the `arm64` configuration to see if it reduces timeouts on the `condition_variable2` test. One of the timed tests fails, which may indicate oversubscription (since we haven't seen it fail on other configurations). Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>

msimberg · 2023-03-27T08:26:53Z

These are some of the more frequently failing tests on CDash: https://cdash.cscs.ch/testOverview.php?project=pika&date=2023-01-31&group=28.

aurianer · 2023-11-03T21:12:27Z

ARCHIVE (bugs solved):

any test with APEX enabled:
- https://cdash.cscs.ch/viewTest.php?onlyfailed&buildid=9673
- This seems to be due to PAPI. Disabling it removed all typically occuring failures.
barrier_cpp20
- https://cdash.cscs.ch/test/32940023
- fixed in Report test errors on global test fixture destruction instead of requiring an explicit call to report_errors #387
cublas_matmul (on ault HIP CI)
- https://cdash.cscs.ch/test/31890299
executor_parameters_timer_hooks (remove functionality?)
- https://cdash.cscs.ch/test/11297026
- Fixed by Reduce chances of false positives in executor_parameters_timer_hooks #426
future (deleted in Delete legacy future and executor functionality #774, Unrevert #774 #832)
- https://github.com/pika-org/pika/runs/8070517198?check_suite_focus=true#step:6:203
jthread1
nth_element_range:
- https://cdash.cscs.ch/test/28784763
- Fixed by Fix nth_element implementation and test #412
partial_sort_range
- https://cdash.cscs.ch/test/39905086
- Not explicitly fixed, but algorithms moved to separate repository (Remove algorithms module #505)
partition
- https://cdash.cscs.ch/test/40419799
- Not explicitly fixed, but algorithms moved to separate repository (Remove algorithms module #505)
shared_mutex2:

countif:

17/632 Test  #17: tests.unit.modules.algorithms.algorithms.countif .............................................***Failed    0.32 sec
using seed: 1650445799
/Users/runner/work/pika/pika/libs/pika/algorithms/tests/unit/algorithms/countif_tests.hpp(80): test 'num_items == 50u' failed in function 'void test_count_if(ExPolicy &&, IteratorTag) [ExPolicy = const pika::execution::parallel_unsequenced_policy &, IteratorTag = std::forward_iterator_tag]': '23507' != '50'
0 sanity checks and 1 test failed.

using seed: 1660666788
/dev/shm/pika/src/libs/pika/algorithms/tests/unit/algorithms/countif_tests.hpp(60): test 'num_items == 50u' failed in function 'void test_count_if(IteratorTag) [with IteratorTag = std::random_access_iterator_tag]': '45014' != '50'

https://cdash.cscs.ch/test/43664415
Not explicitly fixed, but algorithms moved to separate repository (Remove algorithms module #505)

unique
- https://cdash.cscs.ch/test/37089833
- stacktrace ends in cray_sset_HSW; Cray bug?
- Not explicitly fixed, but algorithms moved to separate repository (Remove algorithms module #505)
split_continuation_clear

msimberg added this to pika planning board Apr 12, 2022

msimberg moved this to Backlog in pika planning board Apr 12, 2022

This was referenced Apr 26, 2022

Fix lifetime bug in split #203

Merged

Add std_thread_scheduler #200

Merged

msimberg added effort: 3 A few days of work. effort: 4 A few weeks of work. priority: medium This should be done sooner or later. type: bug labels May 4, 2022

msimberg added this to the 0.6.0 milestone Jun 3, 2022

msimberg moved this from Backlog to Todo in pika planning board Jun 3, 2022

msimberg self-assigned this Jun 3, 2022

msimberg mentioned this issue Jun 8, 2022

Attempt to wait for detached thread to finish in std_thread_scheduler_test #250

Merged

msimberg removed this from the 0.6.0 milestone Jun 23, 2022

msimberg added this to the 0.7.0 milestone Jul 7, 2022

This was referenced Jul 13, 2022

Preliminary support for using the P2300 reference implementation with CUDA #315

Merged

Many tests timeout on FreeBSD #322

Closed

msimberg removed this from the 0.7.0 milestone Jul 19, 2022

This was referenced Aug 22, 2022

Re-enable release builds on Jenkins #395

Merged

Report test errors on global test fixture destruction instead of requiring an explicit call to report_errors #387

Merged

Fix shared_mutex2 test #416

Merged

msimberg added this to the 0.11.0 milestone Nov 3, 2022

msimberg modified the milestones: 0.11.0, 0.12.0 Dec 5, 2022

msimberg assigned aurianer Dec 20, 2022

msimberg mentioned this issue Jan 30, 2023

Disable std_thread_scheduler_test on macOS CI and reduce parallelism on arm64 CircleCI configuration #579

Merged

bors bot added a commit that referenced this issue Jan 30, 2023

Merge #579

8abac66

579: Disable `std_thread_scheduler_test` on macOS CI r=msimberg a=msimberg It fails frequently and is listed on #191. Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>

msimberg modified the milestones: 0.12.0, 0.13.0 Jan 31, 2023

aurianer modified the milestones: 0.13.0, 0.14.0 Mar 8, 2023

msimberg modified the milestones: 0.14.0, 0.15.0 Apr 3, 2023

msimberg modified the milestones: 0.15.0, 0.16.0 May 2, 2023

msimberg modified the milestones: 0.16.0, 0.17.0 May 30, 2023

msimberg removed this from the 0.17.0 milestone Jun 14, 2023

msimberg mentioned this issue Aug 16, 2023

Replace bors with GitHub merge queues #741

Merged

msimberg mentioned this issue Oct 27, 2023

Fix the segfault happening on CSCS CI for tests.unit.modules.executors.std_thread_scheduler #777

Closed

msimberg added this to the 0.21.0 milestone Nov 1, 2023

msimberg modified the milestones: 0.21.0, 0.22.0 Dec 5, 2023

This was referenced Jan 16, 2024

Add gcc10 apex pipeline in CSCS CI #930

Merged

Bump CI image tags #987

Merged

msimberg modified the milestones: 0.22.0, 0.23.0 Jan 22, 2024

msimberg mentioned this issue Jan 24, 2024

Change notification of waiting threads in condition_variable #998

Merged

msimberg removed this from the 0.23.0 milestone Mar 4, 2024

msimberg mentioned this issue Sep 6, 2024

Update stdexec integration with transfer renames and ensure_started/start_detached from pika #1235

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Occasionally failing tests #191

Occasionally failing tests #191

msimberg commented Apr 12, 2022 •

edited

Loading

msimberg commented Jun 3, 2022

msimberg commented Mar 27, 2023

aurianer commented Nov 3, 2023 •

edited by msimberg

Loading

Occasionally failing tests #191

Occasionally failing tests #191

Comments

msimberg commented Apr 12, 2022 • edited Loading

msimberg commented Jun 3, 2022

msimberg commented Mar 27, 2023

aurianer commented Nov 3, 2023 • edited by msimberg Loading

msimberg commented Apr 12, 2022 •

edited

Loading

aurianer commented Nov 3, 2023 •

edited by msimberg

Loading