Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

M1 flaky test - taichi_sparse_test.py #6546

Closed
ailzhang opened this issue Nov 8, 2022 · 0 comments · Fixed by #6550
Closed

M1 flaky test - taichi_sparse_test.py #6546

ailzhang opened this issue Nov 8, 2022 · 0 comments · Fixed by #6550
Assignees

Comments

@ailzhang
Copy link
Contributor

ailzhang commented Nov 8, 2022

Url:
https://github.com/taichi-dev/taichi/actions/runs/3417707889/jobs/5689158013

This happens when running

/Users/github/miniforge3/envs/3.8/bin/python3', '/Users/github/actions-runner/_work/taichi/taichi/tests/cpp/aot/python_scripts/taichi_sparse_test.py'

Example log just in case the link expires:

2022-11-08T09:12:41.7754570Z [Taichi] version 1.3.0, llvm 10.0.0, commit c841716b, osx, python 3.8.10
2022-11-08T09:12:42.0999600Z [W 11/08/22 17:12:42.099 92605387] [misc.py:adaptive_arch_select@755] Arch=[<Arch.cuda: 5>] is not supported, falling back to CPU
2022-11-08T09:12:42.0999840Z [Taichi] Starting on arch=arm64
2022-11-08T09:12:42.5041240Z Running main() from /Users/github/actions-runner/_work/taichi/taichi/external/googletest/googletest/src/gtest_main.cc
2022-11-08T09:12:42.5041580Z Note: Google Test filter = CapiTest.CometTestCuda
2022-11-08T09:12:42.5041760Z [==========] Running 1 test from 1 test suite.
2022-11-08T09:12:42.5041990Z [----------] Global test environment set-up.
2022-11-08T09:12:42.5042180Z [----------] 1 test from CapiTest
2022-11-08T09:12:42.5042320Z [ RUN      ] CapiTest.CometTestCuda
2022-11-08T09:12:42.5042490Z [       OK ] CapiTest.CometTestCuda (0 ms)
2022-11-08T09:12:42.5042680Z [----------] 1 test from CapiTest (1 ms total)
2022-11-08T09:12:42.5042780Z 
2022-11-08T09:12:42.5042890Z [----------] Global test environment tear-down
2022-11-08T09:12:42.5043060Z [==========] 1 test from 1 test suite ran. (1 ms total)
2022-11-08T09:12:42.5049630Z [  PASSED  ] 1 test.
2022-11-08T09:12:42.5901400Z [Taichi] version 1.3.0, llvm 10.0.0, commit c841716b, osx, python 3.8.10
2022-11-08T09:12:42.9204800Z [W 11/08/22 17:12:42.920 92605496] [misc.py:adaptive_arch_select@755] Arch=[<Arch.cuda: 5>] is not supported, falling back to CPU
2022-11-08T09:12:42.9205090Z [Taichi] Starting on arch=arm64
2022-11-08T09:12:43.0757960Z [E 11/08/22 17:12:43.075 92605588] Received signal 11 (Segmentation fault: 11)
2022-11-08T09:12:43.0758330Z 
2022-11-08T09:12:43.0758350Z 
2022-11-08T09:12:43.0819190Z 
2022-11-08T09:12:43.0820330Z                             * Taichi Core - Stack Traceback *                             
2022-11-08T09:12:43.0821260Z ==========================================================================================
2022-11-08T09:12:43.0821910Z |                       Module |  Offset | Function                                      |
2022-11-08T09:12:43.0822290Z |----------------------------------------------------------------------------------------|
2022-11-08T09:12:43.0830370Z * taichi_python.cpython-38-darwin.so |     120 | taichi::Logger::error(std::__1::basic_s |
2022-11-08T09:12:43.0830600Z                                          | tring<char, std::__1::char_traits<char>, std: |
2022-11-08T09:12:43.0830820Z                                          | :__1::allocator<char> > const&, bool)         |
2022-11-08T09:12:43.0831110Z * taichi_python.cpython-38-darwin.so |     228 | taichi::(anonymous namespace)::signal_h |
2022-11-08T09:12:43.0831530Z                                          | andler(int)                                   |
2022-11-08T09:12:43.0831710Z *     libsystem_platform.dylib |      56 | (null)                                        |
2022-11-08T09:12:43.0832000Z * taichi_python.cpython-38-darwin.so |      48 | taichi::lang::Program::this_thread_conf |
2022-11-08T09:12:43.0832190Z                                          | ig()                                          |
2022-11-08T09:12:43.0832520Z * taichi_python.cpython-38-darwin.so |      48 | taichi::lang::Program::this_thread_conf |
2022-11-08T09:12:43.0832720Z                                          | ig()                                          |
2022-11-08T09:12:43.0832980Z * taichi_python.cpython-38-darwin.so |     812 | taichi::lang::(anonymous namespace)::Ta |
2022-11-08T09:12:43.0833200Z                                          | skCodeGenCPU::visit(taichi::lang::OffloadedSt |
2022-11-08T09:12:43.0833370Z                                          | mt*)                                          |
2022-11-08T09:12:43.0834720Z * taichi_python.cpython-38-darwin.so |     116 | taichi::lang::TaskCodeGenLLVM::emit_to_ |
2022-11-08T09:12:43.0835650Z                                          | module()                                      |
2022-11-08T09:12:43.0835950Z * taichi_python.cpython-38-darwin.so |      96 | taichi::lang::TaskCodeGenLLVM::run_comp |
2022-11-08T09:12:43.0836150Z                                          | ilation()                                     |
2022-11-08T09:12:43.0837320Z * taichi_python.cpython-38-darwin.so |      44 | taichi::lang::KernelCodeGenCPU::compile |
2022-11-08T09:12:43.0837530Z                                          | _task(std::__1::unique_ptr<llvm::Module, std: |
2022-11-08T09:12:43.0837720Z                                          | :__1::default_delete<llvm::Module> >&&, taich |
2022-11-08T09:12:43.0839180Z                                          | i::lang::OffloadedStmt*)                      |
2022-11-08T09:12:43.0839750Z * taichi_python.cpython-38-darwin.so |     140 | taichi::lang::KernelCodeGen::compile_ke |
2022-11-08T09:12:43.0840870Z                                          | rnel_to_module()::$_0::operator()() const     |
2022-11-08T09:12:43.0841430Z * taichi_python.cpython-38-darwin.so |    1024 | taichi::lang::ParallelExecutor::worker_ |
2022-11-08T09:12:43.0842720Z                                          | loop()                                        |
2022-11-08T09:12:43.0843130Z * taichi_python.cpython-38-darwin.so |      44 | void* std::__1::__thread_proxy<std::__1 |
2022-11-08T09:12:43.0844350Z                                          | ::tuple<std::__1::unique_ptr<std::__1::__thre |
2022-11-08T09:12:43.0844560Z                                          | ad_struct, std::__1::default_delete<std::__1: |
2022-11-08T09:12:43.0844750Z                                          | :__thread_struct> >, taichi::lang::ParallelEx |
2022-11-08T09:12:43.0845850Z                                          | ecutor::ParallelExecutor(std::__1::basic_stri |
2022-11-08T09:12:43.0846500Z                                          | ng<char, std::__1::char_traits<char>, std::__ |
2022-11-08T09:12:43.0847610Z                                          | 1::allocator<char> > const&, int)::$_0> >(voi |
2022-11-08T09:12:43.0847800Z                                          | d*)                                           |
2022-11-08T09:12:43.0849020Z *      libsystem_pthread.dylib |     320 | (null)                                        |
2022-11-08T09:12:43.0850010Z *      libsystem_pthread.dylib |       8 | (null)                                        |
2022-11-08T09:12:43.0850560Z ==========================================================================================
2022-11-08T09:12:43.0850650Z 
2022-11-08T09:12:43.0850730Z �[38;2;255;165;000m
2022-11-08T09:12:43.0850900Z Internal error occurred. Check out this page for possible solutions:
2022-11-08T09:12:43.0851140Z https://docs.taichi-lang.org/docs/install
2022-11-08T09:12:43.0891920Z [*** LOG ERROR #0001 ***] [2022-11-08 17:12:43] [console] {mutex lock failed: Invalid argument}
2022-11-08T09:12:43.0892240Z �[0m
2022-11-08T09:12:43.0892300Z 
2022-11-08T09:12:43.0892310Z 
2022-11-08T09:12:43.0892310Z 
2022-11-08T09:12:43.0901080Z libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
2022-11-08T09:12:43.0901740Z libc++abi.dylib: terminate_handler unexpectedly threw an exception
2022-11-08T09:12:43.0902000Z libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
2022-11-08T09:12:43.0902240Z libc++abi.dylib: terminate_handler unexpectedly threw an exception
2022-11-08T09:12:43.0902480Z libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
2022-11-08T09:12:43.0902710Z libc++abi.dylib: terminate_handler unexpectedly threw an exception
2022-11-08T09:12:43.0902960Z libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
2022-11-08T09:12:43.0903200Z libc++abi.dylib: terminate_handler unexpectedly threw an exception
2022-11-08T09:12:43.0903440Z libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
2022-11-08T09:12:43.0903660Z libc++abi.dylib: terminate_handler unexpectedly threw an exception
2022-11-08T09:12:43.0903900Z libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
2022-11-08T09:12:43.0904130Z libc++abi.dylib: terminate_handler unexpectedly threw an exception
2022-11-08T09:12:43.0904360Z libc++abi.dylib: terminating with uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument

cc: @lin-hitonami

@taichi-gardener taichi-gardener moved this to Untriaged in Taichi Lang Nov 8, 2022
lin-hitonami added a commit that referenced this issue Nov 9, 2022
Issue: fixes #6546 

### Brief Summary
I've run the test for 1000 times and the flaky test never appear again.

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Repository owner moved this from Untriaged to Done in Taichi Lang Nov 9, 2022
quadpixels pushed a commit to quadpixels/taichi that referenced this issue May 13, 2023
Issue: fixes taichi-dev#6546 

### Brief Summary
I've run the test for 1000 times and the flaky test never appear again.

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants