Stablize cargo test

[yihuaf]

The root cause of the flakiness observed in the hook, clean up file descriptors, and channel tests are the result of running the tests in parallel in the same process by cargo test. Many of the operations have side effects on the underlying OS and the process, such as closing down fds. These tests also depends on the underlying OS state such as file descriptor states. The best we can do is to mark these tests as serial and properly clean up any left over resources such as opened fds before the test ends. If the underlying OS decided to generate errors for some of the IO calls we are doing, then there is not much we can do. Not sure if retry is a valid solution here and retry generally will make the code much more complex.

Note: runc actually doesn't have unit tests on most of these operations. The other solution is to fully depend on the integration tests to test out these code path. It may be the reasonable thing to do if these unit tests prove to be hard to get right.

Note2: Currently, I ran cargo test for 100 iterations and no failed tests. There may still be flakiness, but it should be extremely rare.

[codecov-commenter]

Codecov Report

Merging #277 (4c5d2d6) into main (ef9a92a) will increase coverage by 0.01%.
The diff coverage is 53.84%.

@@            Coverage Diff             @@
##             main     #277      +/-   ##
==========================================
+ Coverage   69.57%   69.59%   +0.01%     
==========================================
  Files          46       46              
  Lines        5710     5651      -59     
==========================================
- Hits         3973     3933      -40     
+ Misses       1737     1718      -19     

[utam0k]

Thank you for your investigation. Can I ask you to write a clear description of the results of this investigation in the section of the test that we are using serial other tests?

The root cause of the flakiness observed in the hook, clean up file descriptors, and channel tests are the result of running the tests in parallel in the same process by cargo test. Many of the operations have side effects on the underlying OS and the process, such as closing down fds. These tests also depends on the underlying OS state such as file descriptor states. The best we can do is to mark these tests as serial and properly clean up any left over resources such as opened fds before the test ends. If the underlying OS decided to generate errors for some of the IO calls we are doing, then there is not much we can do. Not sure if retry is a valid solution here and retry generally will make the code much more complex.

[yihuaf]

Thank you for your investigation. Can I ask you to write a clear description of the results of this investigation in the section of the test that we are using serial other tests?

The root cause of the flakiness observed in the hook, clean up file descriptors, and channel tests are the result of running the tests in parallel in the same process by cargo test. Many of the operations have side effects on the underlying OS and the process, such as closing down fds. These tests also depends on the underlying OS state such as file descriptor states. The best we can do is to mark these tests as serial and properly clean up any left over resources such as opened fds before the test ends. If the underlying OS decided to generate errors for some of the IO calls we are doing, then there is not much we can do. Not sure if retry is a valid solution here and retry generally will make the code much more complex.

Done. I left the tty test alone because it was the original user of serial tests I think and is not part of this change.

[utam0k]

thanks