Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ResourceWarning messages being printed #320

Open
manoelmarques opened this issue Mar 8, 2022 · 1 comment · Fixed by #322
Open

ResourceWarning messages being printed #320

manoelmarques opened this issue Mar 8, 2022 · 1 comment · Fixed by #322
Labels
bug
Milestone
4.0.0

Comments

@manoelmarques
Copy link

Issue description
When PYTHONWARNINGS is set to default, stestr prints ResourceWarning messages for files not closed and subprocesses still running.

Expected behavior and actual behavior

It shouldn't print the messages as they are sometimes interfering with the Totals report.

Steps to reproduce the problem

Run stestr with PYTHONWARNINGS env. set to default

Specifications like the version of the project, operating system, or hardware

System information

  • OS: [Ubuntu 20.04.4, macOS 11.6.4, Windows Server 2019]
  • stestr version (3.2.1):
  • Python release (3.7,3.8,3.9,3.10):
  • pip packages (pip freeze):

Additional information

sys:1: ResourceWarning: unclosed file <_io.FileIO name=1 mode='wb' closefd=True>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/site-packages/stestr/commands/load.py:287: ResourceWarning: unclosed file <_io.FileIO name=6 mode='rb' closefd=True>
  case.run(result)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/subprocess.py:1052: ResourceWarning: subprocess 4002 is still running
  _warn("subprocess %s is still running" % self.pid,
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/opt/hostedtoolcache/Python/3.9.10/x64/lib/python3.9/subprocess.py:1052: ResourceWarning: subprocess 4000 is still running

======
  _warn("subprocess %s is still running" % self.pid,
Totals
ResourceWarning: Enable tracemalloc to get the object allocation traceback
======
Ran: 452 tests in 694.6567 sec.
 - Passed: 452
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 1237.7825 sec.

==============
Worker Balance
==============
 - Worker 0 (226 tests) => 0:09:03.602362
 - Worker 1 (226 tests) => 0:11:34.656738
@mtreinish mtreinish added this to the 4.0.0 milestone Mar 8, 2022
masayukig added a commit to masayukig/stestr that referenced this issue Apr 14, 2022
@masayukig
Add destructor to wait for process termination
da0a011 
This commit adds a destructor to wait for a process termination in
`ReturnCodeToSubunit` class. This commit fixes mtreinish#320 partially.
@masayukig masayukig mentioned this issue Apr 14, 2022
Merged
@mtreinish mtreinish reopened this Apr 15, 2022
@mtreinish
Copy link
Owner

I think there are still other warnings being emitted (I think around open fds) so I reopened this but #322 should have fixed a bunch of the warnings

mtreinish added a commit that referenced this issue Feb 6, 2023
@mtreinish
Fix ResourceWarning from SubunitTestRunner._list()
8d322c0 
This commit fixes the ResourceWarning emitted by stestr's
SubunitRunner._list method caused by a leaking file descriptor when that
method exits. The root cause of this was this method was calling
fdopen() internally to open a new descriptor to the user specified
result stream and never closing it when the method returns. The issue
was that the return from that method had a reference to that fd and
closing it would have resulted in potentially unexpected behavior.
However, this code is not needed, it was just ported from subunit's
SubunitTestRunner class when this code was forked and rewritten to use
unittest instead of testtools. The subunit code is used in a broader
context and the fdopen might be needed there. But for stestr, we
always use stdout for the result stream as this only gets called
internally the worker processes to run tests. If we used something other
than stdout the result stream would not get sent to the parent process.
Since we're alwwys using stdout we don't need an fdopen call because we
never will need a fresh fd for it. This commit removes the legacy
baggage causing this ResourceWarning and just interacts with the stream
directly avoiding an additional open that never gets closed.

Related to #320
@mtreinish mtreinish mentioned this issue Feb 6, 2023
Merged
mtreinish added a commit that referenced this issue Feb 6, 2023
@mtreinish
Fix ResourceWarning from SubunitTestRunner._list()
17f64db 
This commit fixes the ResourceWarning emitted by stestr's
SubunitRunner._list method caused by a leaking file descriptor when that
method exits. The root cause of this was this method was calling
fdopen() internally to open a new descriptor to the user specified
result stream and never closing it when the method returns. The issue
was that the return from that method had a reference to that fd and
closing it would have resulted in potentially unexpected behavior.
However, this code is not needed, it was just ported from subunit's
SubunitTestRunner class when this code was forked and rewritten to use
unittest instead of testtools. The subunit code is used in a broader
context and the fdopen might be needed there. But for stestr, we
always use stdout for the result stream as this only gets called
internally the worker processes to run tests. If we used something other
than stdout the result stream would not get sent to the parent process.
Since we're alwwys using stdout we don't need an fdopen call because we
never will need a fresh fd for it. This commit removes the legacy
baggage causing this ResourceWarning and just interacts with the stream
directly avoiding an additional open that never gets closed.

Related to #320
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
4.0.0
Development

Successfully merging a pull request may close this issue.

Add destructor to wait for process termination masayukig/stestr
2 participants
@mtreinish @manoelmarques