Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Fix logcontext leak from test_sync #4213

Closed
wants to merge 1 commit into from
Closed

Conversation

richvdh
Copy link
Member

@richvdh richvdh commented Nov 20, 2018

Avoid timing out the request, which for long and complicated reasons causes logcontext leaks.

Avoid timing out the request, which for long and complicated reasons causes
logcontext leaks,
@richvdh richvdh requested a review from a team November 20, 2018 22:50
# it completes, we can end up leaking log-contexts (see log_contexts.rst
# for more details).
#
# In short, a timeout here makes things explode down the line.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lack of a timeout here causes an infinite loop on jammed web requests in tests...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't the test have an overall timeout?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I don't mind having an explicit timeout here, but what is a problem is making it an expected part of a test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The timeout is done by the global reactor, this here does not yield to the global reactor, and therefore can run infinitely and not trip the global reactor timeout.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, that's sad, but given that if we're in that situation the test should have failed, is it really a problem?

Copy link
Contributor

@hawkowl hawkowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I like this approach. Should we not instead, in tests.unittest.HomeserverTestCase, reset the log context to None either during setUp, tearDown, or both? Hanging Deferreds, due to how many moving parts there are in the HomeServer, are likely to happen often.

@richvdh
Copy link
Member Author

richvdh commented Nov 21, 2018

the problem is that the logcontext is already clear during setup and teardown. It is then later restored - in the middle of some completely random test - by the orphaned deferred chain being garbage-collected.

I'm with you, it's not a nice approach. But I can't really think of any better ones right now. Options might be:

  • completely rewrite the way we manage logcontexts: this is an epic job and I'm not entirely sure what might be better
  • Hack PreserveLogContext to not restore logcontexts on GeneratorExit, on the assumption that whatever was meant to be clearing the logcontext has already been garbage-collected. This isn't attractive as it's certainly still possible for things to happen which ought to be logged against the request context, and anyway it feels like a massive hack.
  • Make sure that Notifier errbacks any Deferreds which are in its queue before it gets gced. This is certainly an option, but feels like a bit of a rabbithole given I'm supposed to be doing completely different things and just want the tests to pass on my damn PRs.

@hawkowl
Copy link
Contributor

hawkowl commented Nov 21, 2018

@richvdh Can we force a GC between tests to cause the teardown to happen immediately?

@richvdh
Copy link
Member Author

richvdh commented Nov 22, 2018

possibly. It feels horrible though.

@richvdh
Copy link
Member Author

richvdh commented Nov 26, 2018

ok fine. Have done #4227 instead.

@richvdh richvdh closed this Nov 26, 2018
@richvdh richvdh deleted the rav/no_timeout_sync_test branch December 1, 2020 12:36
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants