Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix nunit tests adapter losing async locals #16157

Merged
merged 5 commits into from
Jul 17, 2024

Conversation

maxkatz6
Copy link
Member

What does the pull request do?

NUnit actively uses AsyncLocal to globally store TestContext.
Avalonia Headless integration with NUnit was messing up with ExecutionContext causing weird bugs like:

    [AvaloniaTest]
    public void Test1() {
        // Executes first.
    }

    [AvaloniaTest]
    public void Test2() {
        // Executes second, but test context is from the first test.
        var name = TestContext.CurrentContext.Test.Name;
        Assert.That(name, Is.EqualTo("Test2")); // Fails.
    }

How was the solution implemented (if it's not obvious)?

I couldn't understand exactly how was it broken, but my hypothesis is:

  1. Avalonia headless runs a dedicated thread with its own ExecutionContext.
  2. NUnit caches and saves current TestContext in the AsyncLocal (which uses ExecutionContext) - https://github.com/nunit/nunit/blob/v3.13.3/src/NUnitFramework/framework/Internal/TestExecutionContext.cs#L154
  3. It picks up first TestContext of the first running test, caches it, and never invalidates.
  4. NUnit itself resets TestContext via EstablishExecutionEnvironment. But it has no effect on Avalonia Tests, which don't use the same ExecutionContext.
  5. Manually calling EstablishExecutionEnvironment on each tests also doesn't work, as it resets SynchronizationContext (!!!!) to NUnit's one as well, breaking anything Dispatcher related.

And it was fixed by explicitly capturing ExecutionContext of the invoking test framework (NUnit in this case), and explicitly running tests in the same context, while keeping our own SynchronizationContext.

Checklist

Breaking changes

Obsoletions / Deprecations

Fixed issues

@maxkatz6 maxkatz6 added bug area-headless backport-candidate-11.1.x Consider this PR for backporting to 11.1 branch labels Jun 28, 2024
@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.2.999-cibuild0049593-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@avaloniaui-bot
Copy link

You can test this PR using the following package version. 11.2.999-cibuild0049613-alpha. (feed url: https://nuget-feed-all.avaloniaui.net/v3/index.json) [PRBUILDID]

@maxkatz6 maxkatz6 requested a review from a team July 3, 2024 08:13
@grokys grokys added this pull request to the merge queue Jul 17, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Jul 17, 2024
@maxkatz6 maxkatz6 merged commit 8ea60fe into master Jul 17, 2024
11 checks passed
@maxkatz6 maxkatz6 deleted the fix-mismatch-nunit-test-context branch July 17, 2024 21:52
grokys pushed a commit that referenced this pull request Jul 23, 2024
* Fix NUnit test context not being properly set

* Add failing tests

* Capture ExecutionContext to keep async locals

* Remove explicit EstablishExecutionEnvironment call, as it was a bad idea

* Make ExecutionContext usage disabled by default, and only enabled for NUnit
grokys pushed a commit that referenced this pull request Jul 24, 2024
* Fix NUnit test context not being properly set

* Add failing tests

* Capture ExecutionContext to keep async locals

* Remove explicit EstablishExecutionEnvironment call, as it was a bad idea

* Make ExecutionContext usage disabled by default, and only enabled for NUnit
grokys pushed a commit that referenced this pull request Aug 2, 2024
* Fix NUnit test context not being properly set

* Add failing tests

* Capture ExecutionContext to keep async locals

* Remove explicit EstablishExecutionEnvironment call, as it was a bad idea

* Make ExecutionContext usage disabled by default, and only enabled for NUnit
@grokys grokys added backported-11.1.x and removed backport-candidate-11.1.x Consider this PR for backporting to 11.1 branch labels Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants