Race condition results in error "Some of your tests did a full page reload!"

[nicojs]

Expected behaviour

When running Karma with singleRun = false and autoWatch = false I would expect to be able to schedule a new run as soon as the 'old' run is completed.

Actual behaviour

A race condition in clearContext can report a false error in the new run.

Environment Details

• Karma version (output of karma --version): 4.4.1
• Relevant part of your karma.config.js file

{
  singleRun: false,
  autoWatch: false
}

Steps to reproduce the behaviour

It is difficult to reproduce since it is a race condition. Essentially it can happen in this scenario:

const karam = require('karma');
const server = new karma.Server({ singleRun: false, watch: false });
await server.start();
function runUntilError() {
  karma.runner.run({}, exitCode => {
    if(exitCode === 0) {
       runUntilError();
    } else {
       console.log('Aha! Run errored');
    }
  });
}
runUntilError()

Triage

The problem is in the complete handler. Results are reported to the server before the context is cleared:

karma/client/karma.js

Lines 230 to 242 in 6235e68

	this.complete = function (result) {
	if (resultsBuffer.length) {
	socket.emit('result', resultsBuffer)
	resultsBuffer = []
	}
	if (self.config.clearContext) {
	// give the browser some time to breath, there could be a page reload, but because a bunch of
	// tests could run in the same event loop, we wouldn't notice.
	setTimeout(function () {
	clearContext()
	}, 0)
	}

The clearContext function sets the reloadingContext flag and navigates away.

karma/client/karma.js

Lines 148 to 152 in 6235e68

	function clearContext () {
	reloadingContext = true
	navigateContextTo('about:blank')
	}

Now, just before the browser navigates away, the reloadingContext is reset by the next execution:

karma/client/karma.js

Line 266 in 6235e68

reloadingContext = !self.config.clearContext

The concequence of this, is that the error is reported here:

karma/client/karma.js

Line 132 in 6235e68

self.error('Some of your tests did a full page reload!')

Some of your tests did a full page reload!

However, I didn't reload any page.

Workaround

If you set client.clearContext to false, the context will not be cleared and there is no risk of this race condition.

[johnjbarton]

Ok if I understand your analysis, our scenarios are

PASS:

1. execute test
2. emit result
3. clearContext on next event loop
4. clearContext sets reloadingContext =true
5. reload on next event loop
6. onbeforeunload checks reloadingContext
7. success!

FAIL

1. execute test
2. emit result
3. clearContext on next event loop
4. clearContext sets reloadingContext =true
5. reload on next event loop
6. execute test sets reloadingContext = false
7. onbeforeunload checks reloadingContext
8. fail

[karmarunnerbot]

🎉 This issue has been resolved in version 5.2.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[KurtPreston]

Just reporting that I'm still encountering this issue on Karma 5.2.2

However, setting clearContext to false resolves the issue for me.

[martinraie]

We started experiencing Some of your tests did a full page reload! error message after upgrading karma 5.1.1 => 5.2.0. First test run is OK but it starts failing since the second run. By default we have clearContext set to false. The problem goes away when setting clearContext to true. Could it be related to this issue?

[aronrodrigues]

+1 Karma 5.2.3

[johnjbarton]

Please open a new bug with steps to reproduce. This issue is related to a race in the code and steps to reproduce are essential.