Add --only flag for running test.only tests

[sindresorhus]

From #1467 (comment)

#1467 makes test.only() not be exclusive with multiple test files, so would be nice to have this flag.

[novemberborn]

#1467 makes test.only() not be exclusive with multiple test files, so would be nice to have this flag.

To be exact, AVA won't switch into exclusive mode until the first test.only() file is encountered. For subsequent files only exclusive tests will be run.

[revelt]

For the record, the new .only behaviour is quite annoying and actually impacts my productivity. I often separate the secondary library's functionality into util.js and therefore, unit tests go separately too. When I troubleshoot code, I rely on console.log's and it is especially important that I should be able to isolate that one particular unit test, otherwise, I'll get multiple console.logs from multiple unit tests.

Now, if we'll end up with functionality as Mark says, where t.only kicks in only after first only is encountered, it's pretty much a futile feature – I just checked, nearly always my util unit tests run first in the queue and therefore would contaminate the console. We need something more.

Maybe it's a stupid idea, but why can't we just run a glob against all recognised test files and look for any .onlys in the unit tests? Then, have a think what to do, run .onlys or run everything as it is now? Basically, a one, additional, separate round in the execution chain??? Because we're already globb'ing all test files, we just need to scan them first, before action commences.

What do you think?

[novemberborn]

@revelt,

why can't we just run a glob against all recognised test files and look for any .onlys in the unit tests? Then, have a think what to do, run .onlys or run everything as it is now? Basically, a one, additional, separate round in the execution chain??? Because we're already globb'ing all test files, we just need to scan them first, before action commences.

We did consider this previously. There's many ways of using AVA though, and scanning for the occurrence of .only( is not sufficient.

if we'll end up with functionality as Mark says, where t.only kicks in only after first only is encountered, it's pretty much a futile feature – I just checked, nearly always my util unit tests run first in the queue and therefore would contaminate the console. We need something more.

Yes, that is the behavior. The solution is for you to be able to specify that you're running .only() tests using the --only flag on the CLI. That way AVA will not run any tests that are not marked as .only(). This isn't fantastic but we've held back changing the default concurrency for a very long time because of this, and honestly the best solution we've found is this --only flag.

Perhaps you might have some time to help us out here?

---

I often separate the secondary library's functionality into util.js and therefore, unit tests go separately too. When I troubleshoot code, I rely on console.log's and it is especially important that I should be able to isolate that one particular unit test, otherwise, I'll get multiple console.logs from multiple unit tests.

Sounds like the new t.log() feature might be useful for you.

[revelt]

I'm back after 1.5 years, still using ava, unit tests are getting out of control, it's very difficult to work on a feature by isolating it in one of test files. This feature is a must-have.

[novemberborn]

@revelt perhaps you might have some time to help us out here?

[revelt]

Maybe it's an inefficient way but I think I have an MVP of this feature implemented on my local fork.

I haven't wired up CLI flag --only to trigger this but all files are (synchronously) read and we check for both .only( or secret keyword avaonly. If either keyword is found, that particular test file is output as findTestFiles() result.

This gives actually two new features:

1. .only in any of the files will be picked up, even if there are many test files
2. if you put avaonly in any of test files (as a comment), that test file will be isolated

Second feature helps to deal with large program's unit tests, to concentrate on a particular test file and "nail it", without polluting the terminal with already implemented features from other test files. Both .only and avaonly can co-exist, former overrides latter.

The file read below maybe could be async and --only CLI flag wiring is missing, but it does work and I'm using the below piece, on a cloned, npm-linked ava:

findTestFiles() {
	return handlePaths(this.files, this.extensions, this.excludePatterns, Object.assign({
		cwd: this.cwd,
		expandDirectories: false,
		nodir: false
	}, this.globCaches)).then(filesArr => {
    if (Array.isArray(filesArr) && filesArr.length > 1) {
      for (let i = filesArr.length; i--;) {
        const filesContents = fs.readFileSync(filesArr[i], "utf8");
        if (
          filesContents.includes(".only(") || filesContents.includes("avaonly")
        ) {
          return [filesArr[i]]
        }
      }
      return filesArr;
    }
    return filesArr
	});
}

The addition is .then(filesArr => { and onwards. Let me know what you think.

[novemberborn]

@revelt the problem with looking for substrings is that any other API you're using in your test file that happens to match the substring causes AVA to ignore all other files.

[timoweave]

Is this still opened?

We could generate a hash key from the title (which is unique), so that one can do the following:
--ignore ce2dce052a //like code-time test.skip, but run time
--match c77e93d5f8 // like code-time test.only, but run time
--find c77e93d5f8 // print out the file name and line number

see my pull request #2196

NOTE: All the test runner framework does not have that "hash title" concept...