Skip features that are not selected by tags: i.e. logs and test life cycle (befores/afters)

[mdelapenya]

I noticed that Godog always prints all the features present in the project, although they are filtered by Cucumber tags

As an example:

$ godog -t rediasda
Feature: As a Metricbeat developer I want to check that the Apache module works as expected

Feature: As a Metricbeat developer I want to check that default configuration works as expected

Feature: As a Metricbeat developer I want to check that the MySQL module works as expected

Feature: As a Metricbeat developer I want to check that the Redis module works as expected

Feature: As a Metricbeat developer I want to check that the vSphere module works as expected

No scenarios
No steps
146.152µs

I'd like to omit those features not selected to be run

[mdelapenya]

I noticed it also executes the before and after methods for any FeatureContext. I'd expect that those points in the life cycle would be skipped because of the tag filter.

[mdelapenya]

I'm going deeper in the godog code to understand the lifecycle, and maybe I'm wrong: how the RunWithOptions method selects which FeatureContext to run?

I used godog directly but I'm thinking about moving to go test, but don't want to reinvent the wheel to parse the flags to discover tags and select the featurecontext (I have multiple go files providing steps, one per test suite).

Then I have this question: how many FeatureContext functions are recommended by project? I'd imagine one per module, contributing life cycle methods to the test runner. Then, this runner would select what to run depending on the flags/tags. I'd love any guidance here 🙏

[mdelapenya]

Hey @l3pp4rd , any feedback here? 🙏

[mdelapenya]

In the project I use godog we have different pieces/modules under the same project layout. Each of them is contributing a different set of steps in the form of a func (s *godog.Suite) function, and a structure representing the test suite for that piece. Each feature file is tagged with a tag, representing the module.

What I observe running godog -t $module is that all function are discovered by this execution, so anything in the life cycle of that suite (befores and afters) are executed even though I want to "discover" only those steps contributors for the specified tag.

I created this same repo reproducing the same behaviour: https://github.com/mdelapenya/sample-godog

[mxygem]

Heya @mdelapenya thanks so much for passing along so much info! I'll take a look and see what I can find. Godog's been going through a lot of changes recently and there's definitely more that need to happen. As a heads up, l3pp4rd is heads down on other work unrelated to godog. Right now there's not many super active folks, so please bear with us, especially given the crazy time in the world right now.

If you've got the cycles and are up for submitting a PR, by all means I welcome it!

[mdelapenya]

Thanks @jaysonesmith for the feedback! I've been reading the source code of the builder and, with my current knowledge of the codebase, I see it complex to couple the life cycle of the tags with the feature contexts defining the steps supporting that tag: the godog test binary inspects the exported contexts without taking runtime tags into consideration (i.e. this tag applies to these scenario/example). On the other hand, I have a test demonstrating this behavior in the builder_test.go test file.

If too complex, I'd consider having different subprojects (different go.mod files) per submodule, as in the _examples dir, but with some guidance I could make progress on a feature/fix

[lonnblad]

What do you think, should we handle empty feature files in the same way?
@jaysonesmith @mdelapenya

[mdelapenya]

Adding context to this: I moved the project layout to a more standard one, so each feature context is under one test project. Therefore is a responsibility of the build system or the CI to run one or the other.

Thanks for your feedback here!