Enhancement to allow to skip a test from the code

[juliangp]

🤔 What's the problem you're trying to solve?

We require the means to skip a test from the code. Here are a couple of examples:

1. We run tests that require an environment variable, if that variable is not set then we want to skip the test.
2. We run tests to check tls ciphers on a server for a given tls version, if the server does not support that tls version then we want to skip the test.

✨ What's your proposed solution?

The proposed solution is to have an error (something like godog.SkipTest) that can be returned from the code and that will mark the test/scenario as skipped but not as failed.

⛏ Have you considered any alternatives or workarounds?

For now we have a test context where we set a flag and then we have to add code to skip all godog methods if flagged.
This has the disadvantage that the test is not marked as skipped.

📚 Any additional context?

---

This text was originally generated from a template, then edited by hand. You can modify the template here.

[roskee]

Perhaps you can use the godog.ErrPending error so that the test passes with a warning.

[juliangp]

@roskee unfortunately that marks the tests as failed which is not what I want. I would like something similar, the only difference is that the tests are not marked as failed. I could probably provide a PR for this if required?

[longyue0521]

@juliangp maybe you can use --godog.tags=~@skip，run tests that without skip tag

[juliangp]

@juliangp maybe you can use --godog.tags=~@skip，run tests that without skip tag

@longyue0521 We already use this but we need something dynamic in the test, for instance when starting a test, for a given model, the server might return that the model is not deployed on the server, in this case we want to skip the rest of the test.

[juliangp]

@vearutop Thank you!

[juliangp]

@vearutop do you know if there is a new release planned soon that will contain this feature? Thanks in advance.

[vearutop]

Indeed, there were a few nice improvements lately, let's make a new release.

[vearutop]

v0.13.0 tagged.