plugin support

[davidjgoss]

🤔 What's the problem you're trying to solve?

If you want to augment the core functionality of Cucumber - for something that's not a formatter - you have two choices:

1. Push it into the core (i.e. what has been done for publishing to the reports service)
2. Wrap Cucumber, influencing its inputs and acting on its outputs (i.e. what has been started for TCP-based ordering)

Neither of these options are great for the average user with a good idea, and both of these specific use cases would be better served by an official plugin mechanism so you can hook into Cucumber's test run lifecycle, change things, and react to things.

✨ What's your proposed solution?

TBA!

At a basic level we need two things:

A way to write a plugin

The bare minimum might be something like:

export default {
  async init() {}
  onMessage(envelope) {}
}

We'll also likely need:

• A way to provide configuration for/to the plugin
• A way for the plugin to output to stderr (e.g. the publish banner)
• "Lifecycle hooks" so plugins can provide a function which can influence core logic - so for the TCP case something like async inferOrder(pickles: Pickle[]): Promise<Pickle[]>

A way to reference a plugin

Something like:

npx cucumber-js --plugin @cucumber/publish

---

I think a good start would be to create some plumbing for plugins internally, and reimplement the publish functionality in terms of that. I'm happy to give that a go.

[davidjgoss]

cc @aslakhellesoy @mattwynne

[aurelien-reeves]

I like the idea, and I agree with your analysis and choices 👌

[mattwynne]

I think an approach where we try to identify and refactor existing/internal code towards using an API boundary that we could stabilise and make public sounds great. Happy to pair with you on this @davidjgoss if you want to bounce some ideas around.

Not necessarily solving the same problem but worth adding for context:

When @tooky and I re-wrote the guts of cucumber-ruby, we ended up with a "plugin" mechanism based on the chain-of-responsibility design pattern. Express's request handlers are another example of the same pattern. Each plugin (or "filter" as we called them) receives the stream of test cases (like a pickle, but with extra info about how to invoke it) and is given a receiver (the next filter in the chain) to whom it should pass on whichever test cases it chooses. It can filter out test cases by choosing not to pass them on, and it can also enrich or otherwise mutate test cases before passing them on.

A lot of code fell into place when we adopted this model, allowing us to assemble most of Cucumber's behavour from lots of small pieces, e.g:

• filtering test cases by name
• filtering and sorting test cases by location
• glueing step definition functions to the test case's steps
• randomizing test cases
• initializing the world

I'm not sure if you already have something like this inside cucumber-js, but it might be worth thinking about as part of this. I'm pretty sure it would provide what the TCP extension needs, for example.

[mattwynne]

https://github.com/cucumber/cucumber-electron is another piece worth considering in all this, currently implemented as a wrapper but we agreed (in theory) that it should really be part of this codebase.

[mattwynne]

More prior art is the https://github.com/cucumber/cucumber-ruby-wire/blob/main/lib/cucumber/wire.rb extension for cucumber-ruby which does use an officially-supported (though poorly documented / publicised) plugin API.

[charlierudolph]

Formatters could also be a special form of a plugin. Maybe have someway for the plugin to declare its a formatter and thus to get passed a function to log (which will be either a file stream or stdout)

In general, like the idea. Very curious what exactly will be the scope of what get supported