Improve externally-controlled executor docs to explain internals better

[javaducky]

Feature Description

Attempts to scale up a test run will fail unless a maximum value is also specified or has previously been specified.

Start a script with the basic options below:

export const options = {
  scenarios: {
    k6_workshop: {
      executor: 'externally-controlled',
      duration: '5m',
    },
  },
};

Attempt to scale up using k6 commandline:

$ k6 scale --vus 2
ERRO[0000] Config update error: invalid configuration supplied: the number of active VUs (2) must be less than or equal to the number of maxVUs (0) 

Scaling up while also providing the maximum (e.g. k6 scale --vus 2 --max 10) works successfully, as expected.

I feel that unless explicitly set, the max should be understood as being unlimited. Otherwise, the maxVUs setting should be a required parameter for the externally-controlled executor configuration options.

Suggested Solution (optional)

No response

Already existing or connected issues / PRs (optional)

No response

[na--]

This can't be done because "max" VUs basically means "initialized" VUs, and you can't initialize infinite VUs. To see what I mean, run your script with k6 run --verbose and see what happens when you do k6 scale --max 10.

maxVUs and --max are not great option names, but I think we chose not to change them in grafana/k6#1007 (k6 v0.27.0) to somewhat keep compatibility with previous k6 versions, even though the externally-controlled executor itself was somewhat of a breaking change... 😞

That said, https://k6.io/docs/using-k6/scenarios/executors/externally-controlled can probably be greatly improved, so I'll move this issue to the k6-docs repo and rename it. It somewhat overlaps with #148 and #367, but is specific enough to merit its own issue.

[javaducky]

Should we not code a default maxVUs instead?

[na--]

What value would that have and why? Given that you can adjust the max VUs once a k6 test run has been started, I don't see a lot of utility in it.

Improving the docs and potentially the k6 error messages seems both more beneficial and less problematic 🤷‍♂️

[javaducky]

To me, the error message made perfect sense, but what didn't make sense is the usage.

If maxVUs must be set in order to do anything, why not force the script options to either provide that value, otherwise provide a reasonable default. Providing the ability to change that max using the k6 scale --max 999 is fine as it's very explicit.

I don't deny that documentation can always be improved, but to me, it's not the best usability for the API.

[na--]

If maxVUs must be set in order to do anything, why not force the script options to either provide that value

It doesn't. The k6 test run could have other scenarios besides the externally-controlled, or even just setup() and teardown(). The user might not want to initialize any VUs before some runtime condition determines how many VUs have to be initialized, etc.

otherwise provide a reasonable default.

Again, what do you think is a reasonable default? I don't think such a value can be determined 😕 Maybe 1, but even that's contentious.

I am not saying the current API is the best, I already mentioned that --max / maxVUs is quite misleading IMO. But I don't see how we can implement what you suggest, and I am not sure it would improve things significantly even if we figure that out...