Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k6 v0.42.0 updates - helper scripts (3/5) #23

Merged
merged 4 commits into from
Feb 10, 2023
Merged

Conversation

imiric
Copy link
Contributor

@imiric imiric commented Feb 6, 2023

This is PR 3/5, part of updating the "Running large tests" article and running benchmarks on k6 v0.42.0.

These changes add a couple of helper scripts:

  • k6bench.sh: runs k6 in the background, while collecting performance metrics (CPU/RAM usage of the k6 process, vus and http_reqs metrics from the k6 HTTP API), and writing them to stdout in CSV format.
    The polling period is configurable with the K6_BENCH_SAMPLE_INTERVAL env var, and defaults to 5s.
    Example command:
    ./k6bench.sh --vus=100 --duration=10s scripts/website.js
    It has two external dependencies: smem and jq.
  • k6bench.gnuplot: a gnuplot 5.4 script to generate graphs from the CSV output from the k6bench.sh script.
    Example command:
    gnuplot -e 'k6version="v0.42.0"; ec2instance="m5.large"; script="website.js-10s"' ../../k6bench.gnuplot
    This will generate a m5.large-website.js-10s.png in the current directory.

More command examples and outputs of these scripts can be seen in #24.

The idea is that these scripts will be run in CI and the entire benchmarking process will be automated. I didn't want to rely on k6 Cloud for metric aggregation and graphing for several reasons:

  • Using --out cloud consumes considerable network bandwidth, which could affect some of the network tests, CPU usage or even RPS.
  • The k6 Cloud has no way of measuring CPU and memory usage of the k6 process itself. Not until k6 is able to emit these metrics internally, which I still think is the wrong thing to do, considering how many trivial ways it's possible to do this externally.
  • It's best to keep the benchmarking offline without depending on a separate service so that users can run the same scripts we do and verify the results independently of any 3rd party. We could possibly push the results to Grafana to get nicer and interactive graphs, but the gnuplot graphs are not that bad. :)

I'm open to any improvement suggestions to this process, but so far I think it's the best approach.

PR order: #21 -> #22 -> #23 -> #24 -> #25 (review and merge in reverse)

Ivan Mirić added 4 commits February 6, 2023 18:52
This ensures a more accurate reading and output.
To ensure it outputs 0 in case the PID is not found.
@imiric imiric requested review from mstoykov and sniku February 6, 2023 18:32
@imiric imiric changed the base branch from master to update/k6-v0.42.0-remove-es5 February 6, 2023 18:33
rpsf=$(mktemp)

cleanup() {
rm -f "$cpuf" "$memf" "$vusf" "$rpsf"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I prefer to wrap variables in quotes and in ${} not only prefix them with $.

IIRC that also prevents you from potentially matching a shorter variable that is just prefix of the longer one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, I do that as well where it's useful (e.g. part of an interpolated string), but in this case there's no reason to do so, since they're separate arguments, and there's no risk of mistakenly using a different variable.

Copy link

@mstoykov mstoykov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but I would prefer to get the bash variables wrapped in quotes and in ${}

Base automatically changed from update/k6-v0.42.0-remove-es5 to master February 10, 2023 11:19
@imiric imiric merged commit dcb5359 into master Feb 10, 2023
@imiric imiric deleted the update/k6-v0.42.0-scripts branch February 10, 2023 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants