v0.26.0

k6 v0.26.0 is here! 🎉

This release contains mostly bug fixes, though it also has several new features and enhancements! They include a new JS compatibility mode option, exporting the end-of-test summary to a JSON report file, speedups to the InfluxDB and JSON outputs, http.batch() improvements, a brand new CSV output, multiple layered HTTP response body decompression, being able to use console in the init context, a new optional column in the summary, and Docker improvements!

Thanks to @Sirozha1337, @openmohan, @MMartyn, @KajdeMunter, @dmitrytokarev and @dimatock for contributing to this release!

New features and enhancements!

A new JavaScript compatibility mode option (#1206)

This adds a way to disable the automatic script transformation by Babel (v6.4.2) and loading of core-js (v2) polyfills, bundled in k6. With the new base compatibility mode, k6 will instead rely only on the goja runtime and what is built into k6.
This can be configured through the new --compatibility-mode CLI flag and the K6_COMPATIBILITY_MODE environment variable. The possible values currently are:

• extended: this is the default and current compatibility mode, which uses Babel and core.js to achieve ES6+ compatibility.
• base: an optional mode that disables loading of Babel and core.js, running scripts with only goja's native ES5.1+ compatibility. If the test scripts don't require ES6 compatibility (e.g. they were previously transformed by Babel), this option can be used to reduce RAM usage during test runs.

More info what this means can be found in the documentation.

Our benchmarks show a considerable drop in memory usage - around 80% for simple scripts, and around 50% in the case of 2MB script with a lot of static data in it. The CPU usage is mostly unchanged, except that k6 initializes test runs a lot faster. All of those benefits will be most noticeable if k6 is used with big number of VUs (1k+). More performance comparisons can be found in #1167.

JSON export of the end-of-test summary report (#1168)

This returns (from the very early days of k6) the ability to output the data from the end of test summary in a machine-readable JSON file.
This report can be enabled by the --summary-export <file_path> CLI flag or the K6_SUMMARY_EXPORT environment variable. The resulting JSON file will include data for all test metrics, checks and thresholds.

New CSV output (#1067)

There is an entirely new csv output that can be enabled by using the --out csv CLI flag. There are two things that can be configured: the output file with K6_CSV_FILENAME (by default it's file.csv), and the interval of pushing metrics to disk, which is configured with K6_CSV_SAVE_INTERVAL (1 second by default). Both of those can be configured by the CLI as well: --out csv=somefile.csv will output to somefile.csv and --out file_name=somefile.csv,save_interval=2s will output again to somefile.csv, but will flush the data every 2 seconds instead of every second.

The first line of the output is the names of columns and looks like:

metric_name,timestamp,metric_value,check,error,error_code,group,method,name,proto,status,subproto,tls_version,url,extra_tags
http_reqs,1573131887,1.000000,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_duration,1573131887,116.774321,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_blocked,1573131887,148.691247,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,
http_req_connecting,1573131887,112.593448,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/,

All thanks to @Sirozha1337!

JSON output optimizations (#1114)

The JSON output no longer blocks the goroutine sending samples to the file, but instead (like all other outputs) buffers the samples and writes them at regular intervals (100ms and is currently not configurable). It also uses a slightly faster way of encoding the data, which should decrease the memory usage by a small amount.

Another improvement is the ability to compress the generated JSON file by simply adding .gz to the end of the file name. Compressed files are typically 30x smaller.

InfluxDB output improvements (#1113)

The InfluxDB output has been updated to use less memory and try to send smaller and consistent chunks of data to InfluxDB, in order to not drop packets and be more efficient. This is primarily done by sending data in parallel, as this seems to be better from a performance perspective, and more importantly, queuing data in separate packets, so that we don't send the data for a big time period all at once. Also, the used library was updated, which also decreased the memory usage.

Two new options were added:

• K6_INFLUXDB_PUSH_INTERVAL - configures at what interval the collected data is queued to be sent to InfluxDB. By default this is "1s".
• K6_INFLUXDB_CONCURRENT_WRITES - configures the number of concurrent write calls to InfluxDB. If this limit is reached the next writes will be queued and made when a slot is freed. By default this is 10.

console is now available in the init context (#982):

This wasn't supported for the longest time, which made debugging things outside of VU code much harder, but now it's here! 🎉

In order to get this feature shipped in a timely manner, it currently has a known bug. The output of console calls in the init context will always be written to the stderr, even if the --console-output option is specified. This bug is tracked in #1131

HTTP response body decompression with multiple layered algorithms (#1125)

In v0.25.0 compressing bodies was added and it had support for multiple layered algorithms. Now this is also true for decompressing bodies when k6 gets them as responses.

New optional count column in the end-of-test summary (#1143)

The --summary-trend-stats now also recognizes count as a valid column and will output the count of samples in all Trend metrics. This could be especially useful for custom Trend metrics, since with them you no longer need to specify a separate accompanying Counter metric.

Docker Compose refactor (#1183)

The example docker-compose that enabled easy running of InfluxDB+Grafana+k6 was refactored and all the images were updated to use the latest stable versions.

Thanks, @KajdeMunter!

Also the k6 Dockerfile Alpine version was bumped to 3.10. Thanks @dmitrytokarev!

http.batch() improvements and optimizations (#1259)

We made several small improvements to the mechanism for executing multiple HTTP requests simultaneously from a single VU:

• Calling http.batch() should now be more efficient, especially for many requests, because of reduced locking, type conversions, and goroutine spawning.
• The default value for batchPerHost has been reduced from 0 (unlimited) to 6, to more closely match browser behavior. The default value for the batch option remains unchanged at 20.
• Calling http.batch(arg), where arg is an array, would now return an array. Previously, this would have returned an object with integer keys, as explained in #767... Now http.batch() will return an array when you pass it an array, and return an object when you pass an object.

UX

• Better timeout messages for setup and teardown timeouts, including hints on how to fix them. (#1173)
• When a folder is passed to open(), the resulting error message will now include the path to the specified folder. (#1238)
• The k6 version output will now include more information - the git commit it was built from (in most cases), as well as the used Go version and architecture. (#1235)

Bugs fixed!

• Cloud: Stop sending metrics to the cloud output when the cloud returns that you have reached the limit. (#1130)
• JS: Fail a check if an uncaught error is thrown inside of it. (#1137)
• HTTP: Replace any user credentials in the metric sample tags with * when emitting HTTP metrics. (#1132)
• WS: Many fixes:
  • return an error instead of panicking if an error occurs during the making of the WebSocket connection (#1127)
  • calling the error handler on an error when closing the WebSocket, instead of calling with a null (#1118)
  • correctly handle server initiated close (#1186)
• JSON: Better error messages when parsing JSON fails. Now telling you at which line and row the error is instead of just the offset. Thanks, @openmohan! (#905)
• HTTP: Use Request's GetBody in order to be able to get the body multiple times for a single request as needed in 308 redirects of posts and if the server sends GOAWAY with no error. (#1093)
• JS: Don't export internal go struct fields of script options.(#1151)
• JS: Ignore minIterationDuration for setup and teardown. (#1175)
• HTTP: Return error on any request that returns 101 status code as k6 currently doesn't support any protocol upgrade behavior. (#1172)
• HTTP: Correctly capture TCP reset by peer and broken pipe errors and give them the appropriate error_code metric tag values. (#1164)
• Config: Don't interpret non-K6_ prefixed environment variables as k6 configuration, most notably DURATION and ITERATIONS. (#1215)
• JS/html: Selection.map was not wrapping the nodes it was outputting, which lead to wrongly using the internal Goquery.Selection instead of k6's Selection. Thanks to @MMartyn for reporting this! (#1198)
• HTTP: When there are redirects, k6 will now correctly set the cookie for the current URL, instead of for the one the current response is redirecting to. Thanks @dimatock! (#1201)
• Cloud: Add token to make calls to the cloud API idempotent. (#120...

v0.25.1

A minor release that fixes some of the issues in the v0.25.0 release.

Bugs fixed!

• Config: Properly handle the systemTags JS/JSON option and the K6_SYSTEM_TAGS environment variable. Thanks, @cuonglm! (#1092)
• HTTP: Fix how request bodies are internally specified so we can properly handle redirects and can retry some HTTP/2 requests. (#1093)
• HTTP: Fix the handling of response decoding errors and slightly improve the digest auth and --http-debug code. (#1102)
• HTTP: Always set the correct Content-Length header for all requests. (#1106)
• JS: Fix a panic when executing archive bundles for scripts with unsuccessfull import / require() calls. (#1097)
• JS: Fix some issues related to the handling of exports corner cases. (#1099)

v0.25.0

k6 v0.25.0 is here! 🎉

This release contains mostly bug fixes, though it also has a few new features, enhancements, and performance improvements. These include HTTP request compression, brotli and zstd support, massive VU RAM usage and initialization time decreases, support for importing files via https and file URLs, and opt-in TLS 1.3 support.

Thanks to @THoelzel, @matlockx, @bookmoons, @cuonglm, and @imiric for contributing to this release!

New features and enhancements!

HTTP: request body compression + brotli and zstd decompression (#989, #1082)

Now k6 can compress the body of any HTTP request before sending it (#989). That can be enabled by setting the new compression option in the http.Params object. Doing so will cause k6 to transparently compress the supplied request body and correctly set both Content-Encoding and Content-Length, unless they were manually set in the request headers by the user. The currently supported algorithms are deflate, gzip, brotli and zstd, as well as any combination of them separated by commas (,).

k6 now also transparently decompresses brotli and zstd HTTP responses - previously only deflate and gzip were supported. Thanks, @imiric! (#1082)

import http from 'k6/http';
import { check } from "k6";

export default function () {
    // Test gzip compression
    let gzippedReqResp = http.post("https://httpbin.org/post", "foobar".repeat(1000), { compression: "gzip" });
    check(gzippedReqResp, {
        "request gzip content-encoding": (r) => r.json().headers["Content-Encoding"] === "gzip",
        "actually compressed body": (r) => r.json().data.length < 200,
    });

    // Test br decompression
    let brotliResp = http.get("https://httpbin.org/brotli", {
        headers: {
            "Accept-Encoding": "gzip, deflate, br"
        },
    });
    check(brotliResp, {
        "br content-encoding header": (r) => r.headers["Content-Encoding"] === "br",
        "br confirmed in body": (r) => r.json().brotli === true,
    });

    // Test zstd decompression
    let zstdResp = http.get("https://facebook.com/", {
        headers: {
            "Accept-Encoding": "zstd"
        },
    });
    check(zstdResp, {
        "zstd content-encoding header": (r) => r.headers["Content-Encoding"] === "zstd",
        "readable HTML in body": (r) => r.body.includes("html"),
    });
};

Performance improvement: reuse the parsed core-js library across VUs (#1038)

k6 uses the awesome core-js library to support new JavaScript features. It is included as a polyfill in each VU (i.e. JS runtime) and previously, it was parsed anew for every VU initialization. Now, the parsing result is cached after the first time and shared between VUs, leading to over 2x reduction of VU memory usage and initialization times for simple scripts!

Thanks, @matlockx, for noticing this opportunity for massive optimization!

JS files can now be imported via https and file URLs (#1059)

Previously, k6 had a mechanism for importing files via HTTPS URLs, but required that the used URLs not contain the https scheme. As a move to align k6 more closely with the rest of the JS ecosystem, we now allow and encourage users to use full URLs with a scheme (e.g. import fromurlencoded from "https://jslib.k6.io/form-urlencoded/3.0.0/index.js") when they want to load remote files. file URLs are also supported as another way to load local modules (normal absolute and relative file paths still work) from the local system, which may be especially useful for Windows scripts.

The old way of importing remote scripts from scheme-less URLs is still supported, though except for the GitHub and cdnjs shortcut loaders, it is in the process of deprecation and will result in a warning.

Opt-in support for TLS 1.3 and more TLS ciphers (#1084)

Following its opt-in support in Go 1.12, you can now choose to enable support for TLS 1.3 in your k6 scripts. It won't be used by default, but you can enable it by setting the tlsVersion (or it's max sub-option) to tls1.3:

import http from 'k6/http';
import { check } from "k6";

export let options = {
    tlsVersion: {
        min: "tls1.2",
        max: "tls1.3",
    }
};

export default function () {
    let resp = http.get("https://www.howsmyssl.com/a/check");
    check(resp, {
        "status is 200": (resp) => resp.status === 200,
        "tls 1.3": (resp) => resp.json().tls_version === "TLS 1.3",
    });
};

Also, all cipher suites supported by Go 1.12 are now supported by k6 as well. Thanks, @cuonglm!

Bugs fixed!

• JS: Many fixes for open(): (#965)

  • don't panic with an empty filename ("")
  • don't make HTTP requests (#963)
  • correctly open simple filenames like "file.json" and paths such as "relative/path/to.txt" as relative (to the current working directory) paths; previously they had to start with a dot (i.e. "./relative/path/to.txt") for that to happen
  • windows: work with paths starting with / or \ as absolute from the current drive

• HTTP: Correctly always set response.url to be the URL that was ultimately fetched (i.e. after any potential redirects), even if there were non http errors. (#990)

• HTTP: Correctly detect connection refused errors on dial. (#998)

• JS: Run imports once per VU. (#975, #1040)

• Config: Fix blacklistIPs JS configuration. Thanks, @THoelzel! (#1004)

• HTTP: Fix a bunch of HTTP measurement and handling issues (#1047)

  • the http_req_receiving metric was measured incorrectly (#1041)
  • binary response bodies could get mangled in an http.batch() call (#1044)
  • timed out requests could produce wrong metrics (#925)

• JS: Many fixes for importing files and for URL imports in archives. (#1059)

• Config: Stop saving and ignore the derived execution values, which were wrongly saved in archive bundles' metadata.json by k6 v0.24.0. (#1057, #1076)

• Config: Fix handling of commas in environment variable values specified as CLI flags. (#1077)

Internals

• CI: removed the gometalinter check in CircleCI, since that project was deprecated and now exclusively rely on golangci-lint. (#1039)
• Archive bundles: The support for URL imports included a lot of refactoring and internal k6 changes. This included significant changes in the structure of .tar archive bundles. k6 v0.25.0 is backwards compatible and can execute bundles generated by older k6 versions, but the reverse is not true. (#1059)
• Archive bundles: The k6 version and the operating system are now saved in the archive bundles' metadata.json file. (#1057, #1059)

Breaking changes

• Previously, the Content-Length header value was always automatically set by k6 - if the header value was manually specified by the user, it would have been ignored and silently overwritten. Now, k6 would set the Content-Length value only if it wasn't already set by the user. (#989, #1094)

v0.24.0

v0.24.0 is here! 🎉

Another intermediary release that was mostly focused on refactoring and bugfixes, but also has quite a few new features, including the ability to output metrics to StatsD and Datadog!

Thanks to @cheesedosa, @ivoreis, @bookmoons, and @oboukili for contributing to this release!

New Features!

Redirect console messages to a file (#833)

You can now specify a file to which all things logged by console.log() and other console methods will get written to. The CLI flag to specify the output file path is --console-output, and you can also do it via the K6_CONSOLE_OUTPUT environment variable. For security reasons, there's no way to configure this from inside of the script.

Thanks to @cheesedosa for both proposing and implementing this!

New result outputs: StatsD and Datadog (#915)

You can now output any metrics k6 collects to StatsD or Datadog by running k6 run --out statsd script.js or k6 run --out datadog script.js respectively. Both are very similar, but Datadog has a concept of metric tags, the key-value metadata pairs that will allow you to distinguish between requests for different URLs, response statuses, different groups, etc.

Some details:

• By default both outputs send metrics to a local agent listening on localhost:8125 (currently only UDP is supported as a transport). You can change this address via the K6_DATADOG_ADDR or K6_STATSD_ADDR environment variables, by setting their values in the format of address:port.
• The new outputs also support adding a namespace - a prefix before all the metric names. You can set it via the K6_DATADOG_NAMESPACE or K6_STATSD_NAMESPACE environment variables respectively. Its default value is k6. - notice the dot at the end.
• You can configure how often data batches are sent via the K6_STATSD_PUSH_INTERVAL / K6_DATADOG_PUSH_INTEVAL environment variables. The default value is 1s.
• Another performance tweak can be done by changing the default buffer size of 20 through K6_STATSD_BUFFER_SIZE / K6_DATADOG_BUFFER_SIZE.
• In the case of Datadog, there is an additional configuration K6_DATADOG_TAG_BLACKLIST, which by default is equal to `` (nothing). This is a comma separated list of tags that should NOT be sent to Datadog. All other metric tags that k6 emits will be sent.

Thanks to @ivoreis for their work on this!

k6/crypto: random bytes method (#922)

This feature adds a method to return an array with a number of cryptographically random bytes. It will either return exactly the amount of bytes requested or will throw an exception if something went wrong.

import crypto from "k6/crypto";

export default function() {
    var bytes = crypto.randomBytes(42);
}

Thanks to @bookmoons for their work on this!

k6/crypto: add a binary output encoding to the crypto functions (#952)

Besides hex and base64, you can now also use binary as the encoding parameter for the k6 crypto hashing and HMAC functions.

New feature: unified error codes (#907)

Error codes are unique numbers that can be used to identify and handle different application and network errors more easily. For the moment, these error codes are applicable only for errors that happen during HTTP requests, but they will be reused and extended to support other protocols in future k6 releases.

When an error occurs, its code is determined and returned as both the error_code field of the http.Response object, and also attached as the error_code tag to any metrics associated with that request. Additionally, for more details, the error metric tag and http.Response field will still contain the actual string error message.

Error codes for different errors are as distinct as possible, but for easier handling and grouping, codes in different error categories are also grouped in broad ranges. The current error code ranges are:

• 1000-1099 - General errors
• 1100-1199 - DNS errors
• 1200-1299 - TCP errors
• 1300-1399 - TLS errors
• 1400-1499 - HTTP 4xx errors
• 1500-1599 - HTTP 5xx errors
• 1600-1699 - HTTP/2 specific errors

For a list of all current error codes, see the docs page here.

Internals

• Improvements in the integration with loadimpact.com. (#910 and #934)
• Most of the HTTP request code has been refactored out of the js packages and is now independent from the goja JS runtime. This was done mostly so we can implement the error codes feature (#907), but will allow us more flexibility in the future. (#928)
• As a preparation for the upcoming big refactoring of how VUs are scheduled in k6, including the arrival-rate based execution, we've added the future execution configuration framework. It currently doesn't do anything besides warn users that use execution option combinations that won't be supported in future k6 versions. See the Breaking Changes section in these release notes for more information. (#913)
• Switched to golangci-lint via golangci.com for code linting in this repo. The gometalinter check in CircleCI is still enabled as well, but it will be removed in the following few weeks. (#943)
• Switched to Go 1.12.1 for building and testing k6, removed official support for 1.10. (#944 and #966)

Bugs fixed!

• JS: Consistently report setup/teardown timeouts as such and switch the error message to be more expressive. (#890)
• JS: Correctly exit with non zero exit code when setup or teardown timeouts. (#892)
• Thresholds: When outputting metrics to loadimpact.com, fix the incorrect reporting of threshold statuses at the end of the test. (#894)

• UX: --quiet/-q doesn't hide the summary stats at the end of the test. When necessary, they can still be hidden via the explicit --no-summary flag. Thanks, @oboukili! (#937)

Breaking changes

None in this release, but in preparation for the next one, some execution option combinations will emit warnings, since they will no longer be supported in future k6 releases. Specifically, you won't be able to simultaneously run k6 with stages and duration set, or with iterations and stages, or with duration and iterations, or with all three. These VU schedulers (and much more, including arrival-rate based ones!) will still be supported in future k6 releases. They will just be independent from each other, unlike their current implementation where there's one scheduler with 3 different conflicting constraints.

v0.23.1

A minor release that fixes some of the issues in the v0.23.0 release.

Bugs fixed!

• Cloud: Fixed the interaction between the environment variable and JSON-based configuration, and the Load Impact specific env.loadimpact JS options. Now only the projectID, name and token fields will be populated (without overriding other fields) when executing scripts with k6 cloud, and taken into account when sending metrics to Load Impact Insights with k6 run -o cloud. (#848, #871, #872)
• JS: Fixed a Babel transformation issue that caused closing brackets to sometimes be commented out. (#853)
• JS: Fixed environment variable propagation when executing script bundles. (#853)
• HAR converter: Fixed a panic due to a missing nil check. (#861)
• Cloud: Limit the amount of samples that k6 sends in a single package to the ingest by splitting them up. (#860)
• Metrics: Fix the incorrect tracing of some corner case HTTP requests that resulted in negative or hugely positive metric values. (#862)

v0.23.0

v0.23.0 is here! 🎉

Hopefully this is the last intermediary release before v1.0.0. It is a bit light on new features, but it includes a lot of bug fixes and minor improvements! Also, the latest Docker tag will point to this release until we release the next stable one. Users wanting to use the bleeding edge k6 features can do that via the new master docker tag, which is pushed by every CI build of the git master branch.

Thanks to @sherrman, @ofauchon, @AndriiChuzhynov, @entone, @mariolopjr, and @tkbky for contributing to this release!

To see what's left for the v1.0.0 release, check out this milestone!

Also, have a look at our roadmap for what's up ahead, beyond the v1.0 release.

New Features!

New option: No Cookies Reset (#729)

A new option has been added that disables the default behavior of resetting the cookie jar after each VU iteration. If it's enabled, saved cookies will be persisted across VU iterations. For the moment there's no CLI flag for this option, instead it can only be set via the noCookiesReset key from the exported script options or via the K6_NO_COOKIES_RESET environment variable.

k6/http: New options to discard the response body or to specify its type (#742 and #749)

You can now specify what the type of an HTTP response's body should be with the new responseType request option. The possible values for it are text (the default), binary and none. The default text response type is backward-compatible, it doesn't change the current k6 behavior of returning the body attribute of the http/Response object as a string. It's well suited for working with web pages, text-based APIs and similar HTTP responses, but it can be unsuitable when dealing with binary files.

That's mostly because JavaScript strings are encoded with UTF-16 and converting binary data to it will frequently mangle some of the data. The new binary response type allows us to avoid that, it causes k6 to return the HTTP response's body as a byte array. This allows us to deal with the binary data without mangling it:

import http from 'k6/http';
import { sha256 } from 'k6/crypto';

export default function () {
    const expectedLogoHash = "fce7a09dde7c25b9822eca8438b7a5c397c2709e280e8e50f04d98bc8a66f4d9";

    let resp = http.get("http://test.loadimpact.com/images/logo.png", { responseType: "binary" });
    let logoHash = sha256(resp.body, "hex");
    if (logoHash !== expectedLogoHash) {
        throw new Error(`Expected logo hash to be ${expectedLogoHash} but it was ${logoHash}`);
    }
    http.post("https://httpbin.org/post", resp.body);
};

Saving HTTP response bodies is generally useful, especially when we need to use them (or parts of them) in subsequent requests. But in many cases it makes little to no sense to spend memory on saving the response body. For example, when requesting static website assets (JS, CSS, images etc.) or web pages without needed information, the actual file contents rarely matter when running load tests.

For cases like that, the value none for the responseType option allows k6 to discard incoming data on arrival, in order to save CPU cycles and prevent unnecessary copying of data. When enabled, the actual HTTP response body would be fully downloaded (so that the load test and all HTTP metrics for that request are still accurate), it just won't be saved in memory and passed on to the JavaScript runtime at all - the response.body attribute would be null:

import http from 'k6/http';
import { check } from "k6";

export default function () {
    const url = "http://test.loadimpact.com";
    let resp = http.get(url);
    let cssFile = resp.html().find("link[rel='stylesheet']").attr("href");

    check(http.get(`${url}/${cssFile}`, { responseType: "none" }), {
        "body was empty": (res) => res.body === null,
        "response code was 200": (res) => res.status == 200,
        "timings are present": (res) => res.timings.duration > 0,
    });
};

For convenience, there's also a new global config option that causes k6 to discard response bodies by default by switching the default responseType value to none. It can be enabled via the --discard-response-bodies CLI flag, the K6_DISCARD_RESPONSE_BODIES environment variable, or the discardResponseBodies script option:

import http from 'k6/http';
export let options = {
  discardResponseBodies: true,
};
export default function () {
  let response = http.get("http://test.loadimpact.com", { responseType: "text" });
  // ... do something with the response, but ignore the contents of static files:
  http.batch([
    "http://test.loadimpact.com/images/logo.png",
    "http://test.loadimpact.com/style.css"
  ]);
};

Thanks to @sherrman for reporting the binary handling issues that prompted the addition of the responseType option! And thanks to @ofauchon for implementing both of the discard response body options, of which the local per-request one was later transformed into the responseType=none value!

k6/http: The Response.json() method now supports selectors

The selectors are implemented with the gjson library and allow optimized lookups and basic filtering of JSON elements in HTTP responses, which could be especially useful in combination with k6 checks:

import http from "k6/http";
import { check } from "k6";

export default function () {
	let resp = http.get("https://api.spacexdata.com/v2/launches/");

	let currentYear = (new Date()).getFullYear();
	check(resp, {
		"falcon heavy": (r) => r.json("#[flight_number==55].rocket.second_stage.payloads.0.payload_id") === "Tesla Roadster",
		"no failure this year": (r) => r.json("#[launch_success==false]#.launch_year").every((y) => y < currentYear),
		"success ratio": (r) => r.json("#[launch_success==true]#").length > 10 * r.json("#[launch_success==false]#").length,
	});
}

Thanks to @AndriiChuzhynov for implementing this! (#766)

New option: disable the summary at the end of a test (#729)

A new option that disables the end-of-test summary has been added. That summary is often superfluous when k6 tests are run in a distributed execution mode, or when the generated metrics are piped to an external output like InfluxDB or Load Impact Insights. The option can be enabled with the --no-summary CLI flag or the K6_NO_SUMMARY environment variable. When both it and the and the --no-thresholds option are enabled, k6 won't store any generated metrics in-memory, making the test execution a bit more efficient.

New option: set a minimum iteration duration (#821)

You can now specify the minimum amount of time a single iteration should take via the new minIterationDuration option. It's also configurable via the --min-iteration-duration CLI flag and K6_MIN_ITERATION_DURATION environment variable. This setting only applies for full iterations, so any interrupted iterations due to ramping down of VUs from a stage or at the end of the tests can still be shorter.

UX

• Added a warning when the maximum number of VUs is more than the total number of iterations (#802)

Internals

• Cloud output: improved outlier metric detection for small batches. (#744)
• Use 20 as the the default values of the batch and batchPerHost options. They determine the maximum number of parallel requests (in total and per-host respectively) an http.batch() call will make per VU. The previous value for batch was 10 and for batchPerHost it was 0 (unlimited). We now also use their values to determine the maximum number of open idle connections in a VU. (#685)
• Due to refactoring needed for the redirect fixes, the NTLM authentication library k6 uses is changed from this to this. (#753)
• Switched the default CircleCI tests and linting to use Go 1.11.2, but we still maintain 1.10 compatibility by running all of the tests with Go 1.10.3 too. Official k6 standalone builds will also be done with Go 1.11+ from now on. (#813)
• Automated docker builds of the git master branch will now tag the resulting docker image as master as well. The latest docker tag will point to the latest stable official release, so it will be equivalent to v0.23.0 until we release the next k6 version. (#846)

Bugs fixed!

• UI: The interactive k6 login influxdb command failed to write the supplied options to the config file. (#734)
• UI: Password input is now masked in k6 login influxdb and k6 login cloud. (#734)
• Config: Environment variables can now be used to modify k6's behavior in the k6 login subcommands. (#734)
• HTTP: Binary response bodies were mangled because there was no way to avoid converting them to UTF-16 JavaScript strings. (#749)
• Config: Stages were appended instead of overwritten from upper config "tiers", and were doubled when supplied via the CLI flag. (#759)
• HAR converter: Fixed a panic due to a missing array length check. (#760)
• HTTP: http.batch() calls could panic because of a data race when the batchPerHost global option was used. (#770)
• Docker: Fixed the grafana image in the docker-compose setup. Thanks @entone and @mariolopjr! (#783)
• Config: Stages configured via the script options or environment variables couldn't be disabled via the CLI flags. (#786)
• UI: Don't report infinities and extreme speeds when tests take 0 time. Thanks @tkbky! (#790)
• HTTP: Correct metric tracking when HTTP requests are redirected. (#753)
• HAR converter: Added escaping for page IDs and names in the generated scripts. (#801)
• Setup data: Distinguish between undefined (when there is no setup() function...

v0.22.1

A minor release that adds some UX improvements and fixes some of the issues in the v0.22.0 release.

UX

• The consolidated user-supplied options (the final combination of config file options, exported script options, environment variables and command-line flags) are now exported back into the options script variable and can be accessed from the script. Thanks to @MohanPrasathS for working on this! (#681 and #713)
• Improved error messages when outputting results to or executing tests in the Load Impact cloud (#716)

Bugs fixed!

• Logging: using the --no-color flag caused k6 to print output intended for sdtout to stderr instead. (#712)
• Logging: some error messages originating from Go's standard library did not obey the --logformat option. (#712)
• JSON output: when the standard output was used, the JSON collector closed it before k6 was finished printing the end-of-test summary to it (#715)
• Metrics: some zero-filled metrics were emitted when scaling down the number of VUs (#710)

v0.22.0

v0.22.0 is here! 🎉

We're making an intermediary release before v1.0.0, as we wanted to get some changes out quicker. Thanks to @MohanPrasathS for contributing to this release!

To see what's left for the v1.0.0 release, check out this milestone!

Also, have a look at our roadmap for what's up ahead, beyond the v1.0 release.

New Features!

• New JS API to set seed for PRNG. Now, you are able to set a seed to get reproducible (pseudo-)random numbers. (#677)

import {randomSeed} from "k6";

randomSeed(123456789);
let rnd = Math.random();
console.log(rnd)

• A new option --no-vu-connection-reuse lets users close HTTP keep-alive connections between iterations of a VU. (#676)
• You can now set the minimum and maximum sleep time at the end of an iteration with the new --min-sleep and --max-sleep HAR coverter CLI flags. (#694)
• Another new feature in the HAR converter enabled users to specify a JSON file with script options that would be added to the options of the generated scripts with the new --options flag. (#694)

UX

Automated deb, rpm, msi and nuget package builds (#675)

Previously we only had Homebrew releases for Mac and simple archives with plain binary releases for all other platforms. From now on, we'll also automatically build installation packages for Windows and rpm or deb based Linux distributions and upload them to bintray on every new release: https://bintray.com/loadimpact

For Debian-based Linux distributions, you have to do something like this to install k6:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 379CE192D401AB61
echo "deb https://dl.bintray.com/loadimpact/deb stable main" | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get install k6

And for rpm-based ones like Fedora and CentOS:

wget https://bintray.com/loadimpact/rpm/rpm -O bintray-loadimpact-rpm.repo
sudo mv bintray-loadimpact-rpm.repo /etc/yum.repos.d/
sudo yum install k6

For Windows you can download and install the latest .msi package or, if you use the chocolatey package manager, follow these instructions to set up the k6 repository.

Other UX improvements

• There's a new option to reset the saved cloud token: k6 login cloud --reset (#672)
• The check and group names in the summary at the end of a test now appear in the order they were defined. Thanks to @MohanPrasathS for fixing this! (#674)

Internals

Real-time metrics (#678)

Previously most metrics were emitted only when a script iteration ended. With these changes, metrics would be continuously pushed in real-time, even in the middle of a script iteration. This should slightly decrease memory usage and help a lot with the aggregation efficiency of the cloud collector.

Portable builds (#658)

Before this, k6 builds that were done with just the standard Go language tools (i.e. go get, go build, etc.) were not portable because static resources like JS libraries had to be embedded in the binary after the build. Building fully portable binaries was done with the build-release.sh script (which used go.rice to bundle the static resources in the binary), but now that embedding is done beforehand and is commited in the git repo, so commands like go get/build/install produce fully-portable binary files without extra steps.

Bugs fixed!

• Metrics emitted by setup() and teardown() are not discarded anymore. They are emitted and have the implicit root group tag values of setup and teardown respectively (#678)
• Fixed a potential nil pointer error when the k6 cloud command is interrupted. (#682)

Breaking Changes

• The --no-connection-reuse option has been re-purposed and now disables keep-alive connections globally. The newly added --no-vu-connection-reuse option does what was previously done by --no-connection-reuse - it closes any open connections between iterations of a VU, but allows for reusing them inside of a single iteration. (#676)

v0.21.1

A bug fix release to fix some issues in the v0.21.0 release.

Bugs fixed!

• Multiple bugs in the HTTP request and batch handling (#642)
• Fix negative http_req_sending values in the summary (#651)
• Respect the setup() and teardown() timeouts even when triggered by the API (#661)

v0.21.0

v0.21.0 is here! 🎉

We're happy to see continued contributions from members of the community in this release, from 4 people outside of Load Impact this time around. A big thanks to the following people for contributing to this release: @antekresic, @cyberw, @danron and @jmccann. Also, thanks to everyone that contributed in other ways on Github, in Slack and for spreading the word about k6!

To see the current plan for the next release, check out this milestone, which we aim to make the v1.0 release!

Have a look at our roadmap for what's up ahead, beyond the v1.0 release.

New Features!

CLI/Options: Add --tag flag and tags option to set test-wide tags (#553)

You can now specify any number of tags on the command line using the --tag NAME=VALUE flag. You can also use the tags option to the set tags in the code.

The specified tags will be applied across all metrics. However if you have set a tag with the same name on a request, check or custom metric in the code that tag value will have precedence.

Thanks to @antekresic for their work on this!

Docs: Test wide tags and Options

CLI/Options: Add --summary-time-unit flag (#638)

You can now specify the time unit used to show the summary trend stats. It can be: 's' for seconds, 'ms' for milliseconds or 'us' microseconds.

$ k6 run --summary-time-unit ms ~/script.js

Docs: Options

k6/http: Support for HTTP NTLM Authentication (#556)

import http from "k6/http";
import { check } from "k6";

export default function() {
    // Passing username and password as part of URL plus the auth option will authenticate using HTTP Digest authentication
    let res = http.get("http://user:passwd@example.com/path", {auth: "ntlm"});

    // Verify response
    check(res, {
        "status is 200": (r) => r.status === 200
    });
}

Docs: HTTP Params

HAR converter: Add support for correlating JSON values (#516)

There is now support for correlating JSON values in recordings, replacing recorded request values with references to the previous response.

Thanks to @cyberw for their work on this!

InfluxDB output: Add support for sending certain sample tags as fields (#585)

Since InfluxDB indexes tags, highly variable information like vu, iter or even url may lead to high memory usage. The InfluxDB documentation recommends to use fields in that case, which is what k6 does now. There is a new INFLUXDB_TAGS_AS_FIELDS option (collectors.influxdb.tagsAsFields in the global k6 JSON config) that specifies which of the tags k6 emits will be sent as fields to InfluxDB. By default that's only url (but not name), vu and iter (if enabled).

Thanks to @danron for their work on this!

Configurable setup and teardown timeouts (#602)

Previously the setup() and teardown() functions timed out after 10 seconds. Now that period is configurable via the setupTimeout and teardownTimeout script options or the K6_SETUP_TIMEOUT and K6_TEARDOWN_TIMEOUT environment variables. The default timeouts are still 10 seconds and at this time there are no CLI options for changing them to avoid clutter.

In-client aggregation for metrics streamed to the cloud (#600)

Metrics streamed to the Load Impact cloud can be partially aggregated to reduce bandwidth usage and processing times. Outlier metrics are automatically detected and excluded from that aggregation.

Docs: Load Impact Insights Aggregation

HAR converter: Set more realistic sleep time (#608)

The default sleep time added at the end of the generated test has been changed from 2-4s to 20-40s to be more realistic (although still probably on the low end for some types of sites [1][2]).

[1] - https://moz.com/blog/ecommerce-benchmark-kpi-study-2017
[2] - https://www.brafton.com/blog/strategy/brafton-2017-content-marketing-benchmark-report/

Remote IP address as an optional system metric tag (#616)

It's now possible to add the remote server's IP address to the tags for HTTP and WebSocket metrics. The ip system tag is not included by default, but it could easily be enabled by modifying the systemTags option.

Raw log format (#634)

There is a new log format called raw. When used, it will print only the log message without adding any debug information like, date or the log level. It should be useful for debuging scripts when printing a HTML response for example.

$ k6 run --log-format raw ~/script.js

Option to output metrics to Apache Kafka (#617)

There is now support for outputing metrics to Apache Kafka! You can configure a Kafka broker (or multiple ones), topic and message format directly from the command line like this:

k6 --out kafka=brokers={broker1,broker2},topic=k6,format=json

The default format is json, but you can also use the InfluxDB line protocol for direct ingestion by InfluxDB:

k6 --out kafka=brokers=my_broker_host,topic=k6metrics,format=influxdb

You can even specify format options such as the tagsAsFields option for InfluxDB:

k6 --out kafka=brokers=someBroker,topic=someTopic,format=influxdb,influxdb.tagsAsFields={url,name,myCustomTag}

Docs: Apache Kafka output

Thanks to @jmccann for their work on this!

Multiple outputs (#624)

It's now possible to simultaneously send the emitted metrics to several outputs by using the CLI --out flag multiple times, for example:
k6 run --out json=test.json --out influxdb=http://localhost:8086/k6

Thanks to @jmccann for their work on this!

Cloud execution: CLI flag to exit k6 when test reaches running state (#632)

There's now a new CLI flag --exit-on-running when running cloud tests (k6 cloud ...) to have k6 exit when the test reaches the running state.

UX

• Clearer error message when using open function outside init context (#563)
• Better error message when a script or module can't be found (#565). Thanks to @antekresic for their work on this!

Internals

• Removed all httpbin.org usage in tests, now a local transient HTTP server is used instead (#555). Thanks to @mccutchen for the great go-httpbin library!
• Fixed various data races and enabled automated testing with -race (#564)

Bugs

• Archive: archives generated on Windows can now run on *nix and vice versa. (#566)
• Submetrics are being tagged properly now. (#609)
• HTML: fixed the Selection.each(fn) function, which was returning only the first element. (#610)
• Invalid Stages option won't keep k6 running indefinitely. (#615)
• the --no-color option is now being repected for the logs. (#634)

Breaking changes

• The Load Impact cloud configuration options no_compress and project_id and the payload_size InfluxDB option have been renamed to noCompress, projectID and payloadSize respectively, to match the other JS option names.