Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use duration run,The http_reqs value of K6 is different from the service real request count value. #898

Closed
ltto opened this issue Jan 21, 2019 · 3 comments · Fixed by #1007
Assignees
Milestone

Comments

@ltto
Copy link

ltto commented Jan 21, 2019

k6 script file
image
java server code
image
k6 run result
image
java server result
image

service:221825 > k6:http_reqs..................: 221760

Sometimes it will appear k6>server

But there is no problem with specifying iterations
Is this a thread safety issue or a reality bug?
What can I do?

@na--
Copy link
Member

na-- commented Jan 21, 2019

This likely happens because when you use duration or stages to configure your script execution, k6 enforces a very strict cutoff time for any metrics generated after the specified duration expires - they are simply discarded. So I assume that in your case, k6 started making some requests, the specified duration was reached and the requests either didn't finish, or they finished after it. Thus k6 doesn't count them, while the remote service does.

We're currently working on refactoring big parts of the script execution and scheduling in k6. One of them would allow us to specify that script iterations (even when running the test with a duration) are uninterruptible: #879. In that case, metrics that are generated (including the number of HTTP requests) after the end of the specified duration, will still be processed and tracked. Leaving this issue open for now, will close it when we describe some of those refactoring changes in more details in other issues.

@na-- na-- mentioned this issue May 15, 2019
39 tasks
@na-- na-- added this to the v1.0.0 milestone Aug 27, 2019
@na-- na-- self-assigned this Aug 27, 2019
@priyaananthasankar
Copy link

I have a similar problem but this time the k6 reports more requests. I tried it with 1 user for 1 min and the results were accurate. Then I tried it for 1000 users running in a minute and I got k6 output at 3028 and the app I was testing against as 2993. Is there any other metric I can check?

@na--
Copy link
Member

na-- commented May 19, 2020

@priyaananthasankar, the issue you're having isn't the same one described above. It's possible that that in your case, some of the requests k6 is making aren't reaching the system you're testing, potentially because you're saturating your network/server/app/etc. Do you see a bunch of Request Failed warnings in your terminal?

To illustrate, if you run this k6 script:

import http from "k6/http";

export default function (data) {
    let res = http.get("https://some-non-existent-domain-asdfasdfg.org/");

    console.log(res.error_code);
    console.log(res.error);
    console.log(res.status);
}

You'd still get http_reqs: 1, since k6 attempted to make the request. In this case, it failed at DNS resolution, but normal HTTP requests can fail for a bunch of other reasons midway through, like TLS handshaking, up to their very end, like when the connection is broken before a large transfer is over. So, k6 still measures those, but there are ways to easily distinguish if a request was successful or not in your script. The script above would output something like this:

WARN[0000] Request Failed                                error="Get \"https://some-non-existent-domain-asdfasdfg.org/\": lookup some-non-existent-domain-asdfasdfg.org on 192.168.0.1:53: no such host"
INFO[0000] 1101                                         
INFO[0000] lookup: no such host                         
INFO[0000] 0 

So, in your case, you can use all of these properties of the Response object and k6 custom metrics (likely a Counter) to get the number of successful requests only:

import http from "k6/http";
import { Counter } from 'k6/metrics';

let successfulReqs = new Counter('http_successful_reqs');

function myget(url) {
    let res = http.get(url);
    if (res.status == 200) { // or whatever your definition of successful request is
        successfulReqs.add(1);
    }
}

export default function (data) {
    myget("https://some-non-existent-domain-asdfasdfg.org/");
    myget("https://test-api.k6.io/");
    myget("https://test.k6.io/");
}

the script above would get you

    http_reqs..................: 3      1.752296/s
    http_successful_reqs.......: 2      1.168197/s

@na-- na-- modified the milestones: v1.0.0, v0.27.0 May 21, 2020
@na-- na-- closed this as completed in #1007 Jul 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants