Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename metrics with a browser prefix #903

Merged
merged 10 commits into from
May 24, 2023
Merged

Conversation

ankur22
Copy link
Collaborator

@ankur22 ankur22 commented May 23, 2023

Description of changes

k6 emits http metrics and other metrics (builtin metrics) which are useful for the user to work out how their API under test is behaving. The browser module currently works with these same metrics.

While working on an issue to identify why failed metrics requests weren't being accounted for, it became apparent that there was a mismatch in the tags that were being attached to the metrics between k6 and browser. This could affect the output summary and results in user's dashboards.

This also brought to light that these metrics heavily depend on the http implementation, and so the http metrics that the browser module emits will skew the results of the http metrics that k6 emits.

The first step in mitigating such issues is to easily distinguish between the two metrics. Custom metrics (which are clones of the builtin http metrics from k6) have been added and prefixed with browser_. This also follows the best practices of prometheus metric naming.

Closes: #883

Difference in Summary Results

+ browser_data_received..............: 27 kB  28 kB/s
+ browser_data_sent..................: 1.1 kB 1.1 kB/s
+ browser_http_req_connecting........: avg=79.66ms  min=0s       med=0s       max=239ms    p(90)=191.2ms  p(95)=215.09ms
+ browser_http_req_duration..........: avg=237.38ms min=112.65ms med=223.28ms max=376.22ms p(90)=345.63ms p(95)=360.92ms
+ browser_http_req_receiving.........: avg=111.33ms min=110ms    med=110ms    max=114ms    p(90)=113.2ms  p(95)=113.6ms 
+ browser_http_req_sending...........: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
+ browser_http_req_tls_handshaking...: avg=43.33ms  min=0s       med=0s       max=130ms    p(90)=104ms    p(95)=116.99ms
+ browser_http_reqs..................: 3      3.159238/s
browser_web_vital_fcp.........: avg=504.09ms min=504.09ms med=504.09ms max=504.09ms p(90)=504.09ms p(95)=504.09ms
browser_web_vital_fcp_good....: 1      1.109633/s
browser_web_vital_ttfb........: avg=347.09ms min=347.09ms med=347.09ms max=347.09ms p(90)=347.09ms p(95)=347.09ms
browser_web_vital_ttfb_good...: 1      1.109633/s
+ data_received......................: 0 B    0 B/s
+ data_sent..........................: 0 B    0 B/s
- data_received.................: 27 kB  30 kB/s
- data_sent.....................: 1.1 kB 1.2 kB/s
- http_req_connecting...........: avg=70.33ms  min=0s       med=0s       max=211ms    p(90)=168.8ms  p(95)=189.89ms
- http_req_duration.............: avg=208.79ms min=92.41ms  med=185.53ms max=348.44ms p(90)=315.86ms p(95)=332.15ms
- http_req_receiving............: avg=93.66ms  min=89ms     med=92ms     max=100ms    p(90)=98.4ms   p(95)=99.2ms  
- http_req_sending..............: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
- http_req_tls_handshaking......: avg=41ms     min=0s       med=0s       max=123ms    p(90)=98.4ms   p(95)=110.69ms
- http_reqs.....................: 3      3.328899/s
iteration_duration............: avg=901.83ms min=901.83ms med=901.83ms max=901.83ms p(90)=901.83ms p(95)=901.83ms
iterations....................: 1      1.109633/s

NOTE: data_received and data_sent are still be printed, even though the browser module is no longer interacting with these metrics. This is a known issue and will be looked into with k6.

Testing

You should be able to run any of the example tests.

Optional tasks

Tasks

Pass the browser custom metrics to the network manager. The network
manager will use these metrics instead of the built in ones. This will
enable us to differentiate between k6 and browser metrics.

We could just retrieve the customMetrics from the ctx too, but this
feels a bit more explicit which is easier to understand.
@ankur22 ankur22 marked this pull request as draft May 23, 2023 10:15
Move from working with the builtin data_sent metric to the
browser_data_sent metric. This will help split the browser module
metric from the k6 metric for more fine grained observations of the
data_sent metric.
@ankur22 ankur22 force-pushed the rename/883-browser-prefix-metrics branch from e861722 to a02a7ae Compare May 23, 2023 10:18
This commit helps us move away from the k6 http_reqs metric to a
browser prefixed one. It is custom to the browser module allowing the
user to measure k6 vs browser http_reqs metrics separately.
This commit helps us move away from the k6 http_req_duration metric to
a browser prefixed one. It is custom to the browser module allowing the
user to measure k6 vs browser http_req_duration metrics separately.
@ankur22 ankur22 force-pushed the rename/883-browser-prefix-metrics branch from c723e34 to 78e4249 Compare May 23, 2023 11:06
This commit helps us move away from the k6 data_received metric to
a browser prefixed one. It is custom to the browser module allowing the
user to measure k6 vs browser data_received metrics separately.
This commit helps us move away from the k6 http_req_connecting metric
to a browser prefixed one. It is custom to the browser module allowing
the user to measure k6 vs browser http_req_connecting metrics
separately.
This commit helps us move away from the k6 http_req_tls_handshaking
metric to a browser prefixed one. It is custom to the browser module
allowing the user to measure k6 vs browser http_req_tls_handshaking
metrics separately.
This commit helps us move away from the k6 http_req_sending metric to
a browser prefixed one. It is custom to the browser module allowing the
user to measure k6 vs browser http_req_sending metrics separately.
@ankur22 ankur22 force-pushed the rename/883-browser-prefix-metrics branch from af47d14 to bdb85c1 Compare May 23, 2023 14:34
This commit helps us move away from the k6 http_req_receiving metric to
a browser prefixed one. It is custom to the browser module allowing the
user to measure k6 vs browser http_req_receiving metrics separately.
@ankur22 ankur22 marked this pull request as ready for review May 23, 2023 15:40
Copy link
Collaborator

@ka3de ka3de left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes more sense!
LGTM.

Copy link
Member

@inancgumus inancgumus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👏

@ankur22 ankur22 merged commit 3a32e0a into main May 24, 2023
@ankur22 ankur22 deleted the rename/883-browser-prefix-metrics branch May 24, 2023 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add a prefix to the browser module metrics
4 participants