Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tracing instrumentation #3445

Merged
merged 16 commits into from
Nov 16, 2023
Merged

Add tracing instrumentation #3445

merged 16 commits into from
Nov 16, 2023

Conversation

ka3de
Copy link
Contributor

@ka3de ka3de commented Nov 8, 2023

What?

This PR adds a base tracing instrumentation around opentelemetry-go library. Specifically initializes a TracerProvider based on a new traces ouput configuration option (--traces-output / K6_TRACES_OUTPUT). The TracerProvider is exposed partially to modules through the VU.State() so these can request Tracers initialization that can be used to generate traces.

Why?

There has been various requests to add more tracing functionality to k6, both from k6 users and from extension developers. This PR represents a first step towards promoting tracing as a first class citizen in k6.

Checklist

  • I have performed a self-review of my code.
  • I have added tests for my changes.
  • I have run linter locally (make lint) and all checks pass.
  • I have run tests locally (make tests) and all tests pass.
  • I have commented on my code, particularly in hard-to-understand areas.

@ka3de ka3de added the feature label Nov 8, 2023
@ka3de ka3de self-assigned this Nov 8, 2023
js/common/trace.go Outdated Show resolved Hide resolved
js/modules/modules.go Outdated Show resolved Hide resolved
@mstoykov
Copy link
Contributor

mstoykov commented Nov 8, 2023

a quick feedback: the first two commits should be one commit. Not vendoring the dependencies breaks the build and no commit should break the build if possible.

@mstoykov
Copy link
Contributor

mstoykov commented Nov 8, 2023

I would also recommend to pull the parseutils change in a separate PR so we can review and merge it faster and without all the noise of this one.

You can rebase this one after that one is merged - which hopefully will be really fast.

@codecov-commenter
Copy link

codecov-commenter commented Nov 8, 2023

Codecov Report

Attention: 110 lines in your changes are missing coverage. Please review.

Comparison is base (5e83d38) 73.35% compared to head (f085709) 73.16%.
Report is 4 commits behind head on master.

❗ Current head f085709 differs from pull request most recent head 663c6c2. Consider uploading reports for the commit 663c6c2 to get more accurate results

Files Patch % Lines
lib/trace/otel.go 52.00% 46 Missing and 2 partials ⚠️
js/modules/k6/experimental/fs/errors_gen.go 9.09% 19 Missing and 1 partial ⚠️
js/modules/k6/experimental/fs/module.go 84.25% 11 Missing and 9 partials ⚠️
cmd/run.go 68.96% 7 Missing and 2 partials ⚠️
js/modules/k6/experimental/fs/cache.go 55.55% 4 Missing and 4 partials ⚠️
loader/loader.go 78.94% 2 Missing and 2 partials ⚠️
js/modules/k6/experimental/fs/errors.go 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3445      +/-   ##
==========================================
- Coverage   73.35%   73.16%   -0.20%     
==========================================
  Files         261      265       +4     
  Lines       19778    20083     +305     
==========================================
+ Hits        14509    14694     +185     
- Misses       4372     4471      +99     
- Partials      897      918      +21     
Flag Coverage Δ
ubuntu 73.16% <70.66%> (-0.14%) ⬇️
windows ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ka3de
Copy link
Contributor Author

ka3de commented Nov 8, 2023

a quick feedback: the first two commits should be one commit. Not vendoring the dependencies breaks the build and no commit should break the build if possible.

I would also recommend to pull the parseutils change in a separate PR

I will address these two points. Treat this PR as a draft currently.

@oleiade
Copy link
Member

oleiade commented Nov 8, 2023

@ka3de could you also fill the PR checklist accordingly so as a reviewer, I have an idea of where we're at? 🙇🏻

cmd/root.go Outdated Show resolved Hide resolved
@ka3de
Copy link
Contributor Author

ka3de commented Nov 9, 2023

Once #3446 is merged, I will force push rebasing the changes along the modifications discussed in #3445 (comment), and then set the PR up for review.

This package provides APIs in order to interact with Open Telemetry
tracing instrumentation.
This flag can be used to configure the traces output by setting the
destination endpoint, protocol, extra headers, etc.
Initializes the global state's TracerProvider based on traces output
cmdline flag.
Waits for tracer provider to stop within 5 seconds, otherwise an error
message is logged. Because both methods, stopLoggers() and
stopTracerProvider(), are bounded by a timeout, execute them
concurrently and wait for them to finish.
@ka3de ka3de marked this pull request as ready for review November 10, 2023 11:28
cmd/root.go Outdated Show resolved Hide resolved
lib/trace/otel.go Outdated Show resolved Hide resolved
@mstoykov mstoykov added this to the v0.48.0 milestone Nov 13, 2023
Because traces will only be generated during a test run, move the tracer
provider initialization and stop to the run command.
Additionally, for this same reason, there is no need to wait for the
tracer provider to shutdown unless the actual test run is executed,
therefore wait for it concurrently with metrics processing wait.
Avoid a default initialization for TracerProvider in GlobalState and
instead delegate that to the run command, if executed.
@ka3de ka3de mentioned this pull request Nov 14, 2023
3 tasks
cmd/root.go Outdated Show resolved Hide resolved
Copy link
Contributor

@mstoykov mstoykov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general - just move the flag configuration to the runtime options.

I have not really looked at the tracing parts - I expect @Blinkuu will be way more useful there.

I would also prefer that with/after the merge from k6-browser where this is used to get an integration test that mocks a tracing endpoint and checks that we get something useful.

Copy link
Contributor

@Blinkuu Blinkuu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Blinkuu
Blinkuu previously approved these changes Nov 16, 2023
mstoykov
mstoykov previously approved these changes Nov 16, 2023
Copy link
Contributor

@mstoykov mstoykov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

left a small nitpick

cmd/run.go Outdated
return nil
}

tp, err := trace.TracerProviderFromConfigLine(c.gs.Ctx, ro.TracesOutput.String)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: you can provide the globalcCtx as is done in the cancelation timeout

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, that makes total sense. Fixed in 663c6c2.

@mstoykov
Copy link
Contributor

Also please either squash the whole thing on merge or make it into 2(?) commits instead of 15

@ka3de ka3de dismissed stale reviews from mstoykov and Blinkuu via 663c6c2 November 16, 2023 13:53
@ka3de
Copy link
Contributor Author

ka3de commented Nov 16, 2023

Also please either squash the whole thing on merge or make it into 2(?) commits instead of 15

I don't have permissions to merge. I would say squash at this point if that works for you. Otherwise let me know and I'll try to organize it in a few commits that make sense.

@mstoykov mstoykov merged commit 7a30976 into master Nov 16, 2023
21 of 22 checks passed
@mstoykov mstoykov deleted the feat/tracing branch November 16, 2023 14:28
@olegbespalov olegbespalov added the documentation-needed A PR which will need a separate PR for documentation label Nov 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation-needed A PR which will need a separate PR for documentation feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants