Link OpenTelemetry traces to parent span when publishing test results #414
Conversation
|
cc @cyrille-leclerc / @kuisathaverat is this a sensible approach to make this work properly? |
timja
changed the title
timja
changed the title
| EnvVars envs = new EnvVars(); | ||
| envs.put(OTelEnvironmentVariablesConventions.TRACE_ID, traceId); | ||
| envs.put(OTelEnvironmentVariablesConventions.SPAN_ID, spanId); | ||
| envs.put("TRACEPARENT", parentTraceId); |
There was a problem hiding this comment.
NIT: If the export environment variables is enabled in the environment variables are defined twice, aren't they?
There was a problem hiding this comment.
There'll be nothing interesting in the environment here as it's on the agent. Exporting them doesn't get them passed to the agent down the remoting channel as far as I can tell
src/main/java/io/jenkins/plugins/junit/storage/database/otel/OtelInformation.java
Show resolved
Hide resolved
|
FYI we are simplifying the usage of OpenTelemetry in plugins improving the APIs of the Jenkins OpenTelemetry Plugin: I would like to experiment this integration on a plugin to be sure we define the right APIs |
| SpanBuilder spanBuilder = tracer.spanBuilder(spanName); | ||
|
|
||
| if (otelInformation != null) { | ||
| spanBuilder = spanBuilder.setParent(otelInformation.setup()) |
There was a problem hiding this comment.
@timja I suspect there is a problem in the way the Jenkins OTel Plugin propagates the context if your span lacks a parent. Can we verify the solution doesn't reside in the Jenkins OTel Plugin before rolling out this solution?
There was a problem hiding this comment.
Yes no rush to roll it out, can wait
There was a problem hiding this comment.
Problem understood, context propagation is not done by the Jenkins otel plugin to step executions of other plugins.
I'll try to fix it in the Jenkins otel plugin
Please see my comment above, there may be a problem in the Jenkins OTel Plugin |
|
FYI I have a change in progress in the Jenkins OTel Plugin to properly propagate the trace context to the Junit SQL Storage plugin. I the evolution of the Jenkisn OTel Plugin is successful, then this PR in the Junit SQL Storage Plugin will not be needed. Here is an example that also includes the instrumentation of the JDBC datasource |
|
This PR is no longer needed. The problem is now solved in the Jenkins OTel Plugin by: |
|
On the latest version of the open telemetry plugin I can't seem to get the sql batch insert to show up at all not on master or this branch. Are you running on the controller or a separate agent? Can you share more your configuration please? |
|
Great ctch, I cleaned up my test environments and I still get the trace context propagation successful with the following: Jenkins plugins version
|
|
@cyrille-leclerc how are you testing it? do you have a remote agent? I just tried with this setup: (Inbound agent secret will need replacing) version: '3.1'
services:
db:
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: postgres
ports:
- 5432:5432
jaeger:
image: jaegertracing/all-in-one:1.57
ports:
- "4317:4317"
- "16686:16686"
jenkins:
build:
context: .
ports:
- "8080:8080"
- "50000:50000"
- "5005:5005"
environment:
OTEL_EXPORTER_OTLP_ENDPOINT: http://jaeger:4317
OTEL_SERVICE_NAME: jenkins
OTEL_METRICS_EXPORTER: none
OTEL_LOGS_EXPORTER: none
OTEL_JAVAAGENT_LOGGING: simple
JAVA_OPTS: "-Djenkins.install.runSetupWizard=false -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005"
jenkins-agent:
image: jenkins/inbound-agent
environment:
JENKINS_URL: http://jenkins:8080
JENKINS_SECRET: <replace-this>
JENKINS_AGENT_NAME: agent
JENKINS_AGENT_WORKDIR: /home/jenkins/agentI was on the master branch of this repository and ran it with: (note: I've made a couple of changes to the docker-compose files in the repo to test this fully) and I don't get the sql trace: 
|
|
(you can use the master docker-compose if you add these vars:) Since it'll be fixed in the next version I wasn't going to bother checking it in for now although I lost some time trying to figure out what was happening |
|
Sorry for the delay. Problem reproduced. |
|
Please test the latest Jenkins OpenTelemetry Plugin with the configuration properties: 
Ignore the "INSERT my_shopping_cart.caseResults" and "PoolingDataSource.getConnection" that are an experiment of instrumenting the Jenkins Database Plugin, it's not released yet. |
Requires jenkinsci/junit-plugin#622
Amends #413 to set it up so spans have the correct parent so publishing the test results is tracked properly
Testing done
Freestyle:
Tested on a remote agent for Pipeline:
Submitter checklist