Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POTEL 21 - Drop OpenTelemetry spans for internal Sentry requests #3508

Merged
merged 79 commits into from
Jun 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
305baf5
replace hub with scopes
adinauer Mar 27, 2024
95f5e1b
Add Scopes
adinauer Apr 2, 2024
27f2398
Introduce `IScopes` interface.
adinauer Apr 2, 2024
ce3c14f
Replace `IHub` with `IScopes` in core
adinauer Apr 2, 2024
ce615f4
Replace `IHub` with `IScopes` in android core
adinauer Apr 2, 2024
22ddc00
Replace `IHub` with `IScopes` in android integrations
adinauer Apr 2, 2024
305c217
Replace `IHub` with `IScopes` in apollo integrations
adinauer Apr 2, 2024
da927bc
Replace `IHub` with `IScopes` in okhttp integration
adinauer Apr 2, 2024
8279276
Replace `IHub` with `IScopes` in graphql integration
adinauer Apr 2, 2024
9bfc086
Replace `IHub` with `IScopes` in logging integrations
adinauer Apr 2, 2024
b998e50
Replace `IHub` with `IScopes` in more integrations
adinauer Apr 2, 2024
739827a
Replace `IHub` with `IScopes` in OTel integration
adinauer Apr 2, 2024
69f2d63
Replace `IHub` with `IScopes` in Spring 5 / Spring Boot 2 integrations
adinauer Apr 2, 2024
792d482
Replace `IHub` with `IScopes` in Spring 6 / Spring Boot 3 integrations
adinauer Apr 2, 2024
9bcbce6
Replace `IHub` with `IScopes` in samples
adinauer Apr 2, 2024
3f25a4b
Merge branch 'feat/hsm-13-replacements-in-samples' into feat/hubs-sco…
adinauer Apr 2, 2024
d6fb40a
gitscopes -> github
adinauer Apr 2, 2024
7752bcc
Replace ThreadLocal with ScopesStorage
adinauer Apr 4, 2024
1e329c5
Move client and throwable to span map to scope
adinauer Apr 4, 2024
b0d89ae
Add global scope
adinauer Apr 4, 2024
cdd414a
use global scope in Scopes
adinauer Apr 4, 2024
98da9ff
Implement pushScope popScope and withScope for Scopes
adinauer Apr 4, 2024
2d26033
Add pushIsolationScope; add fork methods to ISCope
adinauer Apr 12, 2024
bbb6700
Use separate scopes for current, isolation and global scope; rename m…
adinauer Apr 12, 2024
c714b21
Allow controlling which scope configureScope uses
adinauer Apr 12, 2024
a474402
Combine scopes
adinauer Apr 12, 2024
ae93e33
Use new API for CRONS integrations
adinauer Apr 12, 2024
b01298b
Add lifecycle helper
adinauer Apr 12, 2024
b64e688
Change spring integrations to use new API
adinauer Apr 12, 2024
d06fc50
Use new API in servlet integrations
adinauer Apr 12, 2024
f0af5c3
Use new API for kotlin coroutines and wrapers for Supplier/Callable
adinauer Apr 12, 2024
2f02001
Discussion TODOs
adinauer Apr 12, 2024
bf4a7bf
Fix breadcrumb ordering
adinauer Apr 15, 2024
62cb91a
Mark TODOS with [HSM]
adinauer Apr 15, 2024
b1630ea
Add getGlobalScope and forkedRootScopes to IScopes
adinauer Apr 16, 2024
136b9ce
Fix EventProcessor ordering on scopes
adinauer Apr 16, 2024
94d54ef
Reuse code in Scopes
adinauer Apr 16, 2024
017599d
No longer replace global scope
adinauer Apr 16, 2024
f4c2b3c
Replace hub occurrences in comments, var names etc.
adinauer Apr 16, 2024
61c9d4a
Implement ScopesTest
adinauer Apr 18, 2024
04f3892
Implement CombinedScopeViewTest
adinauer Apr 18, 2024
840c194
Fix combined contexts
adinauer Apr 19, 2024
ab1c3a6
Use combined scopes for cross platform
adinauer Apr 19, 2024
23506c5
Changes according to reviews of previous PRs
adinauer Apr 23, 2024
c9b6f8b
more
adinauer Apr 23, 2024
696a809
even more
adinauer Apr 23, 2024
847200d
isEnabled checks client instead of having a property on Scopes
adinauer Apr 24, 2024
8e86d3b
Use SentryOptions.empty
adinauer Apr 25, 2024
06db228
Remove Hub
adinauer Apr 25, 2024
37ab4d0
Use OpenTelemetry for Performance and Scopes propagation
adinauer Apr 30, 2024
faef2f8
Promote certain span attributes
adinauer May 2, 2024
c57b2d3
Merge branch '8.x.x' into feat/potel-1-context-forking-and-basics
adinauer May 3, 2024
5b15128
Merge branch '8.x.x' into feat/potel-1-context-forking-and-basics
adinauer May 7, 2024
9d48538
Merge branch 'feat/potel-1-context-forking-and-basics' into feat/pote…
adinauer May 7, 2024
9b900c7
Use OTel in Sentry API
adinauer May 13, 2024
b3919bd
Deduplicate SpanInfo extraction
adinauer May 15, 2024
36ed84a
Forward Sentry API to Sentry through OTel
adinauer May 22, 2024
8f56e3f
Use OTel status for Sentry span API
adinauer May 23, 2024
81982bc
POTel Tracing
adinauer May 28, 2024
fba451c
fix root span detection (remote flag), and scope closing
adinauer May 29, 2024
c787e91
Inherit OTel span IDs when sending to sentry
adinauer May 31, 2024
8c25d81
Fix tracing; parse incoming baggage; add baggage to outgoing
adinauer Jun 3, 2024
ffe1bef
Cleanup
adinauer Jun 5, 2024
a31fc08
Move sampling logic to OTel Sampler
adinauer Jun 6, 2024
789ae1e
Remove internal span attributes so they are not sent to Sentry
adinauer Jun 6, 2024
f0c8cf2
Use transaction name
adinauer Jun 6, 2024
0d84b57
remove obsolete comment
adinauer Jun 6, 2024
ce1f729
Keep OTel and Sentry span name/op in sync
adinauer Jun 10, 2024
720b07c
more cleanup
adinauer Jun 10, 2024
20f8a86
Make it possible to ignore span origins
adinauer Jun 13, 2024
2134179
Reuse TracesSampler instance
adinauer Jun 13, 2024
5ae32b7
Catch exceptions thrown by integration.close
adinauer Jun 17, 2024
6ddb833
Set NoOpSpanFactory as property default
adinauer Jun 17, 2024
507ca74
Use correct SentryOptions for SentryClient ctor
adinauer Jun 17, 2024
f5caf82
PR review feedback
adinauer Jun 17, 2024
ce75759
more
adinauer Jun 17, 2024
f3f75a2
Use SpanOptions.startTimestamp in Span ctor
adinauer Jun 19, 2024
0c7b917
Drop internal Sentry spans in SentrySampler
adinauer Jun 24, 2024
0c0c76c
Merge branch '8.x.x' into feat/potel-21-drop-internal-sentry-spans
adinauer Jun 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ public final class io/sentry/opentelemetry/OpenTelemetryLinkErrorEventProcessor
public fun process (Lio/sentry/SentryEvent;Lio/sentry/Hint;)Lio/sentry/SentryEvent;
}

public final class io/sentry/opentelemetry/OtelInternalSpanDetectionUtil {
public fun <init> ()V
public static fun isSentryRequest (Lio/sentry/IScopes;Lio/opentelemetry/api/trace/SpanKind;Lio/opentelemetry/api/common/Attributes;)Z
}

public final class io/sentry/opentelemetry/OtelSamplingUtil {
public fun <init> ()V
public static fun extractSamplingDecision (Lio/opentelemetry/api/common/Attributes;)Lio/sentry/TracesSamplingDecision;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
package io.sentry.opentelemetry;

import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.semconv.SemanticAttributes;
import io.sentry.DsnUtil;
import io.sentry.IScopes;
import java.util.Arrays;
import java.util.List;
import java.util.Locale;
import org.jetbrains.annotations.ApiStatus;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;

@ApiStatus.Internal
public final class OtelInternalSpanDetectionUtil {

private static final @NotNull List<SpanKind> spanKindsConsideredForSentryRequests =
Arrays.asList(SpanKind.CLIENT, SpanKind.INTERNAL);

@SuppressWarnings("deprecation")
public static boolean isSentryRequest(
final @NotNull IScopes scopes,
final @NotNull SpanKind spanKind,
final @NotNull Attributes attributes) {
if (!spanKindsConsideredForSentryRequests.contains(spanKind)) {
return false;
}

final @Nullable String httpUrl = attributes.get(SemanticAttributes.HTTP_URL);
if (DsnUtil.urlContainsDsnHost(scopes.getOptions(), httpUrl)) {
return true;
}

final @Nullable String fullUrl = attributes.get(SemanticAttributes.URL_FULL);
if (DsnUtil.urlContainsDsnHost(scopes.getOptions(), fullUrl)) {
return true;
}

// TODO [POTEL] should check if enabled but multi init with different options makes testing hard
// atm
// if (scopes.getOptions().isEnableSpotlight()) {
final @Nullable String optionsSpotlightUrl = scopes.getOptions().getSpotlightConnectionUrl();
final @NotNull String spotlightUrl =
optionsSpotlightUrl != null ? optionsSpotlightUrl : "http://localhost:8969/stream";

if (containsSpotlightUrl(fullUrl, spotlightUrl)) {
return true;
}
if (containsSpotlightUrl(httpUrl, spotlightUrl)) {
return true;
}
// }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leftover that can be deleted

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we reintroduce the if above, then it's needed again. We can retest after doing the alpha release.


return false;
}

private static boolean containsSpotlightUrl(
final @Nullable String requestUrl, final @NotNull String spotlightUrl) {
if (requestUrl == null) {
return false;
}

return requestUrl.toLowerCase(Locale.ROOT).contains(spotlightUrl.toLowerCase(Locale.ROOT));
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package io.sentry.opentelemetry;

import static io.sentry.opentelemetry.OtelInternalSpanDetectionUtil.isSentryRequest;

import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.SpanKind;
Expand All @@ -13,7 +15,6 @@
import io.sentry.PropagationContext;
import io.sentry.SamplingContext;
import io.sentry.ScopesAdapter;
import io.sentry.SentryOptions;
import io.sentry.SentryTraceHeader;
import io.sentry.SpanId;
import io.sentry.TracesSamplingDecision;
Expand All @@ -26,10 +27,10 @@
public final class SentrySampler implements Sampler {

private final @NotNull SentryWeakSpanStorage spanStorage = SentryWeakSpanStorage.getInstance();
private final @NotNull SentryOptions options;
private final @NotNull IScopes scopes;

public SentrySampler(final @NotNull IScopes scopes) {
this.options = scopes.getOptions();
this.scopes = scopes;
}

public SentrySampler() {
Expand All @@ -44,7 +45,9 @@ public SamplingResult shouldSample(
final @NotNull SpanKind spanKind,
final @NotNull Attributes attributes,
final @NotNull List<LinkData> parentLinks) {
// TODO [POTEL] use SamplingDecision.DROP sentry internal spans
if (isSentryRequest(scopes, spanKind, attributes)) {
return SamplingResult.drop();
}
// note: parentLinks seems to usually be empty
final @Nullable Span parentOtelSpan = Span.fromContextOrNull(parentContext);
final @Nullable OtelSpanWrapper parentSentrySpan =
Expand All @@ -65,7 +68,7 @@ public SamplingResult shouldSample(

private @NotNull SamplingResult handleRootOtelSpan(
final @NotNull String traceId, final @NotNull Context parentContext) {
if (!options.isTraceSampling()) {
if (!scopes.getOptions().isTraceSampling()) {
// TODO [POTEL] should this return RECORD_ONLY to allow tracing without performance
return SamplingResult.create(SamplingDecision.DROP);
}
Expand All @@ -87,7 +90,10 @@ public SamplingResult shouldSample(
final @NotNull TransactionContext transactionContext =
TransactionContext.fromPropagationContext(propagationContext);
final @NotNull TracesSamplingDecision sentryDecision =
options.getInternalTracesSampler().sample(new SamplingContext(transactionContext, null));
scopes
.getOptions()
.getInternalTracesSampler()
.sample(new SamplingContext(transactionContext, null));
return new SentrySamplingResult(sentryDecision);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

import static io.sentry.TransactionContext.DEFAULT_TRANSACTION_NAME;
import static io.sentry.opentelemetry.InternalSemanticAttributes.IS_REMOTE_PARENT;
import static io.sentry.opentelemetry.OtelInternalSpanDetectionUtil.isSentryRequest;

import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.StatusCode;
import io.opentelemetry.sdk.common.CompletableResultCode;
import io.opentelemetry.sdk.trace.data.SpanData;
Expand All @@ -14,7 +14,6 @@
import io.sentry.Baggage;
import io.sentry.DateUtils;
import io.sentry.DefaultSpanFactory;
import io.sentry.DsnUtil;
import io.sentry.IScopes;
import io.sentry.ISpan;
import io.sentry.ITransaction;
Expand All @@ -37,7 +36,6 @@
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.function.Predicate;
Expand All @@ -54,9 +52,6 @@ public final class SentrySpanExporter implements SpanExporter {
new SpanDescriptionExtractor();
private final @NotNull IScopes scopes;

private final @NotNull List<SpanKind> spanKindsConsideredForSentryRequests =
Arrays.asList(SpanKind.CLIENT, SpanKind.INTERNAL);

private final @NotNull List<String> attributeKeysToRemove =
Arrays.asList(
InternalSemanticAttributes.IS_REMOTE_PARENT.getKey(),
Expand Down Expand Up @@ -134,51 +129,9 @@ private boolean isSpanTooOld(final @NotNull SpanData span, final @NotNull Sentry
}

private @NotNull List<SpanData> filterOutSentrySpans(final @NotNull Collection<SpanData> spans) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there instances where SentrySampler is not invoked and spans need to be filtered here in the exporter?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Theoretically it's possible the span is modified during runtime and then has different attributes in SpanExporter.

return spans.stream().filter((span) -> !isSentryRequest(span)).collect(Collectors.toList());
}

@SuppressWarnings("deprecation")
private boolean isSentryRequest(final @NotNull SpanData spanData) {
final @NotNull SpanKind kind = spanData.getKind();
if (!spanKindsConsideredForSentryRequests.contains(kind)) {
return false;
}

final @Nullable String httpUrl = spanData.getAttributes().get(SemanticAttributes.HTTP_URL);
if (DsnUtil.urlContainsDsnHost(scopes.getOptions(), httpUrl)) {
return true;
}

final @Nullable String fullUrl = spanData.getAttributes().get(SemanticAttributes.URL_FULL);
if (DsnUtil.urlContainsDsnHost(scopes.getOptions(), fullUrl)) {
return true;
}

// TODO [POTEL] should check if enabled but multi init with different options makes testing hard
// atm
// if (scopes.getOptions().isEnableSpotlight()) {
final @Nullable String optionsSpotlightUrl = scopes.getOptions().getSpotlightConnectionUrl();
final @NotNull String spotlightUrl =
optionsSpotlightUrl != null ? optionsSpotlightUrl : "http://localhost:8969/stream";

if (containsSpotlightUrl(fullUrl, spotlightUrl)) {
return true;
}
if (containsSpotlightUrl(httpUrl, spotlightUrl)) {
return true;
}
// }

return false;
}

private boolean containsSpotlightUrl(
final @Nullable String requestUrl, final @NotNull String spotlightUrl) {
if (requestUrl == null) {
return false;
}

return requestUrl.toLowerCase(Locale.ROOT).contains(spotlightUrl.toLowerCase(Locale.ROOT));
return spans.stream()
.filter((span) -> !isSentryRequest(scopes, span.getKind(), span.getAttributes()))
.collect(Collectors.toList());
}

private List<SpanData> maybeSend(final @NotNull List<SpanData> spans) {
Expand Down
Loading