Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stacking of telemetry layers is not possible (tracing-distributed) #18

Open
Infrasonics opened this issue Feb 1, 2022 · 1 comment
Open

Comments

@Infrasonics
Copy link
Contributor

It is currently not possible to register more than one TelemetryLayer with tracing-subscriber. The problem is the inclusion of the timestamp SpanInitAt, since it leads to two spans with the same value which is not allowed by tracing-subscriber and causes a panic at runtime.

Side note: the visitor, as holder of the field values, is included as well, so this leads to the same problem when using the same visitor with different layers, yet is easier to work around.

My use case is to use separate backends for different kinds of traces.

Dropping the timestamp resolves it, yet it might lead to a different behavior for users of tracing-honeycomb. I made a simple sketch in #17 (WIP) so once it works, someone familiar with what Honeycomb expects could check whether things still work properly.

Please let me know what you think about the situation in general.

@Infrasonics
Copy link
Contributor Author

I've been playing around a bit more and with my limited tracing knowledge I don't see an easy way to add the span initialization time and the duration in another place without changing the interface. For extra information and options to discuss in case, here is what I found so far:

  1. Putting the timestamp information into another layer.
    From the documentation of tracing-subscriber::registry::ExtensionMut:

/// Note that extensions are not
/// Layer-specific—they are span-specific. This means that
/// other layers can access and mutate extensions that
/// a different Layer recorded. For example, an application might
/// have a layer that records execution timings, alongside a layer
/// that reports spans and events to a distributed
/// tracing system that requires timestamps for spans.
/// Ideally, if one layer records a timestamp x, the other layer
/// should be able to reuse timestamp x.

  1. Adding the time information to the values of the span instead of its extensions.
  2. Putting this kind of change behind a feature where either compatibility with honeycomb or stacking is possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant