Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource allocation improvement #1463

Merged
merged 15 commits into from
Nov 9, 2020

Conversation

CodeBlanch
Copy link
Member

@CodeBlanch CodeBlanch commented Nov 4, 2020

Fixes #1397.

Changes

To avoid an allocation on every Activity (a result of storing Resource as a custom property) expose ParentProvider and a GetResource extension to Exporters.

Public API Changes

    public abstract class BaseProvider : IDisposable
    {
        public void Dispose()

        protected virtual void Dispose(bool disposing)
    }

    public class TracerProvider : BaseProvider

    public class MeterProvider : BaseProvider

    public abstract class BaseProcessor<T> : IDisposable
    {
        public BaseProvider ParentProvider { get; }
    }

    public abstract class BaseExporter<T> : IDisposable
    {
        public BaseProvider ParentProvider { get; }
    }

    public static Resource GetResource(this BaseProvider baseProvider)

TODOs:

@cijothomas
Copy link
Member

Thanks @CodeBlanch! This looks the best option so far. LGTM overall. Will mark approved when moved as non-draft.
We can wait till evening to give others a chance.

@codecov
Copy link

codecov bot commented Nov 5, 2020

Codecov Report

Merging #1463 (f4620a8) into master (052ecc4) will increase coverage by 0.05%.
The diff coverage is 81.39%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1463      +/-   ##
==========================================
+ Coverage   81.13%   81.19%   +0.05%     
==========================================
  Files         232      233       +1     
  Lines        6309     6311       +2     
==========================================
+ Hits         5119     5124       +5     
+ Misses       1190     1187       -3     
Impacted Files Coverage Δ
src/OpenTelemetry.Api/Metrics/MeterProvider.cs 100.00% <ø> (+20.00%) ⬆️
src/OpenTelemetry.Api/Trace/TracerProvider.cs 100.00% <ø> (ø)
...nTelemetry/Internal/OpenTelemetrySdkEventSource.cs 73.07% <ø> (ø)
...rc/OpenTelemetry.Exporter.Jaeger/JaegerExporter.cs 84.94% <33.33%> (ø)
...try.Exporter.OpenTelemetryProtocol/OtlpExporter.cs 58.53% <50.00%> (-0.44%) ⬇️
src/OpenTelemetry/ProviderExtensions.cs 66.66% <66.66%> (ø)
src/OpenTelemetry.Api/BaseProvider.cs 100.00% <100.00%> (ø)
...metryProtocol/Implementation/ActivityExtensions.cs 86.78% <100.00%> (-0.12%) ⬇️
...rc/OpenTelemetry.Exporter.Zipkin/ZipkinExporter.cs 86.73% <100.00%> (+1.02%) ⬆️
src/OpenTelemetry/BaseExportProcessor.cs 82.35% <100.00%> (+3.78%) ⬆️
... and 11 more

@reyang
Copy link
Member

reyang commented Nov 5, 2020

Stepping back question - do we expect that .NET would add Activity.Resource in the future?
If the answer is yes, we probably don't want to put a public API that we cannot remove later?

@CodeBlanch
Copy link
Member Author

Stepping back question - do we expect that .NET would add Activity.Resource in the future?
If the answer is yes, we probably don't want to put a public API that we cannot remove later?

🤷 IMO Resource is set by the user controlling the application and then used by the Exporter(s) they select. API doesn't know about it at all. So there's not a lot of value for .NET to own that and expose something. But I'm not privy to the planning being done 😄

@cijothomas
Copy link
Member

Since we are not confident of pursuing to make .NET support this natively, we need to do something to associate Resource with trace/metric/log.
Storing Resource with every telemetry item (as currently done) is not very efficient.
Alternate option is to provide a way for Processor/Exporter to grab it, via Provider. I think this PR is so far the best attempt.

We can explore:

  1. Introduce BaseProvider and have Meter/Tracer providers inherit from it. Since Resource is a common concern, BaseProvider can have Resource. This should reduce the ugliness of having SetTracerProvider/MeterProvider in a genric processor, and instead we'll have setprovider only.
  2. Instead of making SetTracerProvider() a method, can we make it a property (public get;internal set) in the baseprocessor and baseexporter, and have it set in the TracerProviderSdk. This way, all exporters and processors will always have the provider whenever it needs, without having to override any method. Simply access this.provider. This also reduces "ugliness" a bit?

build/Common.prod.props Outdated Show resolved Hide resolved
Copy link
Member

@reyang reyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@CodeBlanch CodeBlanch changed the title Resource improvement - Option 3 (Alternative) Resource allocation improvement Nov 8, 2020
@CodeBlanch CodeBlanch marked this pull request as ready for review November 8, 2020 01:46
@CodeBlanch CodeBlanch requested a review from a team November 8, 2020 01:46
@CodeBlanch
Copy link
Member Author

@reyang Would you please take another look at this? Removed the OnSetParentProvider virtuals. @cijothomas and I discussed that on gitter. It was slightly better for perf in the exporters to have those but we decided a cleaner API would add more value than the slight perf boost.

Copy link
Contributor

@eddynaka eddynaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice!!

@@ -20,17 +20,17 @@ jobs:
- name: Setup .NET Core 2.1
uses: actions/setup-dotnet@v1
with:
dotnet-version: 2.1.807
dotnet-version: 2.1.811
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was getting an error that 2.1.807 could not be found. Bumped to latest.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could change to 2.1.x and 3.1.x and remove the rsync script.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up getting a conflict on the .net5 PR so while I was in there merging I switched to the ".x" for 2.1 & 3.1 versions.

@@ -46,6 +46,13 @@ public sealed override void OnStart(T data)
/// <inheritdoc />
public abstract override void OnEnd(T data);

internal override void SetParentProvider(BaseProvider parentProvider)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cijothomas I needed a way to set the Exporter.ParentProvider when the Processor's gets set. Internal to SDK though.

@reyang
Copy link
Member

reyang commented Nov 9, 2020

@reyang Would you please take another look at this? Removed the OnSetParentProvider virtuals. @cijothomas and I discussed that on gitter. It was slightly better for perf in the exporters to have those but we decided a cleaner API would add more value than the slight perf boost.

LGTM. It reminded me of DOM APIs.
At this moment we don't know if there will be multiple providers, and if something like a CompositeProvider would solve the problem. But even if there is, this PR won't corner us from chaining/aggregating providers.

@cijothomas cijothomas merged commit b956a5f into open-telemetry:master Nov 9, 2020
@CodeBlanch CodeBlanch deleted the resource-option-c-alt branch November 9, 2020 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Avoid Activity.SetCustomPropery usage for Resource.
4 participants