Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configuring "Exception" as ExcludedTypes in adaptive sampling still results in ExceptionTelemetry being sampled #2916

Open
nielsruys opened this issue Nov 14, 2024 · 2 comments
Labels

Comments

@nielsruys
Copy link

nielsruys commented Nov 14, 2024

Microsoft.ApplicationInsights.WorkerService: 2.22.0 (same issue with Microsoft.ApplicationInsights.AspNetCore: 2.22.0)

  • Runtime version (e.g. net461, net48, netcoreapp2.1, netcoreapp3.1, etc. You can find this information from the *.csproj file):

net8.0

  • Hosting environment (e.g. Azure Web App, App Service on Linux, Windows, Ubuntu, etc.):

AKS (but reproduceable with a console-app on your local machine)

Describe the bug

Configuring "Exception" as ExcludedTypes in adaptive sampling still results in ExceptionTelemetry being sampled.

To Reproduce

  1. Unzip the project in attachment, see AdaptiveSamplingIssue.zip
  2. Open the project and update the connection-string to your application insights instance (Program.cs)
  3. Run the console app for 5 min on your local machine and wait up until the application shuts down
  4. Query your application insights instance using the below kusto queries:
  • Query to check if all exceptions are retained:
exceptions
| where timestamp > datetime("<startDateTime>")
| summarize RetainedPercentage = 100/avg(itemCount) by bin(timestamp, 1d), itemType

The RetainedPercentage is < 100%, so exceptions are not retained. See Image

  • Query to check the number of exceptions:
exceptions
| where timestamp > datetime("<startDateTime>")
| order by timestamp asc

We expect around 3000 exceptions, but the actual number is lower, so not all exceptions are retained. See Image

Workaround

You might consider implementing and registering a telemetry initializer to retain exceptions while awaiting a fix.

public class ExceptionTelemetryInitializer : ITelemetryInitializer
{
    public void Initialize(ITelemetry telemetry)
    {
        if (telemetry is not ExceptionTelemetry exceptionTelemetry) return;

        ISupportSampling supportSampling = exceptionTelemetry;
        supportSampling.SamplingPercentage = 100;
    }
}
@nielsruys nielsruys added the bug label Nov 14, 2024
@nielsruys nielsruys changed the title Configuring "Exception" as an ExcludedTypes in adaptive sampling still results in ExceptionTelemetry being sampled Configuring "Exception" as ExcludedTypes in adaptive sampling still results in ExceptionTelemetry being sampled Nov 14, 2024
@leendertdommicent
Copy link

leendertdommicent commented Nov 14, 2024

I think your sampling configuration settings are incorrect. I found this documentation page: https://learn.microsoft.com/en-us/azure/azure-monitor/app/sampling-classic-api#configure-sampling-settings

You are missing the build() statement on the telemetryProcessorChainBuilder. You should also set EnableAdaptiveSampling = false in the AddApplicationInsightsTelemetry call.

I adapted your program.cs

builder.ConfigureServices((_, services) =>
{
    services.AddHostedService<SomeBackGroundService>()
        .Configure<TelemetryConfiguration>(config =>
        {
            config.DefaultTelemetrySink.TelemetryProcessorChainBuilder
                .UseAdaptiveSampling(excludedTypes: "Exception").Build();
        }).AddApplicationInsightsTelemetryWorkerService(options =>
        {
            options.ConnectionString = "REDACTED";
            options.EnableAdaptiveSampling = false;
        });
});

I then also added a TrackDependency when TrackException is called:

telemetryClient.TrackException(new Exception("Some Exception"));
telemetryClient.TrackDependency(new DependencyTelemetry("db", "db", "db", "db"));

When I then run the sample, the exceptions are not sampled and the dependencies are.
Image
Image

@nielsruys
Copy link
Author

Thanks, this solves the issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants