Application insights duplicate logs coming up at different intervals

[aditvavikramx]

What version of .NET does your existing project use?

.NET 8

What version of .NET are you attempting to target?

.NET 8

Description

I am experiencing an issue where my logs are getting printed twice on Application Insights. Below are the details of my setup and configuration:
Logging Setup in Startup :

services.AddLogging(loggingBuilder =>
{
    loggingBuilder.SetMinimumLevel(logLevel);
    loggingBuilder.AddFilter<ApplicationInsightsLoggerProvider>("mykafkamodel", logLevel);
    loggingBuilder.AddApplicationInsights(loggingBuilder =>
    {
        loggingBuilder.IncludeScopes = true;
    });
});
services.AddApplicationInsightsTelemetryWorkerService(options: aiServiceOptions);
services.ConfigureFunctionsApplicationInsights();

Application Insights Configuration
By default, Application Insights is set to log warnings, so I reset the rules at bootup:

.ConfigureLogging(logging =>
{
    logLevel = LoadConfigurations.ApplicationInsightsLogLevel(config!);
logging.Services.Configure((Action<LoggerFilterOptions>)(options =>
    {
        var loggerFilterRule = options.Rules.FirstOrDefault(rule => rule.ProviderName == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider");
        if (loggerFilterRule is not null)
        {
            options.Rules.Remove(loggerFilterRule);
        }
        logging.AddFilter<ApplicationInsightsLoggerProvider>("mykafkamodel", logLevel);
    }));
});

Application Insights Service Options
var aiServiceOptions = new ApplicationInsightsServiceOptions { EnableAdaptiveSampling = false, EnableDebugLogger = false, };

Issue

Despite the above configuration, I observe that some logs are being duplicated in the trace logs. I suspect there might be an issue with my logging setup or configuration.

Request
Could someone please help identify if there is something wrong with my configuration or suggest a solution to prevent duplicate logs?

Project configuration and dependencies

I have upgraded my inprocess model to worker model dot net isolated ie dot net 8

Link to a repository that reproduces the issue

No response