Missing EventHubs provider for Aspire / IConfiguration

[jsedlak]

When configuring EventHubs via Aspire for Streaming, the Orleans silo does not properly configure the streaming provider.

This issue is similar to #9075

The following error is received upon startup:

System.InvalidOperationException: 'Could not find Streaming provider named 'AzureEventHubs'. This can indicate that either the 'Microsoft.Orleans.Sdk' or the provider's package are not referenced by your application.'

According to what I can find, there is a set of missing Provider classes to support initializing the provider from configuration. Implementation would be similar to this PR for Storage Queue Streaming.

Pull Request: #8929

From the original support thread:

Packages:

<PackageReference Include="Aspire.Azure.Messaging.EventHubs" Version="8.2.2" />
<PackageReference Include="Aspire.Azure.Data.Tables" Version="8.2.2" />
<PackageReference Include="Aspire.Azure.Storage.Blobs" Version="8.2.2" />
<PackageReference Include="Microsoft.Orleans.Server" Version="8.2.0" />
<PackageReference Include="Microsoft.Orleans.Clustering.AzureStorage" Version="8.2.0" />
<PackageReference Include="Microsoft.Orleans.Persistence.AzureStorage" Version="8.2.0" />
<PackageReference Include="Microsoft.Orleans.Streaming.EventHubs" Version="8.2.0" />
<PackageReference Include="Microsoft.Orleans.Streaming" Version="8.2.0" />

AppHost

/* Adds Streaming Capabilities */
var eventHubs = builder.AddAzureEventHubs("reach-event-hubs")
    .AddEventHub("ReachEvents")
    .RunAsEmulator();

// ... add other resources

var orleans = builder.AddOrleans("reach-cluster")
    .WithClustering(cluster)
    .WithGrainStorage("Default", grainStorage)
    .WithGrainStorage("PubSubStore", grainStorage)
    .WithStreaming(eventHubs);

Silo

builder.AddKeyedAzureEventHubProducerClient("reach-event-hubs", static settings =>
{
    settings.EventHubName = "ReachEvents";
});
builder.AddKeyedAzureEventHubConsumerClient("reach-event-hubs", static settings =>
{
    settings.EventHubName = "ReachEvents";
});

I also tried manually configuring the EventHubs, but did not spend too much time on this (yet)

builder.UseOrleans(o =>
{
    o.AddEventHubStreams("StreamProvider", configurator =>
    {
        configurator.ConfigureEventHub(builder => builder.Configure(options =>
        {
            options.ConfigureEventHubConnection(
                ehConnectionString,
                "ReachEvents",
                "ReachEventsEditorGroup"
            );
        }));
    });

    o.UseDashboard(x => x.HostSelf = true);
});

[joshcornejo]

Look at, I had a similar issue #9241

[jsedlak]

Look at, I had a similar issue #9241

If you look into the source, you'll note there is a RedisClusteringProviderBuilder

https://github.com/dotnet/orleans/blob/main/src/Redis/Orleans.Clustering.Redis/Hosting/RedisClusteringProviderBuilder.cs

However, for EventHubs Streaming (and as linked Storage Queues Streaming) there is no matching provider.

https://github.com/dotnet/orleans/blob/main/src/Azure/Orleans.Streaming.EventHubs/Hosting/SiloBuilderExtensions.cs
Edit: this link is provided only to link to the parent folder, where the provider builder should exist

It is my understanding that the provider attributes allow these classes to initialize the correct provider by utlizing the IConfiguration service.

In your case, adding the package allows the provider to fire correctly, but with EventHubs there is no such provider. Please correct me if I am wrong!

[joshcornejo]

I did start by listing which packages I have - SiloBuilderExtensions.cs is a very different shape of class from the Redis Cluster.

[jsedlak]

I did start by listing which packages I have - SiloBuilderExtensions.cs is a very different shape of class from the Redis Cluster.

I've updated the original post with the details.

[jsedlak]

This issue also seems to be related
dotnet/aspire#5561

In adding support for EventHubs Streaming, an issue occurs where the Consumer Group is not added, because by default it may not be specified in the AppHost.

The provided workaround does address the issue, but Aspire likely needs to be updated to pass in Namespace, Entity and ConsumerGroup data.