Rate Limiting

[JanmanX]

Rate limiting on Hangfire.PostgreSql would be a great feature, and it would be extremely useful to me.

I understand the limited resources on this project, but I wonder what it would take to implement this?

I am not familiar with any Hangfire codebase, but given proper guidance, I'd be willing to try (helping) implement it.

Kind regards,
Jan

[azygis]

What exactly do you mean by "rate limiting"?

[JanmanX]

@azygis Sorry for not elaborating. I refer to the Concurrency and Rate Limiting features of the Hangfire.Throttling package, which does not seem to work with this storage implementation: https://docs.hangfire.io/en/latest/background-processing/throttling.html

Personally, I'd love to have a feature to limit number of concurrent jobs for a specific argument.

For instance, I want to have at most 10 concurrent jobs running for a specific user. Something like this would be extremely useful:

[Limit(user_id, concurrency=10)]
public void Process(Guid user_id, string data) { ... } 

[azygis]

Looks like the library is behind a paid NuGet repository which I personally won't buy. This can only be implemented by whoever has access to it.

Requirements
Supported only for Hangfire.SqlServer (better to use ≥ 1.7) and Hangfire.Pro.Redis (recommended ≥ 2.4.0) storages. Community storage support will be denoted later after defining correctness conditions for storages.

Not exactly sure what that means either. There is no technical documentation for community storage maintainers, as far as I can see.

[latop2604]

You can implement it with a client filter. I don't think it should be part of Hangfire.PostgreSql. But rather a dedicated package.

Here is a simplified implementation I use. It does not require any storage specifics feature. I hope it will help you.

public class MyCustomFilter(IServiceProvider _serviceProvider) : IClientFilter
{
    public void OnCreating(CreatingContext filterContext)
    {
        // JobStorage.Current can be used if you don't want IOC
        var jobStorage = _serviceProvider.GetRequiredService<JobStorage>();
        var monitoringApi = jobStorage.GetMonitoringApi();

        bool shouldCancel = /* custom logic */;

        if (shouldCancel)
        {
            filterContext.Canceled = true;
        }
    }

    public void OnCreated(CreatedContext filterContext) { }
}

// Setup
.AddHangfire((sc, config) => config.UseFilter(new DeDuplicateFilter(sc)));

[JanmanX]

@latop2604 That might work! Do you by any chance know how I could reschedule / delay the job by a few minutes, instead of cancelling it?

[latop2604]

Not sure but something like that could work

public void OnCreated(CreatedContext filterContext)
{
    var backgroundJobClient = _serviceProvider.GetRequiredService<IBackgroundJobClient>();
    var scheduledState = new ScheduledState(DateTime.UtcNow.AddDays(7));
    backgroundJobClient.ChangeState(filterContext.BackgroundJob.Id, scheduledState, null);
}

I tried to find a way to do it in OnCreating but no result. Everything is readonly. And the JobId is not yet available.