Enhancement Proposal: Aggregate Endpoint for Efficient Certificate Management in Large Fleets

[jmcrawford45]

I would like to propose an enhancement to the ARI extension that could be beneficial for users managing large fleets of certificates. Specifically, I suggest optionally allowing Certificate Authorities to return all information matching certain constraints under a given account key.

Background:

We manage a large fleet of a few hundred thousand certificates. Currently, the process involves making individual requests for each certificate, which can be resource-intensive and stateful.

Proposed Enhancement:

Introduce an optional feature that allows CAs to provide an aggregate endpoint where accounts can retrieve all relevant information matching specified constraints under a given account key. For example, an account may be interested in exceptional cases with a small upcoming window outside of the expected value or that include a specific explanation url.
This feature would help reduce the load on both the client and server by minimizing the number of requests needed for renewing large fleets under a single account, particularly in response to mass revocation events.
It would also simplify client-side operations by reducing the amount of state that needs to be maintained.

Benefits:

Improved efficiency for managing large numbers of certificates.
Reduced server load and network traffic.
Simplified client-side implementation and maintenance.
I believe this feature could be valuable for other users with similar needs like #77. Thank you for considering this proposal.

[mholt]

I would like this too, but it has already been discussed and rejected: #41

Again, the spec authors seem to prefer simpler server implementations and more complex client implementations, as opposed to more of a middle ground that balances complexity.

[jmcrawford45]

Thank you for sharing the prior context. @aarongable would the page-able per-account endpoint still be an option under consideration? I think this would still provide most of the benefits I mentioned here (reducing RPS and allowing clients to keep less state). We'd still need to apply the filters on constraints client side, but perhaps this approach strikes a better balance for large fleets than the current get with id endpoint.

[webprofusion-chrisc]

I'd suggest that the main point of interest for anyone is that their account has exceptional ARI status they should be aware of, so they can then go off and poll ARI status per cert (or get a download for their whole account). The main objection I'm seeing so far from client authors is that they don't feel that polling for every cert every day scales particularly well.

[aarongable]

Hi, thanks for the feature request! As said above, this idea has been discussed and rejected earlier in the draft's lifecycle. I'm not opposed to the idea as a general concept, but I have yet to see a specific proposal that is convincingly both efficient and simple.

At this point we're in Working Group Last Call, so a proposal for a new feature like this needs to be much more precise and fleshed out. For example:

• What URL is this bulk request made to?
• What set of HTTP methods (POST, GET, POST-as-GET) are acceptable, and how does their behavior differ, if at all?
• What sort of constraints can be specified, and how are those included in the request?
• What does the response look like? In particular, how does it associate renewal windows with individual certificates?
• What happens if the number of entries in the response is too high? (In particular, if you are proposing pagination, what is the pagination or cursoring scheme?)
• Can you point to a client or server implementation of this proposal?

I think the simplest possible proposal for a bulk endpoint would be "POST to the directory's renewalInfo endpoint with a JSON list of CertIDs; the server rejects the request if the list is too long or if any CertID doesn't belong to the requesting account, and otherwise responds with a JSON object mapping each requested CertID to its suggested renewal window". It's not clear to me if this would satisfy the desires of large client operators.