remove Agora cost models

[Theodus]

Recently I’ve started thinking about the graph network more holistically for the first time in a while. And I think I’ve come to the conclusion that we should abandon Agora cost models for subgraph queries in favor of a simpler pricing system. I have always considered Agora cost models to be over-engineered for the problems that we face in practice, but I had previously believed that they served a role in a future state of the network. But my opinion on this has recently shifted.

This is the current state of Agora cost models from my perspective:

• Indexers don’t use them. And those that do require complex automation like auto-agora. The maintenance plan for auto-agora is currently unknown/stalled.
• We do not have a way to reliably associate GraphQL query shapes with the cost to execute a query.
• Agora does not provide a practical way for indexers to reject incoming queries by excluding patterns manually.
• Compiling cost models in the gateway is often costly. And will become a bigger performance issue if more indexers were to set them. I don’t think anyone can justify rewriting the Agora compiler to fix the performance issues I’ve identified.
• Complex cost models from indexers don’t provide any benefit for price efficiency if the gateway isn’t making pricing decisions based on query shapes. There are currently no plans to pursue this.
• Removing Agora would reduce the maintenance burden for gateway, indexer-service, and indexer-agent.
• DIPs agreements will facilitate indexers expressing their costs for service beyond just queries.

Proposal:

• Indexers set a static price per query, in GRT, for each (deployment, TAP sender), instead of a cost model. This facilitates a much simpler price discovery automation on the indexer side. And is practically equivalent to a cost model of default => ...;.
• The gateway isolates ISA state by (deployment, API key). This means that indexers gain the ability to reject potentially malicious queries from a user without it impacting selection decisions more globally.

Technical plan:

• Remove the /budget endpoint. No automated tool depends on it, and and future iteration of this idea should provide a value in GRT.
• Replace cost model request & compilation with a new interface provided by indexer-service. Ideally, the/cost GraphQL server would be replaced with a fee field in the deployment status from the /status server (entries without a fee set or a fee of 0 should be omitted).
  • As an intermediate step, the gateway will continue to use the /cost server as it currently exists. But it will only support cost models of the form default => x; where x is a numeric literal.
• Isolate indexer-selection state by (deployment, API key). These state isolates should be cached until they remain unused for some period of time (on the order of minutes). Queries made infrequently to the (deployment, API key) pair should fall back to a global indexer-selection state.
• Reduce selection limit from 3 to 2.

[Theodus]

At the time of writing, only one indexer with active allocations has a cost model set. The result of that single cost model could be accomplished with this proposal, without Agora: default => 0.00001;