Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal API for snapshot processing status per epoch #40

Closed
anomit opened this issue Aug 20, 2023 · 0 comments · Fixed by #41
Closed

Internal API for snapshot processing status per epoch #40

anomit opened this issue Aug 20, 2023 · 0 comments · Fixed by #41
Assignees
Labels
enhancement New feature or request

Comments

@anomit
Copy link
Member

anomit commented Aug 20, 2023

Is your feature request related to a problem?
Beginning from the release of an epoch and submission of a snapshot against it, to its finalization on Powerloom protocol, it goes through a sequence of state transitions as detailed below. Its status, at the moment, is non-trivial to diagnose from the available APIs exposed by the snapshotter implementations.

  • EPOCH_RELEASED – epoch is released from the protocol state smart contract for snapshotters to detect and begin work
  • PRELOAD – preloaders are executed for snapshot building workers to extract data according to the snapshotter specific modeuls (uniswapv2 in case of Pooler)
  • SNAPSHOT_BUILD – the snapshot builders as configured in projects.json are executed. Also refer to the case study of the current implementation of Pooler for a detailed look at snapshot building for base as well as aggregates.
  • SNAPSHOT_SUBMIT_PAYLOAD_COMMIT - once a snapshot is built, it is propagated to the payload commit service in Audit Protocol for further submission to the protocol state contract.
  • RELAYER_SEND - Payload commit service has sent the snapshot to a transaction relayer to submit to THE protocol state contract

* SNAPSHOT_SUBMIT_PROTOCOL_CONTRACT - The snapshot submission transaction from the relayer to the protocol state smart contract was successful and a SnapshotSubmitted event was generated

At the moment, SnapshotSubmitted event delivered to processor distributor is a locally mocked event from payload commit service in Audit Protocol. This makes SNAPSHOT_SUBMIT_PROTOCOL_CONTRACT actually not be connected to a state transition corresponding to a protocol state contract event emission, and hence is omitted for this release. A separate state transition may be tracked in the future for such purpose.

  • SNAPSHOT_FINALIZE - Upon reaching consensus, the finalized snapshot accepted against an epoch is published via a SnapshotFinalized event.

Describe the solution you'd like
Expose an API endpoint on the lines of /internal/snapshotter/epochProcessingStatus which will return the status of processing of snasphots against configured project types at every state transition. The response set should be paginated.

{
    "items": [
      {
        "epochId": 43523,
        "transitionStatus": {
          "EPOCH_RELEASED": {
            "status": "success",
            "error": null,
            "extra": null,
            "timestamp": 1692530595
          },
          "PRELOAD": {
            "pairContract_pair_total_reserves": {
              "status": "success",
              "error": null,
              "extra": null,
              "timestamp": 1692530595
            },
          },
          "SNAPSHOT_BUILD": {
            "aggregate_24h_stats_lite:35ee1886fa4665255a0d0486c6079c4719c82f0f62ef9e96a98f26fde2e8a106:UNISWAPV2": {
              "status": "success",
              "error": null,
              "extra": null,
              "timestamp": 1692530596
            },
          },
          "SNAPSHOT_SUBMIT_PAYLOAD_COMMIT": {

          },
         "RELAYER_SEND": {

         },
        "SNAPSHOT_FINALIZE": {

        },
      },
    }
   ],
   "total": 30,
   "page": 1,
   "size": 10,
   "pages": 3
}

Describe alternatives you've considered
The other alternatives to an aggregated API are cumbersome and deal with parsing and connecting several transient datapoints maintained in the Redis cache.

Additional context
NA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant