Cosmos config to control historical swing-store artifact retention #9386
Assignees
Labels
agoric-cosmos
cosmic-swingset
package: cosmic-swingset
enhancement
New feature or request
good first issue
Good for newcomers
Comments
mhofman
added
enhancement
This was referenced May 20, 2024
1 task
mergify bot
added a commit
that referenced
this issue
Sep 6, 2024
Ref #9174 Fixes #9387 Fixes #9386 TODO: - [ ] #9389 ## Description Adds consensus-independent `vat-snapshot-retention` ("debug" vs. "operational") and `vat-transcript-retention` ("archival" vs. "operational" vs. "default") cosmos-sdk swingset configuration (values chosen to correspond with [`artifactMode`](https://github.com/Agoric/agoric-sdk/blob/master/packages/swing-store/docs/data-export.md#optional--historical-data)) for propagation in AG_COSMOS_INIT. The former defaults to "operational" and the latter defaults to "default", which infers a value from cosmos-sdk `pruning` to allow simple configuration of archiving nodes. It also updates the semantics of TranscriptStore `keepTranscripts: false` configuration to remove items from only the previously-current span rather than from all previous spans when rolling over (to avoid expensive database churn). Removal of older items can be accomplished by reloading from an export that does not include them. ### Security Considerations I don't think this changes any relevant security posture. ### Scaling Considerations This will reduce the SQLite disk usage for any node that is not explicitly configured to retain snapshots and/or transcripts. The latter in particular is expected to have significant benefits for mainnet (as noted in #9174, about 116 GB ÷ 147 GB ≈ 79% of the database on 2024-03-29 was vat transcript items). ### Documentation Considerations The new fields are documented in our default TOML template, and captured in a JSDoc type on the JavaScript side. ### Testing Considerations This PR extends coverage TranscriptStore to include `keepTranscripts` true vs. false, but I don't see a good way to cover Go→JS propagation other than manually (which I have done). It should be possible to add testing for the use and validation of `resolvedConfig` in AG_COSMOS_INIT handling, but IMO that is best saved for after completion of split-brain (to avoid issues with same-process Go–JS entanglement). ### Upgrade Considerations This is all kernel code that can be used at any node restart (i.e., because the configuration is consensus-independent, it doesn't even need to wait for a chain software upgrade). But we should mention the new cosmos-sdk configuration in release notes, because it won't be added to existing app.toml files already in use.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What is the Problem Being Solved?
Swing-store currently defaults to a "replay" artifact level, which keeps transcripts of the current incarnation and the latest heap snapshot of all vats. We want to make this configurable for validators, and default to a value derived from the type of node (archive or normal)
Description of the Design
New
app.tomlconfig:For transcripts, the "default" (which is also the default value if not configured) resolution would depend on other values in the config. If the node is configured for
pruning = "nothing"(archival node), it would resolve to "nothing", otherwise it would resolve to prune "everything" (do not keep historical transcripts). If not configured the value for heap-snapshot pruning would be "everything".The config must be plumbed from golang into JS cosmic-swingset, and passed when initializing the swing-store.
This assumes that #9388 is implemented as without that change, creating a state-sync snapshot would fail if pruning everything was configured.
Security Considerations
None
Scaling Considerations
None for this option
Test Plan
TBD
Upgrade Considerations
Cosmos SDK does not have a mechanism to add new options to existing config files. We would need to document and communicate these new settings to validators.
The text was updated successfully, but these errors were encountered: