Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

State sync support #5803

Closed
wants to merge 79 commits into from
Closed

State sync support #5803

wants to merge 79 commits into from

Commits on Mar 13, 2020

  1. Use local iavl module

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    e75f206 View commit details
    Browse the repository at this point in the history

  2. Added initial snapshot settings

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    c6201ba View commit details
    Browse the repository at this point in the history

  3. Initial functional snapshot/restore API

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    bd2af17 View commit details
    Browse the repository at this point in the history

  4. Added compression and chunking

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    2424a7d View commit details
    Browse the repository at this point in the history

  5. Code cleanups

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    383cf94 View commit details
    Browse the repository at this point in the history

  6. Added benchmarks

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    aabc422 View commit details
    Browse the repository at this point in the history

  7. More benchmarks

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    dcc7673 View commit details
    Browse the repository at this point in the history

  8. Buffer snapshot writers

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    780dae5 View commit details
    Browse the repository at this point in the history

  9. Minor tweaks

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    e91f273 View commit details
    Browse the repository at this point in the history

  10. Type fix

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    6bcf763 View commit details
    Browse the repository at this point in the history

  11. Ignore caches during export

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    ef46535 View commit details
    Browse the repository at this point in the history

  12. Use local tm-db as well

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    a939897 View commit details
    Browse the repository at this point in the history

  13. Initial snapshot store

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    4585f8e View commit details
    Browse the repository at this point in the history

  14. Simplified Snapshotter interface

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    88e5355 View commit details
    Browse the repository at this point in the history

  15. Split chunk writer and reader to separate file

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    6aed0b1 View commit details
    Browse the repository at this point in the history

  16. Cleaned up multistore snapshot/restore

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    d7a6dbe View commit details
    Browse the repository at this point in the history

  17. Improved snapshotting

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    672db15 View commit details
    Browse the repository at this point in the history

  18. Properly close exporters and importers

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    cd1b2d8 View commit details
    Browse the repository at this point in the history

  19. Added snapshot loading

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    11de242 View commit details
    Browse the repository at this point in the history

  20. Added snapshot pruning

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    11fb197 View commit details
    Browse the repository at this point in the history

  21. Added snapshot listing

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    ad61eb1 View commit details
    Browse the repository at this point in the history

  22. Use prefix db for snapshot store

    @erikgrinaker
    erikgrinaker committed Mar 13, 2020
    Configuration menu
    Copy the full SHA
    6eb2ffe View commit details
    Browse the repository at this point in the history

Commits on Mar 17, 2020

  1. Added auxiliary snapshot function for BaseApp

    @erikgrinaker
    erikgrinaker committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    dd76cb7 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into erik/snapshot

    @erikgrinaker
    erikgrinaker committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    bf8acad View commit details
    Browse the repository at this point in the history

  3. go.mod: remove local tm-db.

    @erikgrinaker
    erikgrinaker committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    a257ff3 View commit details
    Browse the repository at this point in the history

  4. Moved rootmulti snapshot contents to separate store/types

    @erikgrinaker
    erikgrinaker committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    4251e06 View commit details
    Browse the repository at this point in the history

  5. Moved snapshot store to separate package

    @erikgrinaker
    erikgrinaker committed Mar 17, 2020
    Configuration menu
    Copy the full SHA
    31f8eee View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2020

  1. Added format parameter for Snapshotter interface

    @erikgrinaker
    erikgrinaker committed Mar 18, 2020
    Configuration menu
    Copy the full SHA
    46a3941 View commit details
    Browse the repository at this point in the history

  2. Removed unused snapshotFormat variable

    @erikgrinaker
    erikgrinaker committed Mar 18, 2020
    Configuration menu
    Copy the full SHA
    c7a7513 View commit details
    Browse the repository at this point in the history

  3. Don't set up a snapshot store automatically

    @erikgrinaker
    erikgrinaker committed Mar 18, 2020
    Configuration menu
    Copy the full SHA
    bf54794 View commit details
    Browse the repository at this point in the history

  4. Added tests for snapshots.Store

    @erikgrinaker
    erikgrinaker committed Mar 18, 2020
    Configuration menu
    Copy the full SHA
    08713b1 View commit details
    Browse the repository at this point in the history

  5. Minor tweaks

    @erikgrinaker
    erikgrinaker committed Mar 18, 2020
    Configuration menu
    Copy the full SHA
    56fc9d9 View commit details
    Browse the repository at this point in the history

Commits on Mar 19, 2020

  1. go.mod: use iavl 0.13.2

    @erikgrinaker
    erikgrinaker committed Mar 19, 2020
    Configuration menu
    Copy the full SHA
    394e097 View commit details
    Browse the repository at this point in the history

  2. Updated changelog

    @erikgrinaker
    erikgrinaker committed Mar 19, 2020
    Configuration menu
    Copy the full SHA
    316dace View commit details
    Browse the repository at this point in the history

  3. Appease linter

    @erikgrinaker
    erikgrinaker committed Mar 19, 2020
    Configuration menu
    Copy the full SHA
    a39974f View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into erik/snapshot

    @erikgrinaker
    erikgrinaker committed Mar 19, 2020
    Configuration menu
    Copy the full SHA
    e984eee View commit details
    Browse the repository at this point in the history

  5. Added snapshot options

    @erikgrinaker
    erikgrinaker committed Mar 19, 2020
    Configuration menu
    Copy the full SHA
    d8d455d View commit details
    Browse the repository at this point in the history

  6. Fix nil dereferencing in chunkWriter.CloseWithError()

    @erikgrinaker
    erikgrinaker committed Mar 19, 2020
    Configuration menu
    Copy the full SHA
    bef3c86 View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2020

  1. Merge branch 'master' into erik/snapshot

    @erikgrinaker
    erikgrinaker committed Mar 26, 2020
    Configuration menu
    Copy the full SHA
    c3528dc View commit details
    Browse the repository at this point in the history

  2. Protobuf formatting fix

    @erikgrinaker
    erikgrinaker committed Mar 26, 2020
    Configuration menu
    Copy the full SHA
    076865a View commit details
    Browse the repository at this point in the history

  3. Return chunk metadata as well in Store.LoadChunk()

    @erikgrinaker
    erikgrinaker committed Mar 26, 2020
    Configuration menu
    Copy the full SHA
    fe8c88b View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2020

  1. Add snapshots.Restorer()

    @erikgrinaker
    erikgrinaker committed Mar 27, 2020
    Configuration menu
    Copy the full SHA
    33fa307 View commit details
    Browse the repository at this point in the history

  2. Typo

    @erikgrinaker
    erikgrinaker committed Mar 27, 2020
    Configuration menu
    Copy the full SHA
    a34e12c View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2020

  1. Use table-driven tests for rootmulti.Store snapshot/restore error tests

    @erikgrinaker
    erikgrinaker committed Mar 28, 2020
    Configuration menu
    Copy the full SHA
    ba031f6 View commit details
    Browse the repository at this point in the history

  2. use zlib compression for snapshots

    @erikgrinaker
    erikgrinaker committed Mar 28, 2020
    Configuration menu
    Copy the full SHA
    e507801 View commit details
    Browse the repository at this point in the history

  3. add checksum test for snapshot format stability

    @erikgrinaker
    erikgrinaker committed Mar 28, 2020
    Configuration menu
    Copy the full SHA
    3df5b25 View commit details
    Browse the repository at this point in the history

  4. use larger generated dataset for snapshot checksum test

    @erikgrinaker
    erikgrinaker committed Mar 28, 2020
    Configuration menu
    Copy the full SHA
    09ff5c1 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2020

  1. simplify snapshot management for new ABCI interface

    @erikgrinaker
    erikgrinaker committed Mar 29, 2020
    Configuration menu
    Copy the full SHA
    2f7d969 View commit details
    Browse the repository at this point in the history

  2. bump snapshot chunk size to 10 MB

    @erikgrinaker
    erikgrinaker committed Mar 29, 2020
    Configuration menu
    Copy the full SHA
    67c0faa View commit details
    Browse the repository at this point in the history

  3. use sha256 hashes for snapshot chunks

    @erikgrinaker
    erikgrinaker committed Mar 29, 2020
    Configuration menu
    Copy the full SHA
    828ae14 View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2020

  1. Merge branch 'master' into erik/snapshot

    @erikgrinaker
    erikgrinaker committed Apr 1, 2020
    Configuration menu
    Copy the full SHA
    5714fed View commit details
    Browse the repository at this point in the history

  2. snapshots: added Store.GetLatest()

    @erikgrinaker
    erikgrinaker committed Apr 1, 2020
    Configuration menu
    Copy the full SHA
    624c4f5 View commit details
    Browse the repository at this point in the history

  3. baseapp: check for newer snapshots, to avoid snapshotting during replay

    @erikgrinaker
    erikgrinaker committed Apr 1, 2020
    Configuration menu
    Copy the full SHA
    d3afa8b View commit details
    Browse the repository at this point in the history

  4. baseapp: fix nil dereferencing

    @erikgrinaker
    erikgrinaker committed Apr 1, 2020
    Configuration menu
    Copy the full SHA
    c46438e View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2020

  1. Implemented ABCI snapshot skeleton

    @erikgrinaker
    erikgrinaker committed Apr 2, 2020
    Configuration menu
    Copy the full SHA
    5cac251 View commit details
    Browse the repository at this point in the history

  2. Ported Tendermint API changes

    @erikgrinaker
    erikgrinaker committed Apr 2, 2020
    Configuration menu
    Copy the full SHA
    857b3dd View commit details
    Browse the repository at this point in the history

  3. Implement ABCI snapshot interface

    @erikgrinaker
    erikgrinaker committed Apr 2, 2020
    Configuration menu
    Copy the full SHA
    7d0009c View commit details
    Browse the repository at this point in the history

  4. don't limit ListSnapshots to 100, caller can do this

    @erikgrinaker
    erikgrinaker committed Apr 2, 2020
    Configuration menu
    Copy the full SHA
    994b731 View commit details
    Browse the repository at this point in the history

  5. updated with simplified ABCI interface

    @erikgrinaker
    erikgrinaker committed Apr 2, 2020
    Configuration menu
    Copy the full SHA
    ab71bcb View commit details
    Browse the repository at this point in the history

  6. update with new chunk size

    @erikgrinaker
    erikgrinaker committed Apr 2, 2020
    Configuration menu
    Copy the full SHA
    a753a0e View commit details
    Browse the repository at this point in the history

  7. use sha-256 chunk hashes

    @erikgrinaker
    erikgrinaker committed Apr 2, 2020
    Configuration menu
    Copy the full SHA
    76b79b1 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2020

  1. update with TM rpc/client changes

    @erikgrinaker
    erikgrinaker committed Apr 3, 2020
    Configuration menu
    Copy the full SHA
    5afe4f4 View commit details
    Browse the repository at this point in the history

  2. add snapshots.Manager, restructure code, and write tests

    @erikgrinaker
    erikgrinaker committed Apr 3, 2020
    Configuration menu
    Copy the full SHA
    836fa5e View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2020

  1. reduce timeout

    @erikgrinaker
    erikgrinaker authored Apr 4, 2020
    Configuration menu
    Copy the full SHA
    8efccd5 View commit details
    Browse the repository at this point in the history

  2. add test for restoring empty IAVL stores

    @erikgrinaker
    erikgrinaker committed Apr 4, 2020
    Configuration menu
    Copy the full SHA
    679c2f0 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2020

  1. go.mod: bump iavl

    @erikgrinaker
    erikgrinaker committed Apr 5, 2020
    Configuration menu
    Copy the full SHA
    7a81f0a View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2020

  1. check for error when importing IAVL nodes

    @erikgrinaker
    erikgrinaker committed Apr 14, 2020
    Configuration menu
    Copy the full SHA
    fb6580c View commit details
    Browse the repository at this point in the history

  2. handle empty keys and values via Protobuf

    @erikgrinaker
    erikgrinaker committed Apr 14, 2020
    Configuration menu
    Copy the full SHA
    f895ca1 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2020

  1. Merge branch 'master' into erik/snapshot

    @erikgrinaker
    erikgrinaker committed Apr 24, 2020
    Configuration menu
    Copy the full SHA
    94f1e44 View commit details
    Browse the repository at this point in the history

  2. change tmkv.Pair to abci.EventAttribute

    @erikgrinaker
    erikgrinaker committed Apr 24, 2020
    Configuration menu
    Copy the full SHA
    d2fb90e View commit details
    Browse the repository at this point in the history

  3. initial port to new state sync ABCI interface

    @erikgrinaker
    erikgrinaker committed Apr 24, 2020
    Configuration menu
    Copy the full SHA
    432ff4a View commit details
    Browse the repository at this point in the history

  4. don't snapshot mem.Store stores

    @erikgrinaker
    erikgrinaker committed Apr 24, 2020
    Configuration menu
    Copy the full SHA
    3c52b05 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2020

  1. Merge branch 'master' into erik/snapshot

    @erikgrinaker
    erikgrinaker committed Apr 29, 2020
    Configuration menu
    Copy the full SHA
    e7cb0be View commit details
    Browse the repository at this point in the history

Commits on May 6, 2020

  1. minor tweaks

    @erikgrinaker
    erikgrinaker committed May 6, 2020
    Configuration menu
    Copy the full SHA
    5a9def7 View commit details
    Browse the repository at this point in the history

  2. handle chunk hash verification

    @erikgrinaker
    erikgrinaker committed May 6, 2020
    Configuration menu
    Copy the full SHA
    0888800 View commit details
    Browse the repository at this point in the history

  3. use pruning options for config

    @erikgrinaker
    erikgrinaker committed May 6, 2020
    Configuration menu
    Copy the full SHA
    683bd35 View commit details
    Browse the repository at this point in the history

  4. improve error handling

    @erikgrinaker
    erikgrinaker committed May 6, 2020
    Configuration menu
    Copy the full SHA
    bf04df1 View commit details
    Browse the repository at this point in the history

  5. remove snapshot flags

    @erikgrinaker
    erikgrinaker committed May 6, 2020
    Configuration menu
    Copy the full SHA
    963432f View commit details
    Browse the repository at this point in the history

Commits on May 7, 2020

  1. use new ABCI enums

    @erikgrinaker
    erikgrinaker committed May 7, 2020
    Configuration menu
    Copy the full SHA
    4c3cdac View commit details
    Browse the repository at this point in the history