Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(pruner): Implement full and bridge node pruning #3150

Merged
merged 35 commits into from
Mar 28, 2024
Merged

feat(pruner): Implement full and bridge node pruning #3150

merged 35 commits into from
Mar 28, 2024

Commits on Mar 27, 2024

  1. Initial implmenetation of the full pruner

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    953a407 View commit details
    Browse the repository at this point in the history

  2. feat: initial untested findPruneableHeaders impl, findPruneableHeader… 

    …s unit tests
    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    6ee4305 View commit details
    Browse the repository at this point in the history

  3. feat: full node pruner

    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    3b74604 View commit details
    Browse the repository at this point in the history

  4. refactor(pruner): making Prune signature non-variadic for easier erro… 

    …r handling
    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    1ac1ea9 View commit details
    Browse the repository at this point in the history

  5. fix(pruner/full): short-circuiting on Prune

    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    89621d9 View commit details
    Browse the repository at this point in the history

  6. test(pruner): Fixing expected value in TestService

    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    9b1bcee View commit details
    Browse the repository at this point in the history

  7. feat(pruner): adding metrics

    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    a8fcab1 View commit details
    Browse the repository at this point in the history

  8. test: setting availability window to 1 day and adding full pruner to … 

    …bridge
    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    3c9500b View commit details
    Browse the repository at this point in the history

  9. test: enabling gc

    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    438d8d1 View commit details
    Browse the repository at this point in the history

  10. fix(pruner): add debug logs and init the failed headers map, reduce 

    sampling window, pull in slavas p2p metrics fixes
    @distractedm1nd @renaynay
    distractedm1nd authored and renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    94ed6f3 View commit details
    Browse the repository at this point in the history

  11. feat(pruner): store failed headers in checkpoint + test

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    bc35ace View commit details
    Browse the repository at this point in the history

  12. test(pruner): Test the pruner service, includes commit to enable bloc… 

    …k time in header test
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    7b69de7 View commit details
    Browse the repository at this point in the history

  13. feat(pruner): Implement retry functionality

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    422a949 View commit details
    Browse the repository at this point in the history

  14. feat(pruner): make max pruneable configurable and shorten gc cycle + … 

    …increase default max pruneable
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    cb214e3 View commit details
    Browse the repository at this point in the history

  15. go mod tidy

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    800d00b View commit details
    Browse the repository at this point in the history

  16. feat(nodebuilder/pruner): Enable pruning with --experimental-pruning … 

    …flag
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    24ba17b View commit details
    Browse the repository at this point in the history

  17. chore(nodebuilder/pruner): remove stubbed out code | fix(pruner): con… 

    …text deadline should be per block
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    a9218e9 View commit details
    Browse the repository at this point in the history

  18. fix(pruner): Metrics callback unify

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    06dba21 View commit details
    Browse the repository at this point in the history

  19. fix | test(pruner): Make TestService more deterministic, Make some ot… 

    …her tests deterministic and test one more case for retries
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    2e9bf7f View commit details
    Browse the repository at this point in the history

  20. chore(pruner): remove TODOs

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    6113788 View commit details
    Browse the repository at this point in the history

  21. fix(pruner): findPruneable also prunes genesis block, return windows … 

    …to proper values, sane MaxPruneable values, return gc cycle to every 5 min
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    811cd73 View commit details
    Browse the repository at this point in the history

  22. fix(pruner): Remove maxPruneablePerGC and introduce maxHeadersPerLoop

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    5bfc7be View commit details
    Browse the repository at this point in the history

  23. fix(pruner/full): window should be type AvailabilityWindow

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    f7a183d View commit details
    Browse the repository at this point in the history

  24. fix(pruner): Do not omit FailedHeaders if not set

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    14fbbc6 View commit details
    Browse the repository at this point in the history

  25. chore(pruner): extend prune ctx deadline to 5 sec instead of 1, be a … 

    …bit more generous, fix log
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    32a52d0 View commit details
    Browse the repository at this point in the history

  26. fix(pruner): checkpoint failed map doesnt store error, UTC used for t… 

    …ime, simplify ParseFlags
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    22c1c17 View commit details
    Browse the repository at this point in the history

  27. fix(pruner): remove lastPruned header from checkpoint and just use he… 

    …ight
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    d817db6 View commit details
    Browse the repository at this point in the history

  28. fix(pruner): only log actual errors from findPruneable

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    80feb82 View commit details
    Browse the repository at this point in the history

  29. chore(pruner): idiomatic defer close chan

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    b7cb726 View commit details
    Browse the repository at this point in the history

  30. feat(pruner): change finder algo to use blockTime, remove disabledGC … 

    …opt,
    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    3e8b65f View commit details
    Browse the repository at this point in the history

  31. fix(pruner): Reduce maxHeadersPerLoop to 512

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    eafab98 View commit details
    Browse the repository at this point in the history

  32. chore: lint

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    32b8f91 View commit details
    Browse the repository at this point in the history

  33. chore(pruner): fix comments from hlib

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    7e8fd9c View commit details
    Browse the repository at this point in the history

  34. fix(pruner): just short-circuit when there is nothing left to prune

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    ba66240 View commit details
    Browse the repository at this point in the history

  35. fix(pruner): unregister callbck from meter on stop

    @renaynay
    renaynay committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    d5a9784 View commit details
    Browse the repository at this point in the history