Dynamic trailing computation

[marco6]

This PR adds a non-default strategy to dynamically compute the amount of entries to keep in memory based on the size of the snapshot.

The problem with using a fixed number is that each entry has a different size (depending on the number of pages written for each transaction). As such, the memory used (and then streamed to a lagging follower) might be way bigger than the size of the snapshot itself (even by order of magnitude).

[marco6]

This PR lacks testing for now.

[codecov]

Codecov Report

Attention: Patch coverage is 19.35484% with 25 lines in your changes missing coverage. Please review.

Project coverage is 81.12%. Comparing base (7643fdf) to head (56abe80).
Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
src/raft/replication.c	23.80%	13 Missing and 3 partials ⚠️
src/raft/raft.c	10.00%	9 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #714      +/-   ##
==========================================
+ Coverage   74.02%   81.12%   +7.10%     
==========================================
  Files         192      196       +4     
  Lines       27826    29589    +1763     
  Branches     2796     4096    +1300     
==========================================
+ Hits        20597    24005    +3408     
+ Misses       4905     3920     -985     
+ Partials     2324     1664     -660     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[letFunny]

Thanks Marco for this PR, it looks very good to me, no comments so far. For the testing part of the story, we have the snapshot tests at test/raft/integration/test_snapshot.c so we could probably add this alternate strategy as a parameter for the existing tests there (where it makes sense). There are other tests that might be applicable such as the replication tests (test_replication.c).

We should also add a new test for the dynamic strategy asserting that the entries in the snapshot match the calculation for the expected size. Actually I was going to suggest changing existing tests to use both strategies by abstracting it away but I see that most of them use a threshold of 1 so it will probably require changing all of them, so adding a new one seems easier. I would suggest adding the tests in the unit tests for replication and in the integration tests, either in the replication_test or in snapshot_test.

Lastly, do not look at the tests under test/raft/unit/test_snapshot.c because those are for the new snapshot procedure that was never released.