Add option to record transactions to ledger-tool

[seanyoung]

Problem

When diagnosing a consensus failure, ledger-tool has the --record-slots and --verify-slots option to find the diverging slot. This PR add a --record-slots-config=tx option, which writes all transactions to the recording. This can be be used to find the exact transaction where the failure was.

Short workflow:

• ledger-tool verify --records-slots --record-slots-config=tx
• (make change to the runtime)
• ledger-tool verify --record-slots slots-before.json --record-slots-config=tx
• diff -u slots.json slots-before.json

Longer Workflow if the files get too large:

• ledger-tool verify --records-slots
• (make change to the runtime)
• ledger-tool verify --verify-slots => slot 102 diverges
• ledger-tool verify --record-slots-config=tx --record-slots=after.json --halt-at-slot 102
• (revert runtime change)
• ledger-tool verify --record-slots-config=tx --record-slots=before.json --halt-at-slot 102
• diff -u after.json before.json

Now the diff will show exactly where things went wrong.

Summary of Changes

Fixes #

[seanyoung]

@alessandrod I found this helpful for debugging consensus failures, wdyt

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 59.09091% with 9 lines in your changes are missing coverage. Please review.

Project coverage is 81.8%. Comparing base (025ed45) to head (214cc9b).
Report is 2 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #181   +/-   ##
=======================================
  Coverage    81.8%    81.8%           
=======================================
  Files         846      846           
  Lines      228889   228900   +11     
=======================================
+ Hits       187419   187441   +22     
+ Misses      41470    41459   -11     

[HaoranYi]

@alessandrod I found this helpful for debugging consensus failures, wdyt

I used this PR when working on my features. The transaction log is very helpful. I would say let's merge it.

[steviez]

This is a good idea overall, and piggybacking off of the TransactionStatusSender is a seemingly cleaner solution than the initial crack I took at adding this a while back. No code changes in /ledger/src 🚀 .

A few comments / questions right now, will take a second pass at this later today again

[steviez]

Apologies on the delayed review again. I think we've previously discussed that it would be nice to stream the details as they come in. I had initially been hoping/planning to make the BankHashDetails structure support streaming, and then land your PR (which would hopefully be simplified by doing so). But, that clearly hasn't happened as other things have come up and I haven't found the time.

So, I'll stop letting perfect get in the way of good as this is definitely a useful change. Things look fairly good as-is, just one nit right now + looks like you need to rebase.

[steviez]

I can try to follow up with some of the enhancements we've discussed offline (ie streaming)