ledger-tool: Move blockstore commands to blockstore subcommand

[steviez]

Problem

solana-ledger-tool is a bit of a kitchen sink, and has upwards of 30 commands at the top level (ie 30 commands that can be invoked via solana-ledger-tool <SOME_COMMAND>). UI Aside, the contents of ledger-tool/src/main.rs are somewhat cluttered.

Summary of Changes

To reduce the clutter, this PR:

• Introduces a new blockstore subcommand, and moves all commands that only operate on the blockstore to this new subcommand.
  • As an example, solana-ledger-tool slot is now solana-ledger-tool blockstore slot
• Supports the old command style (solana-ledger-tool slot), but does not display those commands in the top level --help command.
  • This steers new users towards using new calling format while avoiding breaking the old calling format for people who have muscle memory with those commands

With this PR, here is the subcommand section of the output for running solana-ledger-tool help:

USAGE:
    solana-ledger-tool [FLAGS] [OPTIONS] <SUBCOMMAND>
...
SUBCOMMANDS:
    accounts                   Print account stats and contents after processing the ledger
    bank-hash                  Prints the hash of the working bank after reading the ledger
    bigtable                   Ledger data on a BigTable instance
    blockstore                 Commands to interact with a local Blockstore
    capitalization             Print capitalization (aka, total supply) while checksumming it
    compute-slot-cost          runs cost_model over the block at the given slots, computes how expensive a block was
                               based on cost_model
    create-snapshot            Create a new ledger snapshot
    genesis                    Prints the ledger's genesis config
    genesis-hash               Prints the ledger's genesis hash
    graph                      Create a Graphviz rendering of the ledger
    help                       Prints this message or the help of the given subcommand(s)
    latest-optimistic-slots    Output up to the most recent <num-slots> optimistic slots with their hashes and
                               timestamps.
    modify-genesis             Modifies genesis parameters
    program                    Run to test, debug, and analyze on-chain programs.
    shred-version              Prints the ledger's shred hash
    verify                     Verify the ledger

And here is solana-ledger-tool blockstore:

USAGE:
    solana-ledger-tool blockstore [FLAGS] [OPTIONS] <SUBCOMMAND>
...    
SUBCOMMANDS:
    analyze-storage            Output statistics in JSON format about all column families in the ledger rocksdb
    bounds                     Print lowest and highest non-empty slots. Note that there may be empty slots within
                               the bounds
    copy                       Copy the ledger
    dead-slots                 Print all the dead slots in the ledger
    duplicate-slots            Print all the duplicate slots in the ledger
    help                       Prints this message or the help of the given subcommand(s)
    json                       Print the ledger in JSON format
    latest-optimistic-slots    Output up to the most recent <num-slots> optimistic slots with their hashes and
                               timestamps.
    list-roots                 Output up to last <num-roots> root hashes and their heights starting at the given
                               block height
    parse_full_frozen          Parses log for information about critical events about ancestors of the given
                               `ending_slot`
    print                      Print the ledger
    print-file-metadata        Print the metadata of the specified ledger-store file. If no file name is specified,
                               it will print the metadata of all ledger files.
    purge                      Delete a range of slots from the ledger
    remove-dead-slot           Remove the dead flag for a slot
    repair-roots               Traverses the AncestorIterator backward from a last known root to restore missing
                               roots to the Root column
    set-dead-slot              Mark one or more slots dead
    shred-meta                 Prints raw shred metadata
    slot                       Print the contents of one or more slots

Pre-requisite PRs

In order to reduce the noise in this PR, several items that I changed while working on this have been broken out. All below PR's should be merged and this PR rebased on top before it is ready to merge.

• ledger-tool: Move ledger output methods to output.rs #34595
• ledger-tool: Move blockstore arg parsing to open_blockstore() #34596
• ledger-tool: Switch subcommand dispatch from if-else to match #34644

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (744c2cb) 81.8% compared to head (a4eba32) 81.8%.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #34597     +/-   ##
=========================================
- Coverage    81.8%    81.8%   -0.1%     
=========================================
  Files         824      824             
  Lines      222393   222393             
=========================================
- Hits       181984   181924     -60     
- Misses      40409    40469     +60     

[steviez]

Ok, I think I have pulled out everything that I can to shrink this PR. That being said, still a pretty decent line count. But, I think commits are broken up pretty well such that review shouldn't be too insane. Let me know if this is not the case @CriesofCarrots and we can brainstorm on how to break this up even further.

I spot-checked a few items, ie that the following two calls both work:

$ cargo run -- bounds --ledger ./
$ cargo run -- blockstore bounds --ledger ./

I had previously spot-checked every command, but will probably do that again tomorrow just incase. It could be nice to have a unit-test, but that would seemingly entail

1. Moving the App creation and its' args into a subfunction AND
2. Repeating some of the lists of all the subcommand that should show up in both places
   

   solana/ledger-tool/src/main.rs

   Line 1069 in 744c2cb

   let matches = App::new(crate_name!())

So, I'm initially leaning towards no unit test, but open to discussion

[CriesofCarrots]

lgtm! I diffed almost all the move blocks and didn't come up with anything to comment on :)

[CriesofCarrots]

Could we/should we dedupe these with the existing definitions in main.rs at some point?

[steviez]

Yeah definitely; it is not expressed here, but I loosely allude to wanting to do some reorganization on that front in the problem descriptions for #34694 (a PR I have yet to add you too 😉).

The goals there would be deduping argument definitions (there were a few dupes prior to my changes here), and also creating several groupings of arguments. The groupings could be added to subcommands via this function to avoid long repeated lists like this for the [verify, create-snapshot, graph, ...] commands:

solana/ledger-tool/src/main.rs

Lines 1368 to 1392 in 8438544

	.arg(&no_snapshot_arg)
	.arg(&account_paths_arg)
	.arg(&accounts_hash_cache_path_arg)
	.arg(&accounts_index_path_arg)
	.arg(&halt_at_slot_arg)
	.arg(&limit_load_slot_count_from_snapshot_arg)
	.arg(&accounts_index_bins)
	.arg(&accounts_index_limit)
	.arg(&disable_disk_index)
	.arg(&accountsdb_skip_shrink)
	.arg(&accountsdb_verify_refcounts)
	.arg(&accounts_db_test_skip_rewrites_but_include_in_bank_hash)
	.arg(&verify_index_arg)
	.arg(&accounts_db_skip_initial_hash_calc_arg)
	.arg(&ancient_append_vecs)
	.arg(&halt_at_slot_store_hash_raw_data)
	.arg(&hard_forks_arg)
	.arg(&accounts_db_test_hash_calculation_arg)
	.arg(&os_memory_stats_reporting_arg)
	.arg(&allow_dead_slots_arg)
	.arg(&max_genesis_archive_unpacked_size_arg)
	.arg(&debug_key_arg)
	.arg(&geyser_plugin_args)
	.arg(&use_snapshot_archives_at_startup)
	.arg(