[CLI] Restructure local testnet code

[banool]

Stack

• Previous in stack: [CLI] Fix faucet component of local testnet without --force-restart #10214
• Next in stack: [CLI] Add support for running postgres in a container to the local testnet + rest of PRs in stack #10271

Description

When the local testnet was just a node and a faucet, it was somewhat acceptable to just have everything jammed into one file. With testnets becoming more fully featured, this would've become a big mess.

In this PR I introduce the ServiceManager trait. Using this trait, I separate each service into its own file, in which have:

• Configuration necessary for that service.
• Pre-run and post-healthy steps.
• Code for running the service itself.
• Logic for structuring service startup order, leveraging HealthCheckers.

Test Plan

Build a tools image locally:

DOCKER_DEFAULT_PLATFORM=linux/amd64 docker/builder/docker-bake-rust-all.sh tools

I used the remote docker builder for this.

Tag the image as devnet so I could use it as the base network for the CLI E2E tests:

docker tag aptos-core/tools:from-local aptos-core/tools:devnet

Run the CLI E2E tests where we use the local tools image:

cd crates/aptos/e2e
poetry install
DOCKER_DEFAULT_PLATFORM=linux/amd64  poetry run python main.py -d --base-network devnet --test-cli-path ~/a/core/target/debug/aptos --image-repo-with-project aptos-core --no-pull-always

2023-09-27 13:56:24,157 - INFO - All tests passed!

This is what it looks like when you run the local testnet now:

$ cargo run -p aptos -- node run-local-testnet --assume-yes --force-restart
Readiness endpoint: http://0.0.0.0:8070/

Faucet is starting, please wait...
Node API is starting, please wait...
Transaction stream is starting, please wait...

Completed generating configuration:
        Log file: "/Users/dport/.aptos/testnet/validator.log"
        Test dir: "/Users/dport/.aptos/testnet"
        Aptos root key path: "/Users/dport/.aptos/testnet/mint.key"
        Waypoint: 0:db2e0a937d667a20f2cd4b16a51f37d2d8d25e4e87301cbab2e89e33b41399be
        ChainId: 4
        REST API endpoint: http://0.0.0.0:8080
        Metrics endpoint: http://0.0.0.0:9101/metrics
        Aptosnet fullnode network endpoint: /ip4/0.0.0.0/tcp/6181
        Indexer gRPC node stream endpoint: 0.0.0.0:50051

Aptos is running, press ctrl-c to exit

Node API is running. Endpoint: http://0.0.0.0:8080/
Transaction stream is running. Endpoint: http://0.0.0.0:50051/
Faucet is running. Endpoint: http://127.0.0.1:8081/

Applying post startup steps...

Setup is complete, you can now use the local testnet!

[gregnazario]

I want to note that what's was fairly simple with 1-2 files is now, almost 10 files and relatively complex.

Let's make sure we add some more testing to ensure that all of these features work as expected in a followup PR, as well as ensure that the full node does start up with some performance check (I'm not sure we can do this in CI).

[github-actions]

✅ Forge suite realistic_env_max_load success on 7a9ed32e87d32f505b8315f1b46e5b909c41ff0c

two traffics test: inner traffic : committed: 5949 txn/s, latency: 6541 ms, (p50: 6200 ms, p90: 8600 ms, p99: 13100 ms), latency samples: 2593940
two traffics test : committed: 100 txn/s, latency: 2427 ms, (p50: 2200 ms, p90: 2700 ms, p99: 11200 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.207, avg: 0.197", "QsPosToProposal: max: 0.193, avg: 0.169", "ConsensusProposalToOrdered: max: 0.715, avg: 0.646", "ConsensusOrderedToCommit: max: 0.535, avg: 0.501", "ConsensusProposalToCommit: max: 1.237, avg: 1.147"]
Max round gap was 1 [limit 4] at version 1335999. Max no progress secs was 3.4960399 [limit 10] at version 1335999.
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite compat success on aptos-node-v1.6.2 ==> 7a9ed32e87d32f505b8315f1b46e5b909c41ff0c

Compatibility test results for aptos-node-v1.6.2 ==> 7a9ed32e87d32f505b8315f1b46e5b909c41ff0c (PR)
1. Check liveness of validators at old version: aptos-node-v1.6.2
compatibility::simple-validator-upgrade::liveness-check : committed: 3126 txn/s, latency: 7506 ms, (p50: 8200 ms, p90: 9900 ms, p99: 11700 ms), latency samples: 165700
2. Upgrading first Validator to new version: 7a9ed32e87d32f505b8315f1b46e5b909c41ff0c
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1781 txn/s, latency: 15540 ms, (p50: 19200 ms, p90: 21600 ms, p99: 22200 ms), latency samples: 92660
3. Upgrading rest of first batch to new version: 7a9ed32e87d32f505b8315f1b46e5b909c41ff0c
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1761 txn/s, latency: 16249 ms, (p50: 18800 ms, p90: 22500 ms, p99: 22700 ms), latency samples: 93380
4. upgrading second batch to new version: 7a9ed32e87d32f505b8315f1b46e5b909c41ff0c
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3468 txn/s, latency: 8673 ms, (p50: 9900 ms, p90: 11400 ms, p99: 12200 ms), latency samples: 145680
5. check swarm health
Compatibility test for aptos-node-v1.6.2 ==> 7a9ed32e87d32f505b8315f1b46e5b909c41ff0c passed
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions]

❌ Forge suite framework_upgrade failure on aptos-node-v1.5.1 ==> 7a9ed32e87d32f505b8315f1b46e5b909c41ff0c

Forge test runner terminated:
Trailing Log Lines:
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/rt.rs:148:20
  10: main
  11: __libc_start_main
  12: _start
Debugging output:

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking