[develop] added eosio-blocklog options to take last good blocks

[kimjh2005]

Change Description

eosio-blocklog --trim-blocklog allows you to take a block.log + block.index and repair it to a usable situation. You need to know what is a good block to repair to. If you choose a block in the future, it fails.

New options:
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --help
--check-files Check blocks.index and blocks.log.
Must give 'blocks-dir'.
--update-index Update blocks.index with the best
effort to avoid creating a new index
file. Must give 'blocks-dir'. Give
'output-file' relative to current
directory or absolute path (default is
/blocks.index).
--fix-corrupted-file fix the corrupted end of blocks.log and
blocks.index files by trimming the
incomplete block. Must give
'blocks-dir'.

Example)
~/tmp/blocks> ls -al
-rw-r--r-- 1 jongho.kim staff 128 Jun 9 13:52 blocks.index
-rw-r--r-- 1 jongho.kim staff 3198 Jun 9 13:47 blocks.log
drwxr-xr-x 3 jongho.kim staff 96 Jun 9 13:47 reversible

// truncate blocks.log
~/tmp/blocks> truncate -s 3000 blocks.log
~/tmp/blocks> ls -al
-rw-r--r-- 1 jongho.kim staff 128 Jun 9 13:52 blocks.index
-rw-r--r-- 1 jongho.kim staff 3000 Jun 9 14:27 blocks.log
drwxr-xr-x 3 jongho.kim staff 96 Jun 9 13:47 reversible

// smoke-test crashes because of the corrupted file
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --smoke-test

Smoke test of blocks.log and blocks.index in directory "/Users/jongho.kim/tmp/blocks"
zsh: segmentation fault ./eosio-blocklog --blocks-dir ~/tmp/blocks --smoke-test
~/contracts/eos/build/programs/eosio-blocklog>

// check-file got the assertion
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --check-files
Check blocks.index and blocks.log in "/Users/jongho.kim/tmp/blocks"
info 2020-06-09T18:29:08.512 thread-0 block_log.cpp:673 check_corrupted_bloc ] blocks.index file looks good with size of 128 bytes.
info 2020-06-09T18:29:08.513 thread-0 block_log.cpp:754 check_corrupted_bloc ] Found incomplete 186 bytes in blocks.log file
error 2020-06-09T18:29:08.513 thread-0 main.cpp:381 main ] 3190000 block_log_exception: Block log exception
corrupted blocks.log/blocks.index.
{}
thread-0 block_log.cpp:598 check_files

// fix-corrupted-file fixed the blocks.log
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --fix-corrupted-file
Fix corrupted blocks.log and blocks.index files in "/Users/jongho.kim/tmp/blocks"
info 2020-06-09T18:30:02.781 thread-0 block_log.cpp:673 check_corrupted_bloc ] blocks.index file looks good with size of 128 bytes.
info 2020-06-09T18:30:02.781 thread-0 block_log.cpp:754 check_corrupted_bloc ] Found incomplete 186 bytes in blocks.log file
info 2020-06-09T18:30:02.781 thread-0 block_log.cpp:758 check_corrupted_bloc ] Fixed blocks.log file by trimming 186 bytes.

no problems found

// check-file still asserts because of blocks.index not updated
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --check-files

Check blocks.index and blocks.log in "/Users/jongho.kim/tmp/blocks"
info 2020-06-09T18:30:38.378 thread-0 block_log.cpp:673 check_corrupted_bloc ] blocks.index file looks good with size of 128 bytes.
info 2020-06-09T18:30:38.378 thread-0 block_log.cpp:747 check_corrupted_bloc ] blocks.log file looks good with size of 2814 bytes.
info 2020-06-09T18:30:38.378 thread-0 block_log.cpp:658 check_corrupted_file ] No corrupted block or index found!
info 2020-06-09T18:30:38.378 thread-0 block_log.cpp:608 check_files ] blocks.log file /Users/jongho.kim/tmp/blocks/blocks.log has 14 blocks
info 2020-06-09T18:30:38.378 thread-0 block_log.cpp:609 check_files ] blocks.index file /Users/jongho.kim/tmp/blocks/blocks.index has 16 blocks
error 2020-06-09T18:30:38.378 thread-0 main.cpp:381 main ] 3190000 block_log_exception: Block log exception
Need to update index...
{}
thread-0 block_log.cpp:633 check_files

// smoke-test failed, too.
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --smoke-test

Smoke test of blocks.log and blocks.index in directory "/Users/jongho.kim/tmp/blocks"
error 2020-06-09T18:31:35.742 thread-0 main.cpp:381 main ] 3190000 block_log_exception: Block log exception
/Users/jongho.kim/tmp/blocks/blocks.log says it has 14 blocks which disagrees with 16 indicated by /Users/jongho.kim/tmp/blocks/blocks.index
{"block_file_name":"/Users/jongho.kim/tmp/blocks/blocks.log","log_num_blocks":14,"index_num_blocks":16,"index_file_name":"/Users/jongho.kim/tmp/blocks/blocks.index"}
thread-0 block_log.cpp:456 block_log_archive

// update-index
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --update-index

Update blocks.index to blocks.log in "/Users/jongho.kim/tmp/blocks"
info 2020-06-09T18:32:21.662 thread-0 block_log.cpp:539 fix_index_file ] Log is nonempty
info 2020-06-09T18:32:21.663 thread-0 block_log.cpp:546 fix_index_file ] Index is nonempty
info 2020-06-09T18:32:21.663 thread-0 block_log.cpp:554 fix_index_file ] The last block positions from blocks.log and blocks.index are different, Reconstructing index...
info 2020-06-09T18:32:21.663 thread-0 block_log.cpp:557 fix_index_file ] Need to update index file

no problems found

// check-files ok
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --check-files

Check blocks.index and blocks.log in "/Users/jongho.kim/tmp/blocks"
info 2020-06-09T18:32:26.275 thread-0 block_log.cpp:673 check_corrupted_bloc ] blocks.index file looks good with size of 112 bytes.
info 2020-06-09T18:32:26.275 thread-0 block_log.cpp:747 check_corrupted_bloc ] blocks.log file looks good with size of 2814 bytes.
info 2020-06-09T18:32:26.275 thread-0 block_log.cpp:658 check_corrupted_file ] No corrupted block or index found!
info 2020-06-09T18:32:26.275 thread-0 block_log.cpp:608 check_files ] blocks.log file /Users/jongho.kim/tmp/blocks/blocks.log has 14 blocks
info 2020-06-09T18:32:26.275 thread-0 block_log.cpp:609 check_files ] blocks.index file /Users/jongho.kim/tmp/blocks/blocks.index has 14 blocks
info 2020-06-09T18:32:26.275 thread-0 block_log.cpp:635 check_files ] Index is up-to-date

no problems found

// smoke-test ok
~/contracts/eos/build/programs/eosio-blocklog> ./eosio-blocklog --blocks-dir ~/tmp/blocks --smoke-test

Smoke test of blocks.log and blocks.index in directory "/Users/jongho.kim/tmp/blocks"
info 2020-06-09T18:32:28.436 thread-0 block_log.cpp:1225 smoke_test ] blocks.log and blocks.index agree on number of blocks

no problems found

// 2 blocks truncated
~/tmp/blocks> ls -al
-rw-r--r-- 1 jongho.kim staff 112 Jun 9 14:32 blocks.index
-rw-r--r-- 1 jongho.kim staff 2814 Jun 9 14:30 blocks.log

Change Type

Select ONE

• Documentation

• Stability bug fix

• Other

• Other - special case

Consensus Changes

• Consensus Changes

API Changes

• API Changes

Documentation Additions

• Documentation Additions

[kimjh2005]

Modified to use blocks.index to repair damaged blocks.log.

[kimjh2005]

Duplicate fix implemented in PR
#9184