Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration progress in percentage #4725

Merged

Conversation

RickiNano
Copy link
Contributor

Each step in the LMDB -> RocksDb migration now shows the progress in percentage completion.
This pr also adds a missing check for rep_weights count on completion.

Here is an example of the output log after a migration of the current live ledger. I have cut the example down to reduce size.

[2024-09-08 13:27:51.448] [ledger] [info] Migrating LMDB database to RocksDB. This will take a while...
[2024-09-08 13:27:51.700] [ledger] [info] Step 1 of 7: Converting 201939233 entries from blocks table
[2024-09-08 13:28:06.071] [ledger] [info] 5000000 blocks converted (2%)
[2024-09-08 13:28:26.194] [ledger] [info] 10000000 blocks converted (4%)
[2024-09-08 13:28:43.688] [ledger] [info] 15000000 blocks converted (7%)
[2024-09-08 13:29:36.411] [ledger] [info] 20000000 blocks converted (9%)
[2024-09-08 13:31:02.810] [ledger] [info] 25000000 blocks converted (12%)
[2024-09-08 13:31:55.081] [ledger] [info] 30000000 blocks converted (14%)
............
[2024-09-08 14:05:12.277] [ledger] [info] 185000000 blocks converted (91%)
[2024-09-08 14:06:40.593] [ledger] [info] 190000000 blocks converted (94%)
[2024-09-08 14:07:43.115] [ledger] [info] 195000000 blocks converted (96%)
[2024-09-08 14:08:34.332] [ledger] [info] 200000000 blocks converted (99%)
[2024-09-08 14:09:15.069] [ledger] [info] 201939233 entries converted (100%)
[2024-09-08 14:09:15.071] [ledger] [info] Step 2 of 7: Converting 27226087 entries from pending table
[2024-09-08 14:09:15.754] [ledger] [info] 500000 entries converted (1%)
[2024-09-08 14:09:16.513] [ledger] [info] 1000000 entries converted (3%)
[2024-09-08 14:09:17.422] [ledger] [info] 1500000 entries converted (5%)
[2024-09-08 14:09:18.744] [ledger] [info] 2000000 entries converted (7%)
[2024-09-08 14:09:34.113] [ledger] [info] 2500000 entries converted (9%)
[2024-09-08 14:09:35.856] [ledger] [info] 3000000 entries converted (11%)
............
[2024-09-08 14:10:54.568] [ledger] [info] 24500000 entries converted (89%)
[2024-09-08 14:10:57.789] [ledger] [info] 25000000 entries converted (91%)
[2024-09-08 14:11:02.119] [ledger] [info] 25500000 entries converted (93%)
[2024-09-08 14:11:05.754] [ledger] [info] 26000000 entries converted (95%)
[2024-09-08 14:11:11.586] [ledger] [info] 26500000 entries converted (97%)
[2024-09-08 14:11:13.163] [ledger] [info] 27000000 entries converted (99%)
[2024-09-08 14:11:14.047] [ledger] [info] 27226087 entries converted (100%)
[2024-09-08 14:11:14.047] [ledger] [info] Step 3 of 7: Converting 36913275 entries from confirmation_height table
[2024-09-08 14:11:14.656] [ledger] [info] 500000 entries converted (1%)
[2024-09-08 14:11:15.168] [ledger] [info] 1000000 entries converted (2%)
[2024-09-08 14:11:15.630] [ledger] [info] 1500000 entries converted (4%)
[2024-09-08 14:11:16.093] [ledger] [info] 2000000 entries converted (5%)
[2024-09-08 14:11:16.632] [ledger] [info] 2500000 entries converted (6%)
............
[2024-09-08 14:12:58.828] [ledger] [info] 35000000 entries converted (94%)
[2024-09-08 14:12:59.383] [ledger] [info] 35500000 entries converted (96%)
[2024-09-08 14:12:59.678] [ledger] [info] 36000000 entries converted (97%)
[2024-09-08 14:13:15.473] [ledger] [info] 36500000 entries converted (98%)
[2024-09-08 14:13:17.734] [ledger] [info] 36913275 entries converted (100%)
[2024-09-08 14:13:17.734] [ledger] [info] Step 4 of 7: Converting 36913275 entries from accounts table
[2024-09-08 14:13:18.010] [ledger] [info] 500000 entries converted (1%)
[2024-09-08 14:13:18.549] [ledger] [info] 1000000 entries converted (2%)
[2024-09-08 14:13:19.270] [ledger] [info] 1500000 entries converted (4%)
[2024-09-08 14:13:20.118] [ledger] [info] 2000000 entries converted (5%)
[2024-09-08 14:13:20.964] [ledger] [info] 2500000 entries converted (6%)
[2024-09-08 14:13:21.551] [ledger] [info] 3000000 entries converted (8%)
............
[2024-09-08 14:17:30.146] [ledger] [info] 35500000 entries converted (96%)
[2024-09-08 14:17:31.476] [ledger] [info] 36000000 entries converted (97%)
[2024-09-08 14:17:33.273] [ledger] [info] 36500000 entries converted (98%)
[2024-09-08 14:17:34.482] [ledger] [info] 36913275 entries converted (100%)
[2024-09-08 14:17:34.483] [ledger] [info] Step 5 of 7: Converting 5538654 entries from rep_weights table
[2024-09-08 14:17:35.524] [ledger] [info] 500000 entries converted (9%)
[2024-09-08 14:17:36.697] [ledger] [info] 1000000 entries converted (18%)
[2024-09-08 14:17:38.295] [ledger] [info] 1500000 entries converted (27%)
[2024-09-08 14:17:40.180] [ledger] [info] 2000000 entries converted (36%)
[2024-09-08 14:17:41.463] [ledger] [info] 2500000 entries converted (45%)
[2024-09-08 14:17:42.710] [ledger] [info] 3000000 entries converted (54%)
[2024-09-08 14:17:43.672] [ledger] [info] 3500000 entries converted (63%)
[2024-09-08 14:17:46.968] [ledger] [info] 4000000 entries converted (72%)
[2024-09-08 14:17:52.017] [ledger] [info] 4500000 entries converted (81%)
[2024-09-08 14:17:58.306] [ledger] [info] 5000000 entries converted (90%)
[2024-09-08 14:18:01.618] [ledger] [info] 5500000 entries converted (99%)
[2024-09-08 14:18:02.008] [ledger] [info] 5538654 entries converted (100%)
[2024-09-08 14:18:02.008] [ledger] [info] Step 6 of 7: Converting 0 entries from pruned table
[2024-09-08 14:18:02.019] [ledger] [info] 0 entries converted (100%)
[2024-09-08 14:18:02.019] [ledger] [info] Step 7 of 7: Converting 43278268 entries from final_votes table
[2024-09-08 14:18:02.432] [ledger] [info] 500000 entries converted (1%)
[2024-09-08 14:18:03.066] [ledger] [info] 1000000 entries converted (2%)
[2024-09-08 14:18:03.529] [ledger] [info] 1500000 entries converted (3%)
[2024-09-08 14:18:04.145] [ledger] [info] 2000000 entries converted (4%)
[2024-09-08 14:18:04.758] [ledger] [info] 2500000 entries converted (5%)
............
[2024-09-08 14:21:06.247] [ledger] [info] 41000000 entries converted (94%)
[2024-09-08 14:21:19.742] [ledger] [info] 41500000 entries converted (95%)
[2024-09-08 14:21:20.616] [ledger] [info] 42000000 entries converted (97%)
[2024-09-08 14:21:22.728] [ledger] [info] 42500000 entries converted (98%)
[2024-09-08 14:21:23.769] [ledger] [info] 43000000 entries converted (99%)
[2024-09-08 14:21:24.657] [ledger] [info] 43278268 entries converted (100%)
[2024-09-08 14:21:24.657] [ledger] [info] Finalizing migration...
[2024-09-08 14:21:26.886] [ledger] [info] Migration completed. Make sure to enable RocksDb in the config file under [node.rocksdb]
[2024-09-08 14:21:26.886] [ledger] [info] After confirming correct node operation, the data.ldb file can be deleted if no longer required

@clemahieu clemahieu merged commit 0cfb0a7 into nanocurrency:develop Sep 25, 2024
24 of 28 checks passed
@qwahzi qwahzi added this to the V28 milestone Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Merged / V28.0
Development

Successfully merging this pull request may close these issues.

3 participants