Iterator stops at certain block height

[themoritz]

I have a local fully synced blockchain that was written by Bitcoin-qt, and the Bitcoin software runs without complaining.

When I loop through bitcoin.iter_block::<FBlock>(761200, 784540), the iterator always stops at block 761248. I'm not doing anything in the loop.

Do you have any idea of how I could debug this? I already have a logging crate enabled and am looking at TRACE output. These are all the log messages I can see from your lib:

2023-04-08T20:11:28.786Z INFO  [bitcoin_explorer::parser::block_index] Start loading block_index
2023-04-08T20:11:29.688Z INFO  [bitcoin_explorer::parser::tx_index] Successfully opened tx_index DB!

[Congyuwang]

Hi. I assume that this is caused because get_block::<TBlock>(height) returned error at height 761249. The way to debug it is to directly call get_block::<TBlock>(761249) and see what it returns.

[Congyuwang]

Rusty-Bitcoin-Explorer/src/iter/iter_block.rs

Lines 11 to 28 in 2d3d38b

	impl<TBlock> BlockIter<TBlock>
	where
	TBlock: From<Block> + Send + 'static,
	{
	/// the worker threads are dispatched in this `new` constructor!
	pub fn new<T>(db: &BitcoinDB, heights: T) -> Self
	where
	T: IntoIterator<Item = usize> + Send + 'static,
	<T as IntoIterator>::IntoIter: Send + 'static,
	{
	let db_ref = db.clone();
	BlockIter(
	heights.into_par_iter_sync(move |h| match db_ref.get_block::<TBlock>(h) {
	Ok(blk) => Ok(blk),
	Err(_) => Err(()),
	}),
	)
	}

In iter_block, errors are ignored and regarded as iterator terminator.

[themoritz]

Thanks, that helps. I got a block decode error now. Will investigate further into why that is happening.

[themoritz]

I'm not sure how to further debug this.

I reindexed my whole blockchain with bitcoind -reindex, and am also able to successfully run commands like getblockhash, getblock and getblockstats on the block with bitcoin-cli. This leads me to assume that my blockchain data is not corrupted.

Do you know how I could further debug this library to see where it is failing?

[themoritz]

I can even successfully run commands like this (on the relevant block hash):

bitcoin-cli getblock 000000000000000000070742427fa10ec3c66d006160155a704d9f56d090a3ea | jq -r '.tx[]' | xargs -I _ bitcoin-cli getrawtransaction _ 1 > foo.json

One thing that I noticed in this block is that there's a transaction (73be398c4bdc43709db7398106609eea2a7841aaf3a4fa2000dc18184faa2a7e) which has a huge amount of witness data (looks like 500k).

Funnily this tx was created in order to show a consensus diff between bitcoin core and btcd: btcsuite/btcd#1906

[Congyuwang]

Hi. This project depends on rust-bitcoin for decoding binary data. So, I'd suggest that you upgrade the rust-bitcoin dependency and have try. Let me do the upgrade.

[themoritz]

Ah, I think we need to bump the bitcoin dependency to fix this: https://twitter.com/lightningdevkit/status/1587939344530145280

[themoritz]

I can already confirm that bumping bitcoin to 0.28.2 in this lib fixes my issue.

[themoritz]

Bumping further seems to break code.

[Congyuwang]

#25 Bumped to v0.28.2