Re-enable skipping transactions below first inscription height

[casey]

I'm not sure how to do this. We'd need to get UTXO set values when we hit inscriptions. Maybe we only need UTXO values for transactions that contain inscriptions, in which case maybe the impact of fetching input transactions from bitcoin core is low.

We can also use the new verbosity level of getrawtransaction to return the value of all inputs in one go: bitcoin/bitcoin#23319

[raphjaph]

• Check wether getrawtransaciton --verbose is in newest Bitcoin Core release
• Start with PR agains rust-bitcoincore-rpc

[raphjaph]

Waiting for new release of Bitcoin Core

[andrewtoth]

Could we not just use getblock with verbosity=3 to get the blocks with all input values?

[casey]

Could we not just use getblock with verbosity=3 to get the blocks with all input values?

I suspect that this would be really slow. verbosity=3 includes a lot of extra data. However, it might be good to benchmark this and make sure it's the case.

[andrewtoth]

Sorry can you elaborate on why https://github.com/casey/ord/pull/1357 was merged? What was the problem with skipping? I re-enabled it and tried it with p2p indexing. Because of headers sync it syncs to first block in about 5 seconds, then is it getting every single input tx via RPC getrawtransaction? Or just some txs? If the latter we can use REST https://github.com/casey/ord/issues/1558#issuecomment-1421864816 to make that much faster and not have the RPC interface fall over. Or, we can call getblock with verbosity 3 for every block after if the former.
We really don't want to spam the RPC interface with many requests. I think that's the root of a lot of issues users are having.

[andrewtoth]

Also, if we are going the route of calling getrawtransaction individually vs one getblock verbosity=3, then we should probably be using async. We can then hit the REST endpoint concurrently with a join_all instead of sequentially. We could potentially batch the getrawtransaction calls but rust-bitcoincore-rpc doesn't support the batching feature :/.

[andrewtoth]

Ok I figured it out. Managed to sync by combining p2p and REST and reverting #1357 in 1:06. I think we can turn this back on :).

Batching calls would overwhelm bitcoind, and getblock verbosity=3 is slower than just synchronously fetching missing inputs via REST.

[andrewtoth]

With latest REST PR using hyper, myself on Linux and @so7ow on Mac synced in under an hour 🚀 https://github.com/casey/ord/issues/1648#issuecomment-1427946904.

[andrewtoth]

Check out https://github.com/casey/ord/pull/1759. It still hasn't been explained why skipping transactions were removed, but that PR manages to sync efficiently without having all inputs.