ordinals.com indexer down due to reorg in block 781277

[eyusufatik]

No description provided.

[BitcoinWay]

I'm getting this error in ord
error: reorg detected at or before 781277
totally dead in the water

[schlunsen]

same here

[schlunsen]

As I understand by now it seems to be caused by chain reorganization

[0attack]

wow this is such a huge issue. also being discussed in https://github.com/casey/ord/issues/148

[veryordinally]

I triggered a re-index on ordinals.com last night, and it is currently reindexing. There is currently not enough disk space available on the server for backup indices, so had to start from scratch. We will move to better server infra next week and then will be able to restore checkpoint backups of indices.

Will work on handling reorgs seamlessly in the future, but not a huge priority, as reorgs happen infrequently.

[0attack]

so the stated solution is for everyone to reindex from scratch which takes most people days and to expect to need to manually do that multiple times a year... got it. i guess the website will be working again in a couple of days at least. until it goes down from the same bug again...

[DanielHook15]

The problem might be even larger and reindex might not be a sufficient solution. As apezord mentions in https://github.com/casey/ord/issues/148, it is even possible that we lose artifacts.

[utxosvaldo]

A reorg just happened again at blockheight 783830.

Currently, there is no proposed solution other than building the index again from scratch.

[0attack]

wow not handling reorgs is a critical bug... could anyone do a pr to make the reindex either automatic or suggested by the software when a reorg is detected? at least that would somewhat improve things.

[schlunsen]

@0attack I'm looking forward to your pull request

[0attack]

@0attack I'm looking forward to your pull request

well i was asking if anyone else has the ability to do it. i'm not sure i know enough to be helpful, but i guess i'll give it a shot if no one else can. there should at least be a message informing the user a reindex is required...

[0attack]

here's my idea, hopefully it helps, but i don't think it's yet good enough for a pr...

for a better message, return Err(anyhow!("reorg detected at or before {prev_height}")); in the updater.rs file could be changed to return Err(anyhow!("reorg detected at or before {prev_height}, index needs to be rebuilt")); to at least inform users of what needs to be done.

for automatic rebuilding, i don't really know where the code should go, but something like this might work...

//run after reorg detection or at least check at start
if index.is_reorged() {
    //delete index due to reorg
    fs::remove_file("index.redb")?;
    //alert user that index was deleted due to reorg and to restart for reindexing
    //shut down if possible
}

[jun0tpyrc]

may need a more sustainable solution rather than full reindex , reorg detected at or before 789603