Storage migration encounters non-existent slabs

[turbolent]

Description

Both the atree storage health check that is run on the state before the migration, and the migration itself, encounter slabs that reference other slabs that cannot be found.

For example:

ERR failed to run StorageMigration in account c843c1f5a4805c3a, domain contract, key EnumTest: slab (0x0.34) not found: slab not found for stored value
goroutine 1786 [running]:
runtime/debug.Stack()
	/opt/homebrew/Cellar/go@1.20/1.20.14/libexec/src/runtime/debug/stack.go:24 +0x65
github.com/onflow/cadence/migrations.(*StorageMigration).MigrateNestedValue.func1()
	/Users/bastian/go/pkg/mod/github.com/onflow/cadence@v1.0.0-preview.14/migrations/migration.go:178 +0xd4
panic({0x495520, 0x1337dfd11d0})
	/opt/homebrew/Cellar/go@1.20/1.20.14/libexec/src/runtime/panic.go:884 +0x213
github.com/onflow/cadence/runtime/interpreter.DictionaryIterator.Next({0xf8b0c4f128?}, {0x0, 0x0})
	/Users/bastian/go/pkg/mod/github.com/onflow/cadence@v1.0.0-preview.14/runtime/interpreter/value.go:18808 +0x105
github.com/onflow/cadence/migrations.(*StorageMigration).MigrateNestedValue(0xe23d837b50, {{0x66954e, 0x8}, {0xc8, 0x43, 0xc1, 0xf5, 0xa4, 0x80, 0x5c, ...}}, ...)

Another example is /storage/FanTopPermission in 0x5d63c34d7f05e5a4

In particular, the referenced non-existent slabs all seem to have address 0x0, i.e. the temporary/"stack" account.

While investigating the issue, this seems to be a result of bug fixed in #1565.

Suggested Solution

Detect such broken values, and if they are dictionaries keyed by an enum, "repair" them, by skipping broken entries when iterating over dictionaries in the storage migration.

[turbolent]

@fxamacker How could we delete the broken ordered map here? It potentially consists of multiple slabs

[fxamacker]

@fxamacker How could we delete the broken ordered map here? It potentially consists of multiple slabs

@turbolent I looked into this and will open a PR in atree tomorrow.

EDIT: These PRs were opened and merged in onflow/atree:

• Add feature to enable migrations to fix references to non-existent registers atree#387
• Add feature to enable atree inlining migration to fix references to non-existent registers  atree#388

[turbolent]

@fxamacker Re-assigned to you, given you're working on the new migration (and also added the feature needed for it to atree)