Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ein t hours -fl based ODB panics #536

Merged
merged 6 commits into from
Sep 21, 2022
Merged

ein t hours -fl based ODB panics #536

merged 6 commits into from
Sep 21, 2022

Conversation

Byron
Copy link
Member

@Byron Byron commented Sep 20, 2022

The above invocation stresses the ODB in never before seen ways, apparently, as it extracts every single object ever seen. On repos that have multiple packs the parallel pack loading logic fails in ways that cause objects not to be found (reproducibly) or to trigger a panic.

Observations

  • single pack repos work with no issues
  • single-threaded access is fine
  • one or more packs and multi-threaded operation cause panics to hit, crashing all threads at different times even though by then all packs must have been loaded already.
  • It happens only with multi-indices (so removing the multi-index makes the issue disappear), and is related to recursion code that happens if an object refers to a base which is outside of its pack. Maybe not even related to multi-threading. Probably not related to multi-threading.

…lti-threading is requested) (#536)

This avoids duplicating test-runs.
It's supposed to happen when multiple threads, maybe, trigger the
loading of packs at the same time. There are 15 packs to load
here but apparently that doesn't manage to trigger it.
…ct resolution. (#536)

Hard to understand, but the gist seems to be that looking up the base
onbject recursively changes the snapshot layout which makes the
follow-up lookup see a different index, which then won't have the
original id.

To circumvent, we will try to reuse it, but resort to a full search
afterwards.

Unfortunately there is no way to reproduce this, it's just the very
special linux kernel pack I have here.
@Byron Byron merged commit b862fc5 into main Sep 21, 2022
@Byron Byron deleted the fix-odb-race branch September 21, 2022 04:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant