Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: expose migration check #2074

Merged
merged 3 commits into from
Mar 21, 2024

Conversation

wjones127
Copy link
Contributor

@wjones127 wjones127 commented Mar 14, 2024

Exposes a utility function to check whether a Dataset's manifest needs a migration to be considered the latest version. Most users have no business calling this, so it's kept in the private module.

@wjones127 wjones127 marked this pull request as ready for review March 15, 2024 22:45
Copy link
Contributor

@westonpace westonpace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. This will be handy to have in our back pocket if needed.

Fo you also want to check if any indices are missing their fragment bitmap? Or is that redundant given the other checks?

let dataset_ref = &dataset.as_ref(py).borrow().ds;
let manifest = RT
.block_on(Some(py), dataset_ref.latest_manifest())?
.map_err(|err| PyIOError::new_err(format!("Failed to connect to data: {}", err)))?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems a little odd as an error message?

@wjones127
Copy link
Contributor Author

Fo you also want to check if any indices are missing their fragment bitmap? Or is that redundant given the other checks?

Oh this is for index migration? I should check the conditions for that too.

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 33.33333% with 14 lines in your changes are missing coverage. Please review.

❗ No coverage uploaded for pull request base (main@23ecd50). Click here to learn what that means.

Files Patch % Lines
rust/lance/src/io/commit.rs 33.33% 14 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2074   +/-   ##
=======================================
  Coverage        ?   80.70%           
=======================================
  Files           ?      160           
  Lines           ?    47127           
  Branches        ?    47127           
=======================================
  Hits            ?    38035           
  Misses          ?     6931           
  Partials        ?     2161           
Flag Coverage Δ
unittests 80.70% <33.33%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@wjones127 wjones127 merged commit 8c261c4 into lancedb:main Mar 21, 2024
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants