Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable importing from dumps v1 #3170

Merged
merged 8 commits into from
Dec 1, 2022
Merged

Re-enable importing from dumps v1 #3170

merged 8 commits into from
Dec 1, 2022

Conversation

dureuill
Copy link
Contributor

@dureuill dureuill commented Nov 30, 2022

Pull Request

Related issue

Fixes #2985

What does this PR do?

User standpoint

  • Allows importing dumps version 1 (exported with Meilisearch <=v0.20) to modern-day Meilisearch
  • Tasks of type "Customs" are skipped, with a warning
  • Tasks of status "enqueued" are skipped, with a warning
  • The "WordsPosition" ranking rule is skipped when encountered in the ranking rules, with a warning.

After an import from a v1 dump, it is recommended that a user checks each index and its settings.

Implementation standpoint

  • Add a dump v1 reader based on the one by @irevoire
  • Add a v1_to_v2 compatibility layer based on the v2_to_v3 one
    • as v2 requires UUIDs, the v1 indexes are mapped to UUIDs built from their position in the metada file: the first index is given UUID all zeroes, the second one UUID 00000000-0000-0000-0000-000000000001, and so on... This should have no bearing on the final indexes because v6 is not using UUIDs, but this allows us to correctly identify which tasks belong to which index.
  • Modify the v2_to_v3 compatibility layer to account for the fact that the reader can actually be a v1_to_v2 compat layer
  • Make some base dump types Clone
  • impl Display for v2::settings::Criterion

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

@dureuill dureuill added this to the v1.0.0 milestone Dec 1, 2022
@dureuill dureuill changed the title WIP: Reenable dump v1 Re-enable importing from dumps v1 Dec 1, 2022
@dureuill dureuill marked this pull request as ready for review December 1, 2022 11:01
@dureuill dureuill requested a review from irevoire December 1, 2022 11:01
Copy link
Member

@irevoire irevoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, it looks super good.

Could you add one test at the bottom of dump/src/reader/mod.rs that imports our dump v1 through all our compat layers, please? 😁

dump/src/error.rs Show resolved Hide resolved
dump/src/reader/v1/mod.rs Show resolved Hide resolved
dump/src/reader/v1/mod.rs Outdated Show resolved Hide resolved
dump/src/reader/compat/v1_to_v2.rs Show resolved Hide resolved
@dureuill
Copy link
Contributor Author

dureuill commented Dec 1, 2022

Could you add one test at the bottom of dump/src/reader/mod.rs that imports our dump v1 through all our compat layers, please? 😁

Oh sorry I didn't write a message to specify, but I did that.

Also I detailed some of the quirks of the current implementation, format-wise, in #2985 (comment)

Copy link
Member

@irevoire irevoire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, we're good to me!

bors merge

@bors
Copy link
Contributor

bors bot commented Dec 1, 2022

🕐 Waiting for PR status (GitHub check) to be set, probably by CI. Bors will automatically try to run when all required PR statuses are set.

@irevoire irevoire modified the milestones: v1.0.0, v0.29.3 Dec 1, 2022
@irevoire
Copy link
Member

irevoire commented Dec 1, 2022

bors merge

@bors
Copy link
Contributor

bors bot commented Dec 1, 2022

Stopped waiting for PR status (GitHub check) without running due to duplicate requests to run. You may check Bors to see that this PR is included in a batch by one of the other requests.

@bors bors bot merged commit 4033991 into main Dec 1, 2022
@bors
Copy link
Contributor

bors bot commented Dec 1, 2022

@bors bors bot deleted the reenable_dump_v1 branch December 1, 2022 17:51
@meili-bot meili-bot added the v1.0.0 PRs/issues solved in v1.0.0 released on 2023-02-06 label Feb 7, 2023
@irevoire irevoire added version upgrade Anything related to the way we update a meilisearch instance enhancement New feature or improvement labels Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or improvement v1.0.0 PRs/issues solved in v1.0.0 released on 2023-02-06 version upgrade Anything related to the way we update a meilisearch instance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add the support of the dump v1 (Meilisearch v0.20.0 and before)
3 participants