Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Database::get_or_put: insert if value does not exist, otherwise return previous value #252

Merged
merged 7 commits into from
Apr 29, 2024

Conversation

nolanderc
Copy link
Contributor

@nolanderc nolanderc commented Apr 28, 2024

Pull Request

Related issue

N/A
Based on #251

What does this PR do?

  • Adds a new Database::get_or_put method which uses the MDB_NOOVERWRITE flag to simultaneously attempt to insert a key/value pair, or get the previous value if it already exists.

MDB_NOOVERWRITE - enter the new key/data pair only if the key does not already appear in the database. The function will return MDB_KEYEXIST if the key already appears in the database, even if the database supports duplicates (MDB_DUPSORT). The data parameter will be set to point to the existing item.

  • This is used as an optimization over the following pattern:
if let Some(value) = db.get(&txn, key) {
    // ...
} else {
    db.put(&mut txn, key, new_value);
}

which can now be written as (with one fewer lookups):

if let Some(value) = db.get_or_put(&mut txn, key, new_value) {
    // ...
}

One drawback of the above is that it would serialize/allocate the new_value even if it would not end up being inserted. This can be avoided using the get_or_put_reserved* methods, mirroring the put_reserved* method.

PR checklist

Please check if your PR fulfills the following requirements:

  • Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
  • Have you read the contributing guidelines?
  • Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

@nolanderc nolanderc marked this pull request as ready for review April 29, 2024 06:51
Copy link
Member

@Kerollmops Kerollmops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like what you've done. I am going to accept your work (mostly) as is.

However, I am wondering if I don't want to write a system similar to the HashMap::entry method, at some point 🤔 What do you think?

heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
Copy link
Member

@Kerollmops Kerollmops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So style issues but I prefer this way, like you convert before matching 😄

heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
heed/src/database.rs Outdated Show resolved Hide resolved
@nolanderc
Copy link
Contributor Author

However, I am wondering if I don't want to write a system similar to the HashMap::entry method, at some point 🤔 What do you think?

This could probably be done using LMDB's cursor API. Looking at the source, there is an optimization in case a cursor is already positioned on the correct data item for an insert. Thus, an entry-API could efficiently be implemented as (pseudocode):

mdb_cursor_get(&mut cursor, &mut key, &mut data, MDB_SET);
if data.is_none() {
    // ... encode value into `data` ...
    mdb_cursor_put(&mut cursor, &mut key, &mut data);
}

@Kerollmops
Copy link
Member

You are absolutely right, and I already worked on a cursor wrapper in the cursor.rs file, It can maybe be improved 🤔
Thank you for your changes 👍

@Kerollmops Kerollmops merged commit 3406383 into meilisearch:main Apr 29, 2024
8 checks passed
@Kerollmops Kerollmops added this to the v0.20.1 milestone Apr 29, 2024
@nolanderc nolanderc deleted the get-or-put branch April 29, 2024 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants