Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(beatree): introduce a read transaction type #603

Merged
merged 1 commit into from
Dec 9, 2024

Conversation

rphmeier
Copy link
Contributor

@rphmeier rphmeier commented Dec 6, 2024

This introduces a basic read transaction type (not currently used) that is a first step towards enabling a safe asynchronous API for beatree. The immediate motivations are:

  1. The removal of leaf children requires us to query many leaves asynchronously during merkle update.
  2. Likewise, the rollback log requires us to do the same in order to fetch previous values.

A less immediate but also useful goal is read transactions being used in Sessions to enable parallel VMs to operate with fewer threads. This is groundwork for that future.

Read transactions don't block commit or finish_sync (leaving room for "async sync" in the future), but do block sync from beginning. We can expect that sync will start at a natural moment where there are no outstanding read transactions (after a session is committed). When a sync will end is less predictable, and leaving some space for pipelining will be beneficial.

Copy link
Contributor Author

rphmeier commented Dec 6, 2024

@rphmeier rphmeier force-pushed the rh-basic-read-transaction branch from ae57b58 to 1b81432 Compare December 6, 2024 20:23
@rphmeier rphmeier changed the base branch from master to rh-THR-53-overflow-cell-value-hash December 6, 2024 20:23
nomt/src/beatree/mod.rs Outdated Show resolved Hide resolved
nomt/src/beatree/mod.rs Outdated Show resolved Hide resolved
nomt/src/beatree/mod.rs Outdated Show resolved Hide resolved
// be invalidated by any destructive changes.
//
// Once this guard drops, it will be safe to initiate a new read transaction.
let read_tx_guard = read_transaction_latch.block_until_unique();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent some time and I think I don't understand why this only blocks here but imediatelly released after we got the write. Why block then at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It ensures that no new read transactions will be created until the write lock on self.shared is released in any case.

It would be fine to hold read_tx_guard until the end of this scope as well, of course. But not blocking would lead to a race, where we block_until_unique but then a new read transaction is created by another thread before we can acquire shared.write().

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but why is it important to exclude the new readers only for the duration of this scope?

@rphmeier rphmeier force-pushed the rh-basic-read-transaction branch from 1b81432 to 1a981b5 Compare December 8, 2024 21:40
@pepyakin pepyakin changed the base branch from rh-THR-53-overflow-cell-value-hash to graphite-base/603 December 8, 2024 22:05
@pepyakin pepyakin force-pushed the rh-basic-read-transaction branch from 1a981b5 to e08ba49 Compare December 8, 2024 22:05
@pepyakin pepyakin changed the base branch from graphite-base/603 to master December 8, 2024 22:05
@pepyakin pepyakin force-pushed the rh-basic-read-transaction branch from e08ba49 to 6c5cbac Compare December 8, 2024 22:05
nomt/src/beatree/mod.rs Outdated Show resolved Hide resolved
nomt/src/beatree/mod.rs Outdated Show resolved Hide resolved
*self.inner.read_transactions.lock() += 1;
}

fn block_until_unique(&self) -> MutexGuard<usize> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this blocks until X gets unique.

What is X here?

// be invalidated by any destructive changes.
//
// Once this guard drops, it will be safe to initiate a new read transaction.
let read_tx_guard = read_transaction_latch.block_until_unique();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but why is it important to exclude the new readers only for the duration of this scope?

@pepyakin pepyakin merged commit 4ace8d0 into master Dec 9, 2024
8 checks passed
@pepyakin pepyakin deleted the rh-basic-read-transaction branch December 9, 2024 18:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants