fixes race condition in Searcher

[PSeitz]

fixes #1461
fixes a race condition in Searcher, by avoiding repeated calls to open_segment_readers and passing them instead as argument

[fulmicoton]

@shikhar can you review #1464

[codecov-commenter]

Codecov Report

Merging #1464 (227c252) into main (67d94f5) will decrease coverage by 0.00%.
The diff coverage is 95.65%.

@@            Coverage Diff             @@
##             main    #1464      +/-   ##
==========================================
- Coverage   94.07%   94.06%   -0.01%     
==========================================
  Files         238      239       +1     
  Lines       44564    44636      +72     
==========================================
+ Hits        41922    41989      +67     
- Misses       2642     2647       +5     

Impacted Files	Coverage Δ
src/core/searcher.rs	77.93% <85.71%> (+0.39%)	⬆️
src/reader/mod.rs	92.94% <100.00%> (-0.05%)	⬇️
src/fastfield/serializer/mod.rs	90.47% <0.00%> (-1.09%)	⬇️
src/fastfield/writer.rs	90.80% <0.00%> (-0.29%)	⬇️
fastfield_codecs/src/lib.rs	96.87% <0.00%> (-0.03%)	⬇️
src/query/mod.rs	100.00% <0.00%> (ø)
src/query/query.rs	85.71% <0.00%> (ø)
src/query/boost_query.rs	69.76% <0.00%> (ø)
fastfield_codecs/src/bitpacked.rs	100.00% <0.00%> (ø)
src/query/disjunction_max_query.rs	0.00% <0.00%> (ø)
... and 14 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[shikhar]

@PSeitz aside from clippy nits, LGTM

I did have one unrelated change I was sneaking into my PR, that I wanted to ask about.

Is Relaxed memory order safe here? Both a load and store are happening, so AcqRel seems safer to me. And this is by no means a hot path so being conservative seems fair.

tantivy/src/reader/mod.rs

Line 212 in 998b126

let generation_id = searcher_generation_counter.fetch_add(1, atomic::Ordering::Relaxed);

[shikhar]

The previous name seems better to me, since we are creating a SearcherGeneration and making sure to do it in a way that we track it int the inventory, and the inventory tracks SearcherGeneration not segment readers.

[PSeitz]

my issue with the previous name was that it's not clear that inventory is used to track and not the source of the generation

[PSeitz]

@PSeitz aside from clippy nits, LGTM

I did have one unrelated change I was sneaking into my PR, that I wanted to ask about.

Is Relaxed memory order safe here? Both a load and store are happening, so AcqRel seems safer to me. And this is by no means a hot path so being conservative seems fair.

tantivy/src/reader/mod.rs

Line 212 in 998b126

let generation_id = searcher_generation_counter.fetch_add(1, atomic::Ordering::Relaxed);

Yeah, I think there could be scenarios where simultaneous reloads are triggered at the same time from multiple threads.

[fulmicoton]

@shikhar The operation is still Atomic... I think we just want to ensure that the id delivered are unique, and Relaxed is sufficient for that. AcqRel stuff is about instruction reordering.

[PSeitz]

@shikhar I think we just want to ensure that the id delivered are unique, and Relaxed is sufficient for that... That being said it is not a hot part of the code. We can use SeqCst and not sweat it.

I think it could be an issue when two threads increment, but due to weak fencing, we get two times the same value back. If now one of those callers has a new segment in their list, we have the same id to different SearcherGeneration.

It's very unlikely though, and on many platforms Relaxed is already AcqRel.

[adamreichold]

I think it could be an issue when two threads increment, but due to weak fencing, we get two times the same value back.

The memory orderings mostly determine how other memory accesses are ordered w.r.t. the atomic accesses, they do not affect the atomicity of an operation like fetch_add, i.e. multiple threads doing fetch_add(1, Ordering::Relaxed) will never see identical values (ignoring overflow).

However, their accesses to other memory locations that are shared with other threads will not be ordered w.r.t. the ID generation, e.g. their accesses to searcher_generation_inventory are do not necessarily reflect their accesses to searcher_generation_counter, meaning that a thread with lower ID might see the stores to the inventory of a thread with a higher ID.

While I suspect that this is not an issue (this would have involve unsafe code if I understand things correctly), my personal choice would be Ordering::AcqRel as well just to capture any implicit dependencies which I do not see ATM. (I would suggest avoiding Ordering::SeqCst as mixing those two orderings can have unexpected effects and sticking to AcqRel is often preferable if global ordering is not required as it is simpler to reason about the explicit pairings between acquire and release operations to the same memory address.)

[fulmicoton]

It's very unlikely though, and on many platforms Relaxed is already AcqRel.

This is a very bad argument :). Also ARM is pretty popular.

we get two times the same value back

You cannot get two times the same value back... That's not what ordering is about.
The explanation here is ok:
https://en.cppreference.com/w/cpp/atomic/memory_order

...

Anyway let's defensively go for Ordering::AcqRel...

[shikhar]

Thanks folks, I need to learn more about memory orderings :)