Add bucket property node_confirms for physical diversity
#1663
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Briefly, some Riak deployments require that a write is on multiple physical nodes before they're safe. Given a good ring, using Primaries is one way to ensure this physical diversity is to use `pw=2` (for example) to ensure two physical nodes for the write. This has an availability cost. With 5 nodes and 2 nodes down storing on 2 physical nodes is still very possible, but 2 primaries will leave a range of preflists unavailable to write. This commit adds a bucket property and put option of `node_confrims` to the declare the number of diverse physical node acks required for a write to be successful. For the above example `node_confirms=2` would be used, and as long as any two diverse nodes (primary OR fallback) reply, the write is a success.
Description and justification for new node_confirms option.
When adding `node_confirms` a node confirms failure threshold was also added, this brought the arity to 11 for put core init. All quorum failure thresholds are calculated as `N+Q-1`. This commit moves that calculation inside the put core. The arity of init is still 8, which is high.
Take it or leave it, but it's easier to edit without mega length lines, imo.
Lessens the probability that the links will become broken.
objects/counters/datatypes all
Typed buckets do not automatically inherit newly added default properties (old-skool default buckets do thanks to the existence of bucket fixups.) As per the bug report, when PUTing to a Typed bucket after upgrade to 2.2.5 the call to riak_kv_util:expand_rw_value/4 throws if the property `node_confirms` is not in the bucket properties, and for a typed bucket created and activated before upgrade it won't be. This is somewhat contentious. Ex-Basho @seancribbs has pointed out that there is a reason for that throw, and that all nodes should agree on the value, and misconfiguration can be an issue here. There are perhpas more complexities (we're retroactivley adding a reserved word, for example.) Caveats in docs and probability should guard against that latter though. Given that the reasons not to do this are lost to time, and the alternatives unpleasant, I'm content that this is a reasonable way for typed buckets to inherit **NEW** properties added as defaults after typed bucket creation. There is an associated riak_core commit.
This was referenced Feb 22, 2018
ghost
approved these changes
Feb 26, 2018
|
+1 |
|
@bryanhuntesl re vsn 1.4 counter, it will be gone in 3.0 |
|
Neat fix on the pre-upgrade bucket-types. 👍 +1 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is an upstream PR of a few related, merged, PRs/branches from the nhs-riak fork.
The original PRs are:
See doc/Node-Diversity.md for full details.
Briefly, some Riak deployments require that a write is on multiple
physical nodes before they're safe.
Given a good ring, using Primaries is one way to ensure this physical
diversity is to use pw=2 (for example) to ensure two physical nodes
for the write. This has an availability cost. With 5 nodes and 2 nodes
down storing on 2 physical nodes is still very possible, but 2
primaries will leave a range of preflists unavailable to write. This
commit adds a bucket property and put option of
node_confirmstodeclare the number of diverse physical node acks required for a write
to be successful. For the above example
node_confirms=2would beused, and as long as any two diverse nodes (
primaryORfallback)reply, the write is a success.
Further work fixed a related bug (see the commit message for b23b070 for details.
There are associated PB, riakc, httpc, riak-test, and riak_core PRs:
node_confirmsdefault bucket props riak_core#915node_confirmsoption to write messages riak_pb#228node_confirmsbucket property riak_test#1298node_confirmsas put option riak_test#1299node_confirmsoption on PUTS riak-erlang-http-client#69node_confirmsoption to PUT request riak-erlang-client#371