add wm module for GETing (and encoding) active preflist #1077
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jan 29, 2015
Closed
zeeshanlakhani
force-pushed
the
feature/zl/add-wm-code-for-getting-active-preflist
branch
from
c22a4a2
to
b0bfb85
Compare
zeeshanlakhani
force-pushed
the
feature/zl/add-wm-code-for-getting-active-preflist
branch
from
b0bfb85
to
3e2036e
Compare
This was referenced Jan 31, 2015
hmmr
added a commit
that referenced
this pull request
Feb 7, 2015
I used this branch (see PR #1077): feature/zl/add-wm-code-for-getting-active-preflist, which provides useful functions riak_core_apl:get_apl_ann() and friends. There is a matching commit in a newly created branch feature/az/api-entrypoints-coverage, for the pb part to become functional. The following HTTP requests are accepted: /ring/coverage/bucket/B/key/K/?proto=P returning a JSON of the form: {Host:{"ports":[Port]}} indicating Host:Port is where an entry point is for optimal data access to the key K in bucket B, via the given protocol API. Responses are cached for (a default of) 15 secs, configurable via parameter 'ring_vnodes_cache_expiry_time'. For pb, two new messages are provided: RpbApiEntryPointsReq (code 90) and RpbApiEntryPointsResp(code 91), with the relevant snippet in riak_pb/src/riak_kv.proto: message RpbApiEntryPointsReq { required bytes bucket = 1; required bytes key = 2; enum RpbApiProto { PB = 0; HTTP = 1; } optional RpbApiProto proto = 3 [default = PB]; } message RpbApiEntryPointsResp { required bytes host = 1; required int32 port = 2; } This coverage information is obtained in two steps: 1. using riak_core_apl:get_apl_ann(), get active preflist for given Bucket and Key, and extract riak nodes from it; 2. make a rpc call on those nodes to determine which of the relevant listener apps (i.e., riak_api_pb_listener, riak_api_web) are running on those nodes.
hmmr
added a commit
that referenced
this pull request
Feb 13, 2015
I used this branch (see PR #1077): feature/zl/add-wm-code-for-getting-active-preflist, which provides useful functions riak_core_apl:get_apl_ann() and friends. There is a matching commit in a newly created branch feature/az/api-entrypoints-coverage, for the pb part to become functional. The following HTTP requests are accepted: /ring/coverage/bucket/B/key/K/?proto=P returning a JSON of the form: {Host:{"ports":[Port]}} indicating Host:Port is where an entry point is for optimal data access to the key K in bucket B, via the given protocol API. Responses are cached for (a default of) 15 secs, configurable via parameter 'ring_vnodes_cache_expiry_time'. For pb, two new messages are provided: RpbApiEpReq (code 90) and RpbApiEpResp(code 91), with the relevant snippet in riak_pb/src/riak_kv.proto: message RpbApiEpReq { required bytes bucket = 1; required bytes key = 2; enum RpbApiProto { pbc = 0; http = 1; } optional RpbApiProto proto = 3 [default = pbc]; } message RpbApiEpResp { required bytes host = 1; required int32 port = 2; } This coverage information is obtained in two steps: 1. using riak_core_apl:get_apl_ann(), get active preflist for given Bucket and Key, and extract riak nodes from it; 2. make a rpc call on those nodes to determine which of the relevant listener apps (i.e., riak_api_pb_listener, riak_api_web) are running on those nodes. The corresponding new riak python client method is client.get_api_entry_point(bucket, key), included in basho/riak-python-client@5921a8cc1
hmmr
added a commit
that referenced
this pull request
Feb 16, 2015
I used this branch (see PR #1077): feature/zl/add-wm-code-for-getting-active-preflist, which provides useful functions riak_core_apl:get_apl_ann() and friends. There is a matching commit in a newly created branch feature/az/api-entrypoints-coverage, for the pb part to become functional. The following HTTP requests are accepted (values for parameter P are "http", "pbc", assuming the latter by default): 1) /ring/coverage/bucket/B/key/K/?proto=P returning a JSON of the form: {Host:Port} indicating Host:Port is where an entry point is for optimal data access to the key K in bucket B, via the given protocol API. 2) /ring/coverage/bucket/B/key/K/?proto=P returning a JSON of the form: [{Host:Port}] providing a complete API entry point map for access to the entire cluster via that protocol. Responses are cached for (a default of) 15 secs, configurable via parameter 'ring_vnodes_cache_expiry_time'. For pb, two new messages are provided: RpbApiEpReq (code 90), RpbApiEpResp(code 91), RpbApiEpMapReq (code 92), RpbApiEpMapResp(code 93), with the relevant snippet in riak_pb/src/riak_kv.proto: enum RpbApiProto { pbc = 0; http = 1; } message RpbApiEp { required bytes host = 1; required int32 port = 2; } message RpbApiEpReq { required bytes bucket = 1; required bytes key = 2; optional RpbApiProto proto = 3 [default = pbc]; } message RpbApiEpResp { optional RpbApiEp ep = 1; } message RpbApiEpMapReq { optional RpbApiProto proto = 1 [default = pbc]; } message RpbApiEpMapResp { repeated RpbApiEp eplist= 1; } The coverage information is obtained in two steps: 1. using riak_core_apl:get_apl_ann(), get active preflist for given Bucket and Key, and extract riak nodes from it; 2. make a rpc call on those nodes to determine which of the relevant listener apps (i.e., riak_api_pb_listener, riak_api_web) are running on those nodes. The corresponding new riak python client methods are * client.get_api_entry_point(bucket, key, proto), * client.get_api_entry_points_map(proto) included in github.com/basho/riak-python-client/tree/feature/az/api-entrypoints-coverage
This was referenced Feb 16, 2015
|
Closing... will open a PR against develop. |
|
My piggyback PR #1084. On Mon, Feb 16, 2015 at 9:26 PM, Zeeshan Lakhani notifications@github.com
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note: Bors will fail until basho/riak_core#705 and co is reviewed/merged.
Update for webmachine.
Description
Part of RIAK-1481.
The active preflist will return primary/fallback partitions and nodes for the available nodes at the time of query. Primary/fallback will be annotated.
This involves updates to our RiakKV WB code, RiakPB, RiakCore, RiakAPI, Riak-Erlang-Http-Client, Riak-Erlang-Client.
*The impetus for adding this to the API started with a mailing list question answered by Charlie Voiselle, http://lists.basho.com/pipermail/riak-users_lists.basho.com/2015-January/016527.html, which involved a snippet of code that we've given clients multiple times. This was then discussed with Russell Brown on HipChat, https://basho.hipchat.com/history/room/867200/2015/01/13?q=enterprising&t=rid-867200#12:23:22.
Other PRs in the series: