Make consistentHashing to a generic tools class #17718
Conversation
|
Automated checks report:
Some checks failed. Please fix the reported issues and reply 'alluxio-bot, check this please' to re-run checks. |
maobaolong
changed the title
|
Automated checks report:
All checks passed! |
| * | ||
| * @param nodes the nodes to select | ||
| */ | ||
| public ModularHashingNodeProvider(List<T> nodes) { |
There was a problem hiding this comment.
what is the use case for this hash+mode algorithm?
There was a problem hiding this comment.
Modular hash is simple fast and do not care the node id, ConsistentHashing is complex and have to maintain the node list dynamically.
We use these two algorithm to supply a fake file block location, so that the application can dispatch the split to the node what we want.
There was a problem hiding this comment.
Modular hash does not support dynamic worker membership, so I imagine you have:
- If you don't expect dynamic worker membership, modular hash or this jump consistent hash Support Various Kinds of Consistent Hash #17817 may be your choice
- If the worker memberships may change and you want consistent cache hits, consistent hash will be your choice.
I don't think I fully understood how you fake block locations by modular operation. But I think sure you can implement this, and let us know how that works out in your environment.
In #17817 we have extracted WorkerLocationPolicy interface. Could you move your implementation under WorkerLocationPolicy ? Thanks!
apc999
force-pushed
the
main
branch
2 times, most recently
from
2fec0ec
to
b597c61
Compare
What changes are proposed in this pull request?
Make ConsistentHashing to a generic tool class, so that we can use it in various place.