[Transform] implement node.transform to control where to run a transform #52712
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Pinging @elastic/ml-core (:ml/Transform) |
hendrikmuhs
force-pushed
the
transform-node-attributes
branch
from
66081db to
71b427d
Compare
hendrikmuhs
changed the title
hendrikmuhs
commented
Feb 25, 2020
droberts195
reviewed
Feb 25, 2020
x-pack/plugin/transform/src/main/java/org/elasticsearch/xpack/transform/Transform.java
Outdated
Show resolved
Hide resolved
x-pack/plugin/transform/src/main/java/org/elasticsearch/xpack/transform/Transform.java
Show resolved
Hide resolved
...main/java/org/elasticsearch/xpack/transform/transforms/TransformPersistentTasksExecutor.java
Show resolved
Hide resolved
droberts195
reviewed
Feb 25, 2020
x-pack/plugin/transform/src/main/java/org/elasticsearch/xpack/transform/Transform.java
Outdated
Show resolved
Hide resolved
|
run elasticsearch-ci/2 |
droberts195
approved these changes
Feb 28, 2020
There was a problem hiding this comment.
LGTM if you could just change the 3 things I saw.
Also @lcawl would you like to check the setting descriptions for consistency with other ES docs? What's said is factually accurate (except for the tiny typo I noted), but maybe there are things you'd like changed related to consistency.
x-pack/plugin/transform/src/main/java/org/elasticsearch/xpack/transform/Transform.java
Show resolved
Hide resolved
...main/java/org/elasticsearch/xpack/transform/transforms/TransformPersistentTasksExecutor.java
Outdated
Show resolved
Hide resolved
szabosteve
reviewed
Feb 28, 2020
szabosteve
approved these changes
Feb 28, 2020
There was a problem hiding this comment.
The docs changes LGTM. I opened a PR for the redirects, will merge after this PR.
Redirect PR: #52944
|
run elasticsearch-ci/bwc |
…odes and test which nodes are allowed to do remote connections
Co-Authored-By: David Roberts <dave.roberts@elastic.co>
This reverts commit 870e716671680714a00728ef47c838747b27540d.
hendrikmuhs
force-pushed
the
transform-node-attributes
branch
from
25d66bc to
1a80758
Compare
|
run elasticsearch-ci/bwc |
This was referenced Mar 2, 2020
hendrikmuhs
pushed a commit
that referenced
this pull request
Mar 2, 2020
fix test logic in transform tests, regression introduced in #52712
hendrikmuhs
pushed a commit
that referenced
this pull request
Mar 2, 2020
hendrikmuhs
pushed a commit
that referenced
this pull request
Mar 17, 2020
hendrikmuhs
pushed a commit
that referenced
this pull request
Mar 17, 2020
hendrikmuhs
pushed a commit
that referenced
this pull request
Mar 11, 2021
The node attribute transform.node has been introduced in #52712 (7.7) to implement dedicated transform nodes. A technical limitation in the stack made it necessary to use this workaround. This limitation has been removed by #54998 (7.9). It is now possible to find out, if a node is a transform node without using the node attribute. This PR deprecates the node attribute and bases the placement decision on the node role. The attribute gets unused for placement, however we can not remove the node attribute yet, because in a mixed version cluster the attribute might still be used, especially when master gets updated last. The node attribute will be completely removed in 8, as the upgrade path requires to upgrade to 7.last before upgrading to 8.x.
hendrikmuhs
pushed a commit
that referenced
this pull request
Mar 11, 2021
…70291) The node attribute transform.node has been introduced in #52712 (7.7) to implement dedicated transform nodes. A technical limitation in the stack made it necessary to use this workaround. This limitation has been removed by #54998 (7.9). It is now possible to find out, if a node is a transform node without using the node attribute. This PR deprecates the node attribute and bases the placement decision on the node role. The attribute gets unused for placement, however we can not remove the node attribute yet, because in a mixed version cluster the attribute might still be used, especially when master gets updated last. The node attribute will be completely removed in 8, as the upgrade path requires to upgrade to 7.last before upgrading to 8.x. backport #70234
probakowski
pushed a commit
to probakowski/elasticsearch
that referenced
this pull request
Mar 15, 2021
The node attribute transform.node has been introduced in elastic#52712 (7.7) to implement dedicated transform nodes. A technical limitation in the stack made it necessary to use this workaround. This limitation has been removed by elastic#54998 (7.9). It is now possible to find out, if a node is a transform node without using the node attribute. This PR deprecates the node attribute and bases the placement decision on the node role. The attribute gets unused for placement, however we can not remove the node attribute yet, because in a mixed version cluster the attribute might still be used, especially when master gets updated last. The node attribute will be completely removed in 8, as the upgrade path requires to upgrade to 7.last before upgrading to 8.x.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
implement transform node attributes to disable transform on certain nodes and test which nodes are allowed to do remote connections
closes #52200
closes #50033
closes #48734
Docs preview: http://elasticsearch_52712.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/transform-settings.html