Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Transform] Do not use PIT in the presence of remote indices in source #99803

Merged
merged 3 commits into from
Sep 22, 2023

Conversation

przemekwitek
Copy link
Contributor

@przemekwitek przemekwitek commented Sep 22, 2023

Today transforms defaults to using point-in-time (PIT) readers. These can make the transform more efficient in many scenarios, however, there are some where they can make the transform considerably less efficient.

One of such scenarios is when some of the source indices are on remote clusters.
This PR disables PIT in such a case.

Relates #99742

@przemekwitek przemekwitek removed the WIP label Sep 22, 2023
@przemekwitek przemekwitek marked this pull request as ready for review September 22, 2023 10:15
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@elasticsearchmachine elasticsearchmachine added the Team:ML Meta label for the ML team label Sep 22, 2023
@elasticsearchmachine
Copy link
Collaborator

Hi @przemekwitek, I've created a changelog YAML for you.

Copy link
Contributor

@droberts195 droberts195 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
8.10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :ml/Transform Transform Team:ML Meta label for the ML team v8.10.3 v8.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants