Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixup rare terms #64366

Merged
merged 2 commits into from
Oct 29, 2020
Merged

Fixup rare terms #64366

merged 2 commits into from
Oct 29, 2020

Conversation

nik9000
Copy link
Member

@nik9000 nik9000 commented Oct 29, 2020

In #64016 I caused a bug in rare_terms where we would rewrite the
current leaf and remove all hits from it, causing us to trip an
assertion. This didn't happen before because previously we never rewrote
the current leaf. The fix involves cleaning up the state that the
deferring collector uses if we end up removing all hits.

This requires some super intimate knowledge of how
BestBucketsDeferringCollector works so I decided to move the
implementation of that mergin from MergingBucketsDeferringCollector
into BestBucketsDeferringCollector. This means that
MergingBucketsDeferringCollector is pretty much empty now. So I've
deprecated it and will remove it entirely in an subsequent change.

Closes #64356

In elastic#64016 I caused a bug in rare_terms where we would rewrite the
current leaf and remove all hits from it, causing us to trip an
assertion. This didn't happen before because previously we never rewrote
the current leaf. The fix involves cleaning up the state that the
deferring collector uses if we end up removing all hits.

This requires some super intimate knowledge of how
`BestBucketsDeferringCollector` works so I decided to move the
implementation of that mergin from `MergingBucketsDeferringCollector`
into `BestBucketsDeferringCollector`. This means that
`MergingBucketsDeferringCollector` is pretty much empty now. So I've
deprecated it and will remove it entirely in an subsequent change.

Closes elastic#64356
Copy link
Member

@not-napoleon not-napoleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable to me.


/**
* Build the {@link DeferringBucketCollector}. The default implementation
* selects the best buckets but some aggs want to do something else.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you just transcribed this from the old comment, but it might be nice to define "best" here. Like what criteria it uses.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@nik9000 nik9000 merged commit b4b01ec into elastic:master Oct 29, 2020
@ywangd
Copy link
Member

ywangd commented Oct 30, 2020

I believe this PR needs to be backported. The same failure is still happening with branch 7.x: https://gradle-enterprise.elastic.co/s/ywuksrllpmfie
Re-openned #64383 for tracking

@ywangd
Copy link
Member

ywangd commented Oct 30, 2020

Another one (7.x) and this time is for testNestedTerms: https://gradle-enterprise.elastic.co/s/grg67mxulb23i
Relevant issue is #64356 and it is closed since it is fixed for master. I am leaving it closed since I already re-openned #64383 for tracking a similar failure.

@henningandersen
Copy link
Contributor

nik9000 added a commit to nik9000/elasticsearch that referenced this pull request Oct 30, 2020
In elastic#64016 I caused a bug in rare_terms where we would rewrite the
current leaf and remove all hits from it, causing us to trip an
assertion. This didn't happen before because previously we never rewrote
the current leaf. The fix involves cleaning up the state that the
deferring collector uses if we end up removing all hits.

This requires some super intimate knowledge of how
`BestBucketsDeferringCollector` works so I decided to move the
implementation of that mergin from `MergingBucketsDeferringCollector`
into `BestBucketsDeferringCollector`. This means that
`MergingBucketsDeferringCollector` is pretty much empty now. So I've
deprecated it and will remove it entirely in an subsequent change.

Closes elastic#64356
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (:Analytics/Aggregations)

@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Oct 30, 2020
@nik9000
Copy link
Member Author

nik9000 commented Oct 30, 2020

Sorry I hadn't opened the backport yesterday! I've just opened it and will merge when jenkins is happy.

nik9000 added a commit that referenced this pull request Oct 30, 2020
In #64016 I caused a bug in rare_terms where we would rewrite the
current leaf and remove all hits from it, causing us to trip an
assertion. This didn't happen before because previously we never rewrote
the current leaf. The fix involves cleaning up the state that the
deferring collector uses if we end up removing all hits.

This requires some super intimate knowledge of how
`BestBucketsDeferringCollector` works so I decided to move the
implementation of that mergin from `MergingBucketsDeferringCollector`
into `BestBucketsDeferringCollector`. This means that
`MergingBucketsDeferringCollector` is pretty much empty now. So I've
deprecated it and will remove it entirely in an subsequent change.

Closes #64356
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request Nov 9, 2020
Removes `MergingBucketsDeferringCollector` which I deprecated in elastic#64366.
@nik9000 nik9000 mentioned this pull request Nov 9, 2020
nik9000 added a commit that referenced this pull request Nov 11, 2020
Removes `MergingBucketsDeferringCollector` which I deprecated in #64366.
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request Nov 11, 2020
Removes `MergingBucketsDeferringCollector` which I deprecated in elastic#64366.
nik9000 added a commit that referenced this pull request Nov 11, 2020
Removes `MergingBucketsDeferringCollector` which I deprecated in #64366.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations >refactoring Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v7.11.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reproducible failure in RareTermsAggregatorTests
6 participants