Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DateHistogram bucket gap #2183

Merged
merged 2 commits into from
Sep 21, 2023
Merged

Fix DateHistogram bucket gap #2183

merged 2 commits into from
Sep 21, 2023

Commits on Sep 21, 2023

  1. Fix DateHistogram bucket gap

    Fixes a computation issue of the number of buckets needed in the
    DateHistogram.
    
    This is due to a missing normalization from request values (ms) to fast field
    values (ns), when converting an intermediate result to the final result.
    This results in a wrong computation by a factor 1_000_000.
    The Histogram normalizes values to nanoseconds, to make the user input like
    extended_bounds (ms precision) and the values from the fast field (ns precision for date type) compatible.
    This normalization happens only for date type fields, as other field types don't have precision settings.
    The normalization does not happen due a missing `column_type`, which is not
    correctly passed after merging an empty aggregation (which does not have a `column_type` set), with a regular aggregation.
    
    Another related issue is an empty aggregation, which will not have
    `column_type` set, will not convert the result to human readable format.
    
    This PR fixes the issue by:
    - Limit the allowed field types of DateHistogram to DateType
    - Instead of passing the column_type, which is only available on the segment level, we flag the aggregation as `is_date_agg`.
    - Fix the merge logic
    
    Add a flag to to normalization only once. This is not an issue
    currently, but it could become easily one.
    
    closes quickwit-oss/quickwit#3837
    PSeitz committed Sep 21, 2023
    Configuration menu
    Copy the full SHA
    7b0f53c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1a51c5f View commit details
    Browse the repository at this point in the history