Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

output/cloudv2: Flush chunks #3108

Merged
merged 1 commit into from
Jun 9, 2023
Merged

output/cloudv2: Flush chunks #3108

merged 1 commit into from
Jun 9, 2023

Conversation

codebien
Copy link
Contributor

@codebien codebien commented Jun 1, 2023

If the number of time series is higher than the maximum batch size than split them in chunks.

Part of #3117

@codebien codebien added this to the v0.46.0 milestone Jun 1, 2023
@codebien codebien self-assigned this Jun 1, 2023
@codecov-commenter
Copy link

codecov-commenter commented Jun 1, 2023

Codecov Report

Merging #3108 (7fd710a) into master (0a31512) will increase coverage by 0.07%.
The diff coverage is 55.55%.

❗ Current head 7fd710a differs from pull request most recent head 6e63afd. Consider uploading reports for the commit 6e63afd to get more accurate results

@@            Coverage Diff             @@
##           master    #3108      +/-   ##
==========================================
+ Coverage   73.74%   73.82%   +0.07%     
==========================================
  Files         241      243       +2     
  Lines       18460    18474      +14     
==========================================
+ Hits        13614    13638      +24     
+ Misses       3972     3966       -6     
+ Partials      874      870       -4     
Flag Coverage Δ
ubuntu 73.74% <55.55%> (-0.01%) ⬇️
windows 73.64% <55.55%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cloudapi/config.go 89.47% <ø> (ø)
output/cloud/expv2/flush.go 84.61% <50.00%> (+0.74%) ⬆️
output/cloud/expv2/output.go 84.33% <100.00%> (+0.19%) ⬆️

... and 11 files with indirect coverage changes

This was referenced Jun 7, 2023
@codebien codebien marked this pull request as ready for review June 8, 2023 12:18
@github-actions github-actions bot requested review from imiric and mstoykov June 8, 2023 12:19
Comment on lines +44 to +46
if len(msb.seriesIndex) < f.maxSeriesInSingleBatch {
continue
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was bit weary of an if in this loop ... but to be honest addTimeBucket is so much more complicated that this is likely not an issue.

If it is we can always optimize it.

mstoykov
mstoykov previously approved these changes Jun 8, 2023
Copy link
Contributor

@mstoykov mstoykov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Left a nitpick that is unrelated to the current PR

Copy link
Contributor

@imiric imiric left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't fully grok the new cloud output, but this change LGTM.

output/cloud/expv2/flush.go Show resolved Hide resolved
@codebien codebien merged commit 9254ce7 into master Jun 9, 2023
@codebien codebien deleted the cloud-v2-flush-chunks branch June 9, 2023 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants