Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[aggregator] Add M3Msg client and server for M3Aggregator #2171

Merged
merged 17 commits into from
Apr 19, 2020

Conversation

robskillington
Copy link
Collaborator

What this PR does / why we need it:

This adds the ability for users to use M3Msg to write and consume.

Special notes for your reviewer:

Does this PR introduce a user-facing and/or backwards incompatible change?:

NONE

Does this PR require updating code package or user-facing documentation?:

NONE

union *encoding.UnaggregatedMessageUnion,
msg consumer.Message,
) error {
defer msg.Ack()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all of the below error cases non-retryable? If we unconditionally ack we'll never retry from the coordinator client to the aggregator right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we probably should avoid retries on this, if aggregator gets overloaded the amount of buffering/retries will probably quickly overwhelm the coordinators.

@codecov
Copy link

codecov bot commented Feb 26, 2020

Codecov Report

Merging #2171 into master will not change coverage by %.
The diff coverage is n/a.

Impacted file tree graph

@@          Coverage Diff           @@
##           master   #2171   +/-   ##
======================================
  Coverage    71.9%   71.9%           
======================================
  Files        1024    1024           
  Lines       89726   89726           
======================================
  Hits        64581   64581           
  Misses      20827   20827           
  Partials     4318    4318           
Flag Coverage Δ
#aggregator 77.6% <0.0%> (ø)
#cluster 84.8% <0.0%> (ø)
#collector 82.8% <0.0%> (ø)
#dbnode 79.2% <0.0%> (ø)
#m3em 74.4% <0.0%> (ø)
#m3ninx 74.4% <0.0%> (ø)
#m3nsch 51.1% <0.0%> (ø)
#metrics 17.5% <0.0%> (ø)
#msg 74.9% <0.0%> (ø)
#query 68.2% <0.0%> (ø)
#x 83.1% <0.0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02d1877...48beec5. Read the comment docs.

Copy link
Contributor

@notbdu notbdu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments.

Changes overall LGTM but I don't have any context on things like why the legacy protocol has to watch for placements while the new m3msg protocol does not.

src/aggregator/client/client.go Show resolved Hide resolved
src/aggregator/server/m3msg/server.go Show resolved Hide resolved
src/cmd/services/m3aggregator/serve/serve.go Show resolved Hide resolved
@robskillington robskillington merged commit 7c81c68 into master Apr 19, 2020
@justinjc justinjc deleted the r/add-m3msg-aggregator-client-server branch May 14, 2020 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants