Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid logging errors due to RequestCanceled errors #465

Closed
leth opened this issue Jun 19, 2017 · 3 comments
Closed

Avoid logging errors due to RequestCanceled errors #465

leth opened this issue Jun 19, 2017 · 3 comments

Comments

@leth
Copy link
Contributor

leth commented Jun 19, 2017

There are a lot of errors logged when query requests are cancelled.
This is normal behaviour and should not be logged, as it masks genuine errors.

We should, however, ensure that these errors are included in a metric somewhere.

@leth
Copy link
Contributor Author

leth commented Jun 20, 2017

We see this when people have dashboards open, refreshing at a rate faster than cortex can respond to.

@bboreham
Copy link
Contributor

I see a lot of these, which I think are down to the distributor cancelling when it has (N-1) answers:

level=warn ts=2019-02-04T08:55:28.07467732Z caller=gokit.go:46 method=/cortex.Ingester/Query duration=97.494µs err="context canceled" msg="gRPC\n"
level=warn ts=2019-02-04T08:55:28.075459936Z caller=gokit.go:46 method=/cortex.Ingester/Query duration=89.658µs err="context canceled" msg="gRPC\n"
level=warn ts=2019-02-04T08:55:28.075643591Z caller=gokit.go:46 method=/cortex.Ingester/Query duration=70.858µs err="context canceled" msg="gRPC\n"

Seems like the same issue - we don't want to log "context canceled" all day.
Nor do we want it flagged as an error in Jaeger.

@bboreham
Copy link
Contributor

Closed by #1339

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants