Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amend Retry-After parsing logic for negative durations #283

Merged
merged 3 commits into from
Jun 20, 2024

Conversation

tpaschalis
Copy link
Member

@tpaschalis tpaschalis commented Jun 7, 2024

The initial implementation of this PR attempted to fix the TestExtractRetryAfterHeaderHttpDate test by making sure the test duration was in the future. After the comment by Tigran I've pivoted to fix the parsing logic instead.

This PR amends the parseHTTPDate and parseDelaySeconds helpers to not return an error if the parsed duration is in the past; instead they now return a zero duration and nil error to trigger immediate polling.

In the current logic and the two places (1, 2) it's being used, it looks that in the case that a negative seconds or a date in the past was set in the header, both the HTTP and WebSocket clients would reuse the last valid calculated interval, while now it would correctly poll right away.

Old PR description

This PR fixes a timing issue with TestExtractRetryAfterHeaderHttpDate that makes it flaky.

To see the failure, run the following command a couple of times

$ go test -run TestExtractRetryAfterHeaderHttpDate -count=20000 -shuffle=on ./...

The failure itself has to do with the following snippet; technically we're only adding >=0 seconds of delay. If you're lucky enough to add 0s as the duration, then the time.Until condition hits and the header cannot be parsed properly

retryIntervalSec := rand.Intn(9999)
expectedDuration := time.Second * time.Duration(retryIntervalSec)

if err == nil {
if duration := time.Until(t); duration > 0 {
return duration, nil
}
}
return 0, errCouldNotParseRetryAfterHeader

Fixes #181

Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
@tpaschalis tpaschalis requested a review from a team June 7, 2024 22:44
@tigrannajaryan
Copy link
Member

Hmm, I am not sure the test is wrong. I wonder if the code is wrong. Why do we need the date returned in Retry-After to be in the future? I would argue that is not necessary. If the duration is exactly 0 or even negative that's fine, we should return 0 duration from parseHTTPDate and nil error. This will be overridden by callers since we have a floor of the duration, but that's not this functions concern.

I think we should fix parseHTTPDate to return 0 duration if the calculated duration is <=0.

@open-telemetry/opamp-go-approvers what do you think?

@tpaschalis
Copy link
Member Author

tpaschalis commented Jun 11, 2024

Thanks for the comment; I think you're right. At first I thought there might be some design decision behind this, so I chose to not challenge it.

After looking at the two places (1, 2) where the result of parseHTTPDate might be used, a duration of zero would just immediately trigger the timer, so I don't see a drawback in this. (Edit: In the current logic, it looks like returning a header in the past will reuse the last calculated interval in both the HTTP and WS client implementations). The same holds for the parseDelaySeconds function.

I can amend the PR and relevant tests to fix the parsing logic instead to get a feel for this.

Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
@tpaschalis tpaschalis changed the title Fix timing issue on TestExtractRetryAfterHeaderHttpDate Amend Retry-After parsing logic for negative durations Jun 11, 2024
Signed-off-by: Paschalis Tsilias <paschalis.tsilias@grafana.com>
Copy link

codecov bot commented Jun 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.82%. Comparing base (1a9603e) to head (dd2a1d6).
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #283      +/-   ##
==========================================
- Coverage   75.96%   75.82%   -0.14%     
==========================================
  Files          25       25              
  Lines        1656     1659       +3     
==========================================
  Hits         1258     1258              
- Misses        291      293       +2     
- Partials      107      108       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tpaschalis
Copy link
Member Author

cc @tigrannajaryan feel free to take another look!

Copy link
Member

@tigrannajaryan tigrannajaryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for nicely commented code.

@tigrannajaryan tigrannajaryan merged commit 8f7a652 into open-telemetry:main Jun 20, 2024
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TestExtractRetryAfterHeaderHttpDate is flaky
2 participants