Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky TestTrafficSegmentSplit #2665

Closed
AlexanderYastrebov opened this issue Oct 9, 2023 · 6 comments · Fixed by #3330
Closed

Flaky TestTrafficSegmentSplit #2665

AlexanderYastrebov opened this issue Oct 9, 2023 · 6 comments · Fixed by #3330
Labels
flakytest

Comments

@AlexanderYastrebov
Copy link
Member 

--- FAIL: TestTrafficSegmentSplit (1.49s)
    segment_test.go:164: Response codes: map[200:5074 201:3085 202:1841]
    segment_test.go:168: 
        	Error Trace:	/home/runner/work/skipper/skipper/predicates/traffic/segment_test.go:93
        	            				/home/runner/work/skipper/skipper/predicates/traffic/segment_test.go:168
        	Error:      	Max difference between 2000 and 1841 allowed is 120, but difference was 159
        	Test:       	TestTrafficSegmentSplit
@szuecs
Copy link
Member

szuecs commented May 6, 2024

https://github.com/zalando/skipper/actions/runs/8962466724/job/24611376295?pr=3060

--- FAIL: TestTrafficSegmentSplit (1.63s)
    segment_test.go:164: Response codes: map[200:4941 201:2937 202:2122]
    segment_test.go:168: 
        	Error Trace:	/home/runner/work/skipper/skipper/predicates/traffic/segment_test.go:93
        	            				/home/runner/work/skipper/skipper/predicates/traffic/segment_test.go:168
        	Error:      	Max difference between 2000 and 2122 allowed is 120, but difference was -122
        	Test:       	TestTrafficSegmentSplit
FAIL

@szuecs
Copy link
Member

szuecs commented Jul 4, 2024

master build fails:

--- FAIL: TestTrafficSegmentSplit (1.67s)
    segment_test.go:164: Response codes: map[200:4930 201:2922 202:2148]
    segment_test.go:168: 
        	Error Trace:	/home/runner/work/skipper/skipper/predicates/traffic/segment_test.go:93
        	            				/home/runner/work/skipper/skipper/predicates/traffic/segment_test.go:168
        	Error:      	Max difference between 2000 and 2148 allowed is 120, but difference was -148
        	Test:       	TestTrafficSegmentSplit

@MustafaSaber
Copy link
Member

Another occurrence at #3326 see #3326 (comment)

@AlexanderYastrebov
Copy link
Member Author

I've already tried to fix it by #2423

@AlexanderYastrebov
Copy link
Member Author

AlexanderYastrebov commented Dec 5, 2024 •
edited
Loading

I think the way TrafficSegment works makes this test to have a non-zero chance to fail.
I can reproduce it via:

$ go test ./predicates/traffic/ -run=TestTrafficSegmentSplit -count=1000 -failfast -trimpath
--- FAIL: TestTrafficSegmentSplit (0.36s)
    segment_test.go:164: Response codes: map[200:5131 201:2991 202:1878]
    segment_test.go:168: 
                Error Trace:    github.com/zalando/skipper/predicates/traffic/segment_test.go:93
                                                        github.com/zalando/skipper/predicates/traffic/segment_test.go:168
                Error:          Max difference between 2000 and 1878 allowed is 120, but difference was 122
                Test:           TestTrafficSegmentSplit
FAIL
FAIL    github.com/zalando/skipper/predicates/traffic   9.645s
FAIL

There is an interesting proposal to deal with flaky tests in stdlib golang/go#62244 but its not implemented yet.

AlexanderYastrebov added a commit that referenced this issue Dec 5, 2024
@AlexanderYastrebov
predicates/traffic: deflake TestTrafficSegmentSplit
5ac95ff 
Use fixed random sequence to deflake TestTrafficSegmentSplit.

Fixes #2665

Signed-off-by: Alexander Yastrebov <alexander.yastrebov@zalando.de>
@AlexanderYastrebov AlexanderYastrebov mentioned this issue Dec 5, 2024
Merged
@AlexanderYastrebov
Copy link
Member Author

Created #3330 that uses fixed random sequence to deflake this test.

AlexanderYastrebov added a commit that referenced this issue Dec 5, 2024
@AlexanderYastrebov
predicates/traffic: deflake TestTrafficSegmentSplit
309fe1f 
Use fixed random sequence to deflake TestTrafficSegmentSplit.

Fixes #2665

Signed-off-by: Alexander Yastrebov <alexander.yastrebov@zalando.de>
AlexanderYastrebov added a commit that referenced this issue Dec 6, 2024
@AlexanderYastrebov
predicates/traffic: deflake TestTrafficSegmentSplit
5d908a4 
Use fixed random sequence to deflake TestTrafficSegmentSplit.

Fixes #2665

Signed-off-by: Alexander Yastrebov <alexander.yastrebov@zalando.de>
AlexanderYastrebov added a commit that referenced this issue Dec 10, 2024
@AlexanderYastrebov
predicates/traffic: deflake TestTrafficSegmentSplit (#3330)
010c033 
Use fixed random sequence to deflake TestTrafficSegmentSplit.

Fixes #2665

Signed-off-by: Alexander Yastrebov <alexander.yastrebov@zalando.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flakytest
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

predicates/traffic: deflake TestTrafficSegmentSplit zalando/skipper
3 participants
@szuecs @AlexanderYastrebov @MustafaSaber