Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[receiver/windowseventlog] Flaky test TestReadWindowsEventLogger #34687

Open
pjanotti opened this issue Aug 14, 2024 · 6 comments · Fixed by #34793
Open

[receiver/windowseventlog] Flaky test TestReadWindowsEventLogger #34687

pjanotti opened this issue Aug 14, 2024 · 6 comments · Fixed by #34793
Labels
flaky test a test is flaky os:windows receiver/windowseventlog

Comments

@pjanotti
Copy link
Contributor

Component(s)

receiver/windowseventlog

Describe the issue you're reporting

Flaky test: https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/10391822945/job/28775648414?pr=34656#step:6:571

=== FAIL: . TestReadWindowsEventLogger (20.08s)
    receiver_windows_test.go:108: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:108
        	Error:      	Condition never satisfied
        	Test:       	TestReadWindowsEventLogger

=== FAIL: . TestReadWindowsEventLogger (re-run 1) (20.02s)
    receiver_windows_test.go:108: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:108
        	Error:      	Condition never satisfied
        	Test:       	TestReadWindowsEventLogger
@pjanotti pjanotti added the needs triage New item requiring triage label Aug 14, 2024
@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@pjanotti pjanotti mentioned this issue Aug 14, 2024
Merged
@pjanotti
Copy link
Contributor Author

/label os:windows

@djaglowski djaglowski removed the needs triage New item requiring triage label Aug 16, 2024
@djaglowski
Copy link
Member

+1: https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/10422926675/job/28869515325?pr=34720#step:6:253

I thought this was due to other changes in the PR but it appears to be pre-existing.

@pjanotti
Copy link
Contributor Author

Yes, @djaglowski - I was glancing at the test code and there are many non-reliable things happening in the tests. I will start working on improving these tests.

@songy23 songy23 added the flaky test a test is flaky label Aug 16, 2024
@songy23
Copy link
Member

songy23 commented Aug 16, 2024

+1 https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/10425747888/job/28877304723

@github-actions github-actions bot mentioned this issue Aug 20, 2024
Closed
This was referenced Aug 21, 2024
Closed
Merged
f7o pushed a commit to f7o/opentelemetry-collector-contrib that referenced this issue Sep 12, 2024
@pjanotti @f7o
[chore] Fix windowseventlogreceiver test flakiness (open-telemetry#34793
8423a05 
)

**Description:**
There are 3 main issues being addressed on this PR:

* There is a race between start and generating the events in the tests:
the start require goroutines so a delay before generating the events
will be advisable (not perfect but good enough per my tests)
* The tests assume that only the tests are generating events in that
time window, that is impossible to guarantee on a windows box, so
instead of checking for exactly one event they need to get new events
and check the event provider instead
* The exclusion test checks if zero events were delivered by the
receiver, it should check if one not filtered out was received, and this
one should be generated after the event from the provider that should be
excluded.

**Link to tracking Issue:**
Fixes open-telemetry#34687

**Testing:**
Multiple local runs, we should do a few runs on CI given the runner
characteristics.

**Documentation:**
N/A
@songy23
Copy link
Member

songy23 commented Oct 16, 2024

This is still happening https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/11371732717/job/31634469890

=== Failed
=== FAIL: . TestReadWindowsEventLoggerRaw (13.26s)
    receiver_windows_test.go:328: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:330
        	            				C:/hostedtoolcache/windows/go/1.22.8/x64/src/runtime/asm_amd64.s:1695
        	Error:      	"[]" should have 1 item(s), but has 0
    receiver_windows_test.go:328: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:328
        	            				D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:209
        	Error:      	Condition never satisfied
        	Test:       	TestReadWindowsEventLoggerRaw
    receiver_windows_test.go:210: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:210
        	Error:      	"[]" should have 1 item(s), but has 0
        	Test:       	TestReadWindowsEventLoggerRaw

=== FAIL: . TestExcludeProvider/with_Raw (13.01s)
    receiver_windows_test.go:328: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:330
        	            				C:/hostedtoolcache/windows/go/1.22.8/x64/src/runtime/asm_amd64.s:1695
        	Error:      	"[]" should have 1 item(s), but has 0
    receiver_windows_test.go:328: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:328
        	            				D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:277
        	Error:      	Condition never satisfied
        	Test:       	TestExcludeProvider/with_Raw
    receiver_windows_test.go:278: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:278
        	Error:      	"[]" should have 1 item(s), but has 0
        	Test:       	TestExcludeProvider/with_Raw

=== FAIL: . TestExcludeProvider (17.78s)

=== FAIL: . TestReadWindowsEventLoggerRaw (re-run 1) (13.27s)
    receiver_windows_test.go:328: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:330
        	            				C:/hostedtoolcache/windows/go/1.22.8/x64/src/runtime/asm_amd64.s:1695
        	Error:      	"[]" should have 1 item(s), but has 0
    receiver_windows_test.go:328: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:328
        	            				D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:209
        	Error:      	Condition never satisfied
        	Test:       	TestReadWindowsEventLoggerRaw
    receiver_windows_test.go:210: 
        	Error Trace:	D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/windowseventlogreceiver/receiver_windows_test.go:210
        	Error:      	"[]" should have 1 item(s), but has 0
        	Test:       	TestReadWindowsEventLoggerRaw

DONE 2 runs, 20 tests, 4 failures in 68.683s

@songy23 songy23 reopened this Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flaky test a test is flaky os:windows receiver/windowseventlog
Projects
None yet
Milestone
No milestone
3 participants
@djaglowski @pjanotti @songy23