Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Increase timeout on test raw #566

Merged
merged 1 commit into from
Aug 31, 2021

Conversation

Itxaka
Copy link
Contributor

@Itxaka Itxaka commented Aug 31, 2021

Currently the github runners performance is not stable enough to let the
auto retry on the test framework to work as expected in the given time
(180s)

This patch bumps the timeout to 300 seconds instead which should be more
than enough for the VM to come up.

Signed-off-by: Itxaka igarcia@suse.com

@Itxaka Itxaka force-pushed the bump_test_raw_images_wait branch from f15836a to c7b1655 Compare August 31, 2021 10:59
@kkaempf
Copy link
Contributor

kkaempf commented Aug 31, 2021

I'd rather see such constants be defined in a central place instead of being embedded in the code.

Copy link
Contributor

@mudler mudler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good here

@Itxaka
Copy link
Contributor Author

Itxaka commented Aug 31, 2021

This is part of the test stuff, Im not sure we need a central place for the test config??? I can create one and move it there but....I dont know if it would be useful...

@Itxaka Itxaka force-pushed the bump_test_raw_images_wait branch from c7b1655 to 4e816b7 Compare August 31, 2021 12:05
@mudler
Copy link
Contributor

mudler commented Aug 31, 2021

This is part of the test stuff, Im not sure we need a central place for the test config??? I can create one and move it there but....I dont know if it would be useful...

I'd say we are fine with the defaults already defined in the SUT, this is just overriding for the special case

@mudler
Copy link
Contributor

mudler commented Aug 31, 2021

mmh still fails also with the timeout bumped:

cOS Recovery deploy tests after running recovery from the raw_disk image 
  uses cos-deploy to install
  /Users/runner/work/cOS-toolkit/cOS-toolkit/tests/recovery-raw-disk/recovery_raw_disk_test.go:19
[BeforeEach] cOS Recovery deploy tests
  /Users/runner/work/cOS-toolkit/cOS-toolkit/tests/recovery-raw-disk/recovery_raw_disk_test.go:13

make: *** [raw_disk_test_deploy] Error 1
• Failure in Spec Setup (BeforeEach) [300.016 seconds]
cOS Recovery deploy tests
/Users/runner/work/cOS-toolkit/cOS-toolkit/tests/recovery-raw-disk/recovery_raw_disk_test.go:10
  after running recovery from the raw_disk image [BeforeEach]
  /Users/runner/work/cOS-toolkit/cOS-toolkit/tests/recovery-raw-disk/recovery_raw_disk_test.go:18
    uses cos-deploy to install
    /Users/runner/work/cOS-toolkit/cOS-toolkit/tests/recovery-raw-disk/recovery_raw_disk_test.go:19

    Timed out after 300.015s.
    Unexpected error:
        <*errors.errorString | 0xc00005d680>: {
            s: "ssh: handshake failed: read tcp 127.0.0.1:49416->127.0.0.1:2222: read: connection reset by peer",
        }
        ssh: handshake failed: read tcp 127.0.0.1:49416->127.0.0.1:2222: read: connection reset by peer
    occurred

    /Users/runner/work/cOS-toolkit/cOS-toolkit/tests/sut/sut.go:161
------------------------------


Summarizing 1 Failure:

[Fail] cOS Recovery deploy tests [BeforeEach] after running recovery from the raw_disk image uses cos-deploy to install 
/Users/runner/work/cOS-toolkit/cOS-toolkit/tests/sut/sut.go:161

Ran 1 of 1 Specs in 300.016 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestTests (300.02s)
FAIL

Maybe a bug is hiding there?

@Itxaka
Copy link
Contributor Author

Itxaka commented Aug 31, 2021

I think its just a timeout, if you check the serial log it shows that the machine wasnt up when we recovered it, so it was slow to boot. Plus the usual grub fallback also doesnt help (needs to fallback 3??4?? times until it launches the recovery)

EDIT: yep, this time it was on the cOS system early rootfs setup hwne the timeout ocurred, so slow launch :/

Currently the github runners performance is not stable enough to let the
auto retry on the test framework to work as expected in the given time
(180s)

This patch bumps the timeout to 300 seconds instead which should be more
than enough for the VM to come up.

Signed-off-by: Itxaka <igarcia@suse.com>
@Itxaka Itxaka force-pushed the bump_test_raw_images_wait branch from 4e816b7 to ea538b9 Compare August 31, 2021 13:06
@mudler mudler merged commit 49ef4b1 into rancher:master Aug 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants