You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In clusters where the Kubernetes API can have intermittent connectivity issues (noticed more in Azure environments that AWS) a build can become orphaned. This usually happens when the build controller is trying to update a build status and cannot make the API request because of connectivity issues. This looks to prevent the build status from being updated which causes subsequent builds to be queued indefinitely.
This usually manifests with build errors that look like:
##############################################
Start Build Process
##############################################
++ set +x
Unable to connect to the server: net/http: TLS handshake timeout
To Reproduce
Steps to reproduce the behavior:
Trigger a build
Cause some sort of net interruption between APIs
Build status is stuck in "started"
Expected behavior
A process exists that reaps orphaned builds.
Screenshots
Additional context
The text was updated successfully, but these errors were encountered:
shreddedbacon
changed the title
API issues result in orphaned builds which never complete causing infinite queueing
Kubernetes API issues result in orphaned builds which never complete causing infinite queueing
Aug 8, 2023
Describe the bug
In clusters where the Kubernetes API can have intermittent connectivity issues (noticed more in Azure environments that AWS) a build can become orphaned. This usually happens when the build controller is trying to update a build status and cannot make the API request because of connectivity issues. This looks to prevent the build status from being updated which causes subsequent builds to be queued indefinitely.
This usually manifests with build errors that look like:
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A process exists that reaps orphaned builds.
Screenshots
Additional context
The text was updated successfully, but these errors were encountered: