roachprod: shorten the connection timeout for ssh connections to remote hosts #47567
Assignees
Labels
Comments
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Apr 14, 2021
Based on cockroachdb#47567 we need to ensure that roachprod has timout for ssh connection after 5 seconds. We add a constant for timeout and using it inside newSSHClient function. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Apr 20, 2021
Based on cockroachdb#47567 we need to ensure that roachprod has timout for ssh connection after 5 seconds. We are adding a timeout inside newRemoteSession function so the roachtest context cancellation does hide failure information by kill hanging processes before they fail. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Apr 23, 2021
Second part of cockroachdb#47567 where we need to ensure no dependencie is getting affected when we remove ssh.go. So we are removing cassandra.go and all its dependencies. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Apr 27, 2021
Based on cockroachdb#47567 we need to ensure that roachprod has timout for ssh connection after 5 seconds. We are adding a timeout inside newRemoteSession function so the roachtest context cancellation does hide failure information by kill hanging processes before they fail. Release note: None
craig bot
pushed a commit
that referenced
this issue
Apr 29, 2021
63607: roachprod: shorten the connection timeout for ssh r=jlinder a=alan-mas Based on #47567 we need to ensure that `roachprod` has timout for ssh connection after 5 seconds. We are adding a timeout inside `newRemoteSession` function so the `roachtest` context cancellation does hide failure information by kill hanging processes before they fail. This is the first part of the effort, in further changes we will modify/rename `ssh.go` to avoid confusion and ensure every dependency of this is not impacted. Release note: None 64351: clusterversion: mint 21.1 cluster version r=j-low a=j-low Release note: None Co-authored-by: Alanmas <acostas.alan@gmail.com> Co-authored-by: Joseph Lowinske <joe@cockroachlabs.com>
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
May 13, 2021
Second part of cockroachdb#47567 where we need to ensure no dependencie is getting affected when we remove ssh.go. So we are removing cassandra.go and all its dependencies. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Jun 17, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Jul 14, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Jul 28, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Aug 5, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Aug 6, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Aug 6, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
alan-mas
added a commit
to alan-mas/cockroach
that referenced
this issue
Aug 9, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
craig bot
pushed a commit
that referenced
this issue
Aug 10, 2021
64156: roachprod: adding unit test and renaming ssh.go r=jlinder a=alan-mas Third part of #47567 and fixes #65928 where we need to refactor `ssh.go` (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to `io.go` and only keeping `ProgressWriter` struct and function. Also we are adding some unit testing for this new `go` file. Release note: None Co-authored-by: Alanmas <acostas.alan@gmail.com>
sajjadrizvi
pushed a commit
to sajjadrizvi/cockroach
that referenced
this issue
Aug 10, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
RaduBerinde
pushed a commit
to RaduBerinde/cockroach
that referenced
this issue
Nov 9, 2021
Based on cockroachdb#47567 we need to ensure that roachprod has timout for ssh connection after 5 seconds. We are adding a timeout inside newRemoteSession function so the roachtest context cancellation does hide failure information by kill hanging processes before they fail. Release note: None
RaduBerinde
pushed a commit
to RaduBerinde/cockroach
that referenced
this issue
Nov 9, 2021
Second part of cockroachdb#47567 where we need to ensure no dependency is getting affected when we remove ssh.go. So we are removing cassandra.go and all its dependencies. Release note: None
RaduBerinde
pushed a commit
to RaduBerinde/cockroach
that referenced
this issue
Nov 9, 2021
Third part of cockroachdb#47567 where we need to refactor ssh.go (as one type structure is in use, we cannot remove it completely). So we are refactoring ssh.go and change its name to io.go and only keeping ProgressWriter struct and function Also we are adding a unit test go code to ensure it works as expected. Release note: None
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Sometimes the standard connection timeout length is to long and the calling program kills the roachprod process before it returns an error. Change roachprod so ssh connections time out after 5 or 10 seconds instead of the minute+ default timeout.
Related thread.
The text was updated successfully, but these errors were encountered: