[🐛 Bug]: Unexpected HTTP response status code 400 error when running tests inside chrome-standalone docker container

[jonn-set]

What happened?

I recently upgraded to 4.26 from 4.23 and have been facing intermittent issue running tests inside a Selenium Standalone chrome docker container v 130

The point where this is happening is when I call new Augmenter().augment(driver). However the same code works fine for a lot of tests before it starts failing. The exact error is jdk.internal.net.http.websocket.CheckFailedException: Unexpected HTTP response status code 400.

I am trying to intercept a HTTP Request (Response as well sometimes), but before I do that I have to augment the driver instance as its a RemoteWebDriver instance and that's when the exception is being thrown. I have not seen this happen when I am running it locally on a Windows 10 machine.

org.openqa.selenium.remote.http.ConnectionFailedException: JdkWebSocket initial request execution error
Build info: version: '4.26.0', revision: '8ccf0219d7'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '4.18.0-553.22.1.el8_10.x86_64', java.version: '11.0.25'
Driver info: driver.version: unknown
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.openSocket(JdkHttpClient.java:249)
	at org.openqa.selenium.devtools.Connection.<init>(Connection.java:89)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.<init>(SeleniumCdpConnection.java:36)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.lambda$create$3(SeleniumCdpConnection.java:103)
	at java.base/java.util.Optional.map(Optional.java:265)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:103)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:49)
	at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:50)
	at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:29)
	at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:207)
	at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:178)
	at com.jcs.race.steps.BaseSteps.interceptResponseEndingWith(BaseSteps.java:207)
	at com.jcs.race.steps.SearchSteps.lambda$new$0(SearchSteps.java:32)
	at ✽.I search for a valid customer(file:///app/bygg/teamcity-agent/Search.feature:5)
Caused by: java.net.http.WebSocketHandshakeException
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.resultFrom(OpeningHandshake.java:224)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:610)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:840)
	at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: jdk.internal.net.http.websocket.CheckFailedException: Unexpected HTTP response status code 400
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.checkFailed(OpeningHandshake.java:341)
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.handleResponse(OpeningHandshake.java:250)
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.resultFrom(OpeningHandshake.java:220)
	... 10 more

How can we reproduce the issue?

private CommandExecutor getShutterBugExecutor() {
    Tracer tracer = OpenTelemetryTracer.getInstance();
    ClientConfig config = ClientConfig.defaultConfig().baseUrl("http://localhost:4444");
    CommandExecutor executor =
        new HttpCommandExecutor(
            Collections.emptyMap(),
            config,
            new TracedHttpClient.Factory(tracer, Factory.createDefault()));
    return new TracedCommandExecutor(executor, tracer);
  }

  @Bean
  private WebDriver setupChrome() {
    ChromeOptions options = new ChromeOptions();
    options.addArguments(List.of(
                "--enable-automation",
                "--disable-extensions",
                "--disable-client-side-phishing-detection",
                "--no-default-browser-check",
                "--no-first-run",
                "--enable-logging=stderr",
                "--log-level=0",
                "--disable-dev-shm-usage",
                "--user-data-dir=/home/seluser/.config/google-chrome/default"
    ));
    ImmutableCapabilities capabilities = new ImmutableCapabilities("browserName", "chrome");
    return new RemoteWebDriver(getShutterBugExecutor(), capabilities.merge(options));
  }

// The above bean is autowired into tests

// The below is a method in the test which uses this instance of driver:
protected DevTools interceptRequestEndingWith(String urlPrefix) {
    driver = new Augmenter().augment(driver);    /*<--This is the point of failure*/
    DevTools devTools = ((HasDevTools) driver).getDevTools();
    devTools.createSessionIfThereIsNotOne();
    devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
    devTools.addListener(
        Network.requestWillBeSent(),
        requestWillBeSent -> {
          Request request = requestWillBeSent.getRequest();
          if (request.getUrl().endsWith(urlPrefix)
              && !request.getUrl().contains("www.google-analytics.com")) {
            scenarioContext.requestBody = request.getPostData().get();
          }
        });
    return devTools;
  }

// The same method works for close to 20 mins before it fails.

Relevant log output

org.openqa.selenium.remote.http.ConnectionFailedException: JdkWebSocket initial request execution error
Build info: version: '4.26.0', revision: '8ccf0219d7'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '4.18.0-553.22.1.el8_10.x86_64', java.version: '11.0.25'
Driver info: driver.version: unknown
	at org.openqa.selenium.remote.http.jdk.JdkHttpClient.openSocket(JdkHttpClient.java:249)
	at org.openqa.selenium.devtools.Connection.<init>(Connection.java:89)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.<init>(SeleniumCdpConnection.java:36)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.lambda$create$3(SeleniumCdpConnection.java:103)
	at java.base/java.util.Optional.map(Optional.java:265)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:103)
	at org.openqa.selenium.devtools.SeleniumCdpConnection.create(SeleniumCdpConnection.java:49)
	at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:50)
	at org.openqa.selenium.devtools.DevToolsProvider.getImplementation(DevToolsProvider.java:29)
	at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:207)
	at org.openqa.selenium.remote.Augmenter.augment(Augmenter.java:178)
	at com.jcs.race.steps.BaseSteps.interceptResponseEndingWith(BaseSteps.java:207)
	at com.jcs.race.steps.SearchSteps.lambda$new$0(SearchSteps.java:32)
	at ✽.I search for a valid customer(file:///app/bygg/teamcity-agent/Search.feature:5)
Caused by: java.net.http.WebSocketHandshakeException
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.resultFrom(OpeningHandshake.java:224)
	at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:610)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:840)
	at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:479)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: jdk.internal.net.http.websocket.CheckFailedException: Unexpected HTTP response status code 400
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.checkFailed(OpeningHandshake.java:341)
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.handleResponse(OpeningHandshake.java:250)
	at java.net.http/jdk.internal.net.http.websocket.OpeningHandshake.resultFrom(OpeningHandshake.java:220)
	... 10 more

Operating System

RHEL8

Selenium version

Java 4.26

What are the browser(s) and version(s) where you see this issue?

Chrome 130

What are the browser driver(s) and version(s) where you see this issue?

chromedriver 130

Are you using Selenium Grid?

No response

[github-actions]

@jonn-set, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[joerg1985]

#14410 did limit the number of websocket connections per session, i think you hit this limit.
It might be better to change this to a limit of concurrent connections per session.

[joerg1985]

@jonn-set are you calling interceptRequestEndingWith multiple times for one session? I guess everytime the agumenting is done a new devtools connection is established.

[jonn-set]

Yes, I am, will try doing it only once when I set it up and check. Will try increasing the nodes setting if the first option doesn't work.

Thanks @joerg1985, appreciate your help.

Will get back whatever the result is.

Thank you.

[jonn-set]

@joerg1985 augmenting only once seem to have worked. Thanks for your help, appreciate your keen eye to details.

Out of interest, how do you override the max websocket connections per session? Where we do do that config?

[joerg1985]

The connection-limit-per-session flag can be set in the node section of the config or as argument to the start up command.

This should be better documented on the webpage, i did not expect the PR to get merged this fast.

@pujagani do you think it does make sense to add a warning to the log, when one driver instance is augmented multiple times? Or silently returing the previously created agumented driver should be possible too.

[jonn-set]

Thanks @joerg1985 is the configuration something we can set when starting the selenium-standalone Docker Container, if yes, is it this variable SE_NODE_MAX_SESSIONS?

[joerg1985]

NODE_CONNECTION_LIMIT_PER_SESSION is the name of the enviroment variable.

[pujagani]

@joerg1985 I think its a good idea to add a warning to the log when augmented more than once.