Sticky session doesn't work with websocket #3343

gaetancollaud · 2018-05-17T12:51:56Z

Do you want to request a feature or report a bug?

Bug

What did you do?

I wanted to load balance a websocket service and have the sticky session enabled. Each user open multiple websocket streams and they need to end up at the same place (not my code, not something we can change right now).

I use docker but I tried to configure manually the load balancing using a file with rules and I had the same result.

I created a simple websocket echo server to easily test the feature. See configuration below.

What did you expect to see?

I expect my websocket connexion to be sticky.

What did you see instead?

The sticky session cookies is not created and the session is not sticky. It works great for non websocket services.

Http request:

GET ws://www.mydomain.com/ws-echo HTTP/1.1
Host: www.mydomain.com
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http://www.mydomain.com
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.170 Safari/537.36
DNT: 1
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,fr-CH;q=0.8,fr;q=0.7
Cookie: test-cookie=whatever
Sec-WebSocket-Key: zBL1pMw2BV/LQ4TCImZRYw==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Sec-WebSocket-Protocol: control

Http response:

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: zDU6V6kSaLkVeBY3l7ZyNFFMEcI=
Sec-Websocket-Protocol: data
Server: WebSocket++/0.7.0

The dashboard shows the stickiness as enabled:

Output of `traefik version`: (What version of Traefik are you using?)

Version:      v1.6.1
Codename:     tetedemoine
Go version:   go1.10.2
Built:        2018-05-14_07:16:56PM
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?

logLevel = "INFO"
defaultEntryPoints = ["http"]

[api]
  [api.statistics]
    recentErrors = 100

[file]
watch = true
filename = "/etc/traefik/rules.toml"

[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "docker.local"
watch = true
exposedbydefault = false

[entryPoints]
  [entryPoints.traefik]
  address = ":8082"

  [entryPoints.http]
  address = ":80"

test websocket echo server

version: '3'
services:
  ws:
    image: gaetancollaud/websocket-echo-server
    labels:
      - "traefik.enable=true"
      - "traefik.frontend.rule=PathPrefix:/ws-echo"
      - "traefik.port=8080"
      - "traefik.backend.loadbalancer.stickiness=true"
      - "traefik.backend.loadbalancer.stickiness.cookieName=LB_TRAEFIK"

If applicable, please paste the log output in DEBUG level (`--logLevel=DEBUG` switch)

This is the log just after I started my container We can see that the sticky session is enabled with the cookie name LB_TRAEFIK: Sticky session with cookie LB_TRAEFIK.

time="2018-05-17T12:44:14Z" level=debug msg="Provider event received {Status:start ID:545f5ffa9672abc6ccf7d015097d36f334f5510106d2fc4a94eef8e9592d70a6 From:gaetancollaud/websocket-echo-server Type:container Action:start Actor:{ID:545f5ffa9672abc6ccf7d015097d36f334f5510106d2fc4a94eef8e9592d70a6 Attributes:map[com.docker.compose.version:1.19.0 com.docker.compose.service:ws traefik.docker.network:dockerdev_backend com.docker.compose.config-hash:891a6338d453fe2d40981f41319de6b4c34c021407f0f757672e96e7e38c62fb com.docker.compose.container-number:1 image:gaetancollaud/websocket-echo-server name:websocketechoserver_ws_1 traefik.backend.loadbalancer.stickiness:true traefik.backend.loadbalancer.stickiness.cookieName:LB_TRAEFIK traefik.enable:true traefik.port:8080 com.docker.compose.oneoff:False com.docker.compose.project:websocketechoserver traefik.frontend.rule:PathPrefix:/ws-echo]} Scope:local Time:1526561054 TimeNano:1526561054641414841}"
time="2018-05-17T12:44:14Z" level=debug msg="originLabelsmap[com.docker.compose.project:websocketechoserver com.docker.compose.service:ws com.docker.compose.version:1.19.0 traefik.backend.loadbalancer.stickiness.cookieName:LB_TRAEFIK traefik.docker.network:dockerdev_backend traefik.port:8080 com.docker.compose.config-hash:891a6338d453fe2d40981f41319de6b4c34c021407f0f757672e96e7e38c62fb com.docker.compose.container-number:1 com.docker.compose.oneoff:False traefik.backend.loadbalancer.stickiness:true traefik.enable:true traefik.frontend.rule:PathPrefix:/ws-echo]"
time="2018-05-17T12:44:14Z" level=debug msg="allLabelsmap[:map[traefik.backend.loadbalancer.stickiness:true traefik.enable:true traefik.frontend.rule:PathPrefix:/ws-echo traefik.backend.loadbalancer.stickiness.cookieName:LB_TRAEFIK traefik.docker.network:dockerdev_backend traefik.port:8080]]"
time="2018-05-17T12:44:14Z" level=debug msg="Filtering disabled container /vpdev-traefik"
time="2018-05-17T12:44:14Z" level=debug msg="originLabelsmap[com.docker.compose.project:websocketechoserver com.docker.compose.service:ws com.docker.compose.version:1.19.0 traefik.backend.loadbalancer.stickiness.cookieName:LB_TRAEFIK traefik.docker.network:dockerdev_backend traefik.port:8080 com.docker.compose.config-hash:891a6338d453fe2d40981f41319de6b4c34c021407f0f757672e96e7e38c62fb com.docker.compose.container-number:1 com.docker.compose.oneoff:False traefik.backend.loadbalancer.stickiness:true traefik.enable:true traefik.frontend.rule:PathPrefix:/ws-echo]"
time="2018-05-17T12:44:14Z" level=debug msg="allLabelsmap[:map[traefik.docker.network:dockerdev_backend traefik.port:8080 traefik.backend.loadbalancer.stickiness:true traefik.enable:true traefik.frontend.rule:PathPrefix:/ws-echo traefik.backend.loadbalancer.stickiness.cookieName:LB_TRAEFIK]]"
time="2018-05-17T12:44:14Z" level=debug msg="Configuration received from provider docker: {\"backends\":{\"backend-ws-websocketechoserver\":{\"servers\":{\"server-websocketechoserver-ws-1\":{\"url\":\"http://172.19.0.3:8080\",\"weight\":1}},\"loadBalancer\":{\"method\":\"wrr\",\"stickiness\":{\"cookieName\":\"LB_TRAEFIK\"}}}},\"frontends\":{\"frontend-PathPrefix-ws-echo-0\":{\"entryPoints\":[\"http\"],\"backend\":\"backend-ws-websocketechoserver\",\"routes\":{\"route-frontend-PathPrefix-ws-echo-0\":{\"rule\":\"PathPrefix:/ws-echo\"}},\"passHostHeader\":true,\"priority\":0,\"basicAuth\":[]}}}"
time="2018-05-17T12:44:14Z" level=debug msg="Creating frontend frontend-PathPrefix-ws-echo-0"
time="2018-05-17T12:44:14Z" level=debug msg="Wiring frontend frontend-PathPrefix-ws-echo-0 to entryPoint http"
time="2018-05-17T12:44:14Z" level=debug msg="Creating route route-frontend-PathPrefix-ws-echo-0 PathPrefix:/ws-echo"
time="2018-05-17T12:44:14Z" level=debug msg="Creating backend backend-ws-websocketechoserver"
time="2018-05-17T12:44:14Z" level=debug msg="Creating load-balancer wrr"
time="2018-05-17T12:44:14Z" level=debug msg="Sticky session with cookie LB_TRAEFIK"
time="2018-05-17T12:44:14Z" level=debug msg="Creating server server-websocketechoserver-ws-1 at http://172.19.0.3:8080 with weight 1"
time="2018-05-17T12:44:14Z" level=debug msg="Creating frontend angular"
time="2018-05-17T12:44:14Z" level=debug msg="Wiring frontend angular to entryPoint http"
time="2018-05-17T12:44:14Z" level=debug msg="Creating route test_1 PathPrefix:/angular,/sockjs-node"
time="2018-05-17T12:44:14Z" level=debug msg="Creating backend angular"
time="2018-05-17T12:44:14Z" level=debug msg="Creating load-balancer wrr"
time="2018-05-17T12:44:14Z" level=debug msg="Creating server server1 at http://www.mydomain.com:4200 with weight 0"
time="2018-05-17T12:44:14Z" level=debug msg="Creating frontend audit"
time="2018-05-17T12:44:14Z" level=debug msg="Wiring frontend audit to entryPoint http"
time="2018-05-17T12:44:14Z" level=debug msg="Creating route test_1 PathPrefix:/audit-api"
time="2018-05-17T12:44:14Z" level=debug msg="Creating backend audit"
time="2018-05-17T12:44:14Z" level=debug msg="Creating load-balancer wrr"
time="2018-05-17T12:44:14Z" level=debug msg="Creating server server1 at http://www.mydomain.com:8087 with weight 0"
time="2018-05-17T12:44:14Z" level=debug msg="Creating frontend base"
time="2018-05-17T12:44:14Z" level=debug msg="Wiring frontend base to entryPoint http"
time="2018-05-17T12:44:14Z" level=debug msg="Creating route test_1 PathPrefix:/"
time="2018-05-17T12:44:14Z" level=debug msg="Reusing backend angular"
time="2018-05-17T12:44:14Z" level=debug msg="Creating frontend keycloak"
time="2018-05-17T12:44:14Z" level=debug msg="Wiring frontend keycloak to entryPoint http"
time="2018-05-17T12:44:14Z" level=debug msg="Creating route test_1 PathPrefix:/auth"
time="2018-05-17T12:44:14Z" level=debug msg="Creating backend keycloak"
time="2018-05-17T12:44:14Z" level=debug msg="Creating load-balancer wrr"
time="2018-05-17T12:44:14Z" level=debug msg="Creating server server1 at http://www.mydomain.com:8086 with weight 0"
time="2018-05-17T12:44:14Z" level=debug msg="Creating frontend streamedian"
time="2018-05-17T12:44:14Z" level=debug msg="Wiring frontend streamedian to entryPoint http"
time="2018-05-17T12:44:14Z" level=debug msg="Creating route test_1 PathPrefix:/streamedian,/ext-lib"
time="2018-05-17T12:44:14Z" level=debug msg="Creating backend whitelabel"
time="2018-05-17T12:44:14Z" level=debug msg="Creating load-balancer wrr"
time="2018-05-17T12:44:14Z" level=debug msg="Creating server server1 at http://www.mydomain.com:8080 with weight 0"
time="2018-05-17T12:44:14Z" level=debug msg="Creating frontend test"
time="2018-05-17T12:44:14Z" level=debug msg="Wiring frontend test to entryPoint http"
time="2018-05-17T12:44:14Z" level=debug msg="Creating route test_1 PathPrefix:/test"
time="2018-05-17T12:44:14Z" level=debug msg="Creating backend test"
time="2018-05-17T12:44:14Z" level=debug msg="Creating load-balancer wrr"
time="2018-05-17T12:44:14Z" level=debug msg="Creating server server1 at http://www.mydomain.com:8081 with weight 0"
time="2018-05-17T12:44:14Z" level=debug msg="Creating frontend whitelabel"
time="2018-05-17T12:44:14Z" level=debug msg="Wiring frontend whitelabel to entryPoint http"
time="2018-05-17T12:44:14Z" level=debug msg="Creating route test_1 PathPrefix:/wl"
time="2018-05-17T12:44:14Z" level=debug msg="Reusing backend whitelabel"
time="2018-05-17T12:44:14Z" level=info msg="Server configuration reloaded on :8082"
time="2018-05-17T12:44:14Z" level=info msg="Server configuration reloaded on :80"

The text was updated successfully, but these errors were encountered:

daniellavoie · 2018-05-28T00:39:11Z

I have come across the same issue and conclusion. With some guidance, I could help out with a PR.

geraldcroes · 2018-05-31T10:08:57Z

I've reproduced the issue.

It seems that the problem comes from vulcand/oxy that ignores existing headers in the upgrade.

vulcand/oxy#137

daniellavoie · 2018-06-01T13:27:35Z

Hi @emilevauge, any target release of traefik that will embed vulcand/oxy#137 fix?

Thanks!

ldez · 2018-06-01T14:23:52Z

@daniellavoie the PR on Traefik will come very soon

daniellavoie · 2018-06-01T14:26:15Z

If you need help I can make it an excuse for my first PR on traefik :P

emilevauge · 2018-06-01T15:31:41Z

@daniellavoie with pleasure, if you make it, that would be on branch 1.6 :)

geraldcroes · 2018-06-01T15:41:36Z

Oh, sorry guys, didn't see your messages ...

traefiker · 2018-06-04T12:14:05Z

Closed by #3425.

gaetancollaud · 2018-06-18T08:52:48Z

I can confirm that since 1.6.3, the sticky cookie is correctly sent in the websocket response. Thanks guys !

traefiker added the status/0-needs-triage label May 17, 2018

ldez added area/websocket kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. labels May 17, 2018

geraldcroes self-assigned this May 18, 2018

geraldcroes removed the status/0-needs-triage label May 18, 2018

geraldcroes added kind/bug/confirmed a confirmed bug (reproducible). and removed kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. labels May 29, 2018

geraldcroes mentioned this issue Jun 1, 2018

Includes the headers in the HTTP response of a websocket request #3425

Merged

2 tasks

traefiker added this to the 1.6 milestone Jun 4, 2018

traefiker closed this as completed Jun 4, 2018

traefik locked and limited conversation to collaborators Sep 1, 2019

traefiker added the status/5-frozen-due-to-age label Sep 1, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sticky session doesn't work with websocket #3343

Sticky session doesn't work with websocket #3343

gaetancollaud commented May 17, 2018 •

edited

Loading

daniellavoie commented May 28, 2018

geraldcroes commented May 31, 2018

daniellavoie commented Jun 1, 2018

ldez commented Jun 1, 2018

daniellavoie commented Jun 1, 2018

emilevauge commented Jun 1, 2018

geraldcroes commented Jun 1, 2018

traefiker commented Jun 4, 2018

gaetancollaud commented Jun 18, 2018

Sticky session doesn't work with websocket #3343

Sticky session doesn't work with websocket #3343

Comments

gaetancollaud commented May 17, 2018 • edited Loading

Do you want to request a feature or report a bug?

What did you do?

What did you expect to see?

What did you see instead?

Output of traefik version: (What version of Traefik are you using?)

What is your environment & configuration (arguments, toml, provider, platform, ...)?

test websocket echo server

If applicable, please paste the log output in DEBUG level (--logLevel=DEBUG switch)

daniellavoie commented May 28, 2018

geraldcroes commented May 31, 2018

daniellavoie commented Jun 1, 2018

ldez commented Jun 1, 2018

daniellavoie commented Jun 1, 2018

emilevauge commented Jun 1, 2018

geraldcroes commented Jun 1, 2018

traefiker commented Jun 4, 2018

gaetancollaud commented Jun 18, 2018

gaetancollaud commented May 17, 2018 •

edited

Loading

Output of `traefik version`: (What version of Traefik are you using?)

If applicable, please paste the log output in DEBUG level (`--logLevel=DEBUG` switch)