Error: seat reservation expired

[0xBreath]

Hi all,
We scaled Colyseus to multiple instances across thousands of users, but we keep seeing this error Error: seat reservation expired.
I understand from the docs this could be because of a request delay, but that is almost certainly not the case.

We have half a dozen instances all running the same code, with zero communication between them. So horizontal scaling.
All instances run behind a load balancer and do not communication with each other.
We are not using match making or Redis.

The hypothesis I'm about to explain is based on the assumption that joinOrCreate is two requests to connect to a room.
One to check for seat reservations, and if successful, then it joins.

If there is a way to joinOrCreate a room with one request this would be the ideal solution.

Our hypothesis:
User calls joinOrCreate, and it talks to Instance 1 behind the load balancer to reserve a seat.
Instance 1 responds with success, there is a seat for you.
User sends the follow up request to actually join, but the load balancer goes to Instance 2.
Instance 2 knows nothing of this user, and so the request fails with Error: seat reservation expired.

I see there can be session IDs passed to join a specific room, but that would require a whole added layer to synchronize which machine you're talking to behind the load balancer. We're trying to avoid this.

Again, the perfect solution is joinOrCreate in one request. Not a two request process, like I believe it is.

Is there some form of joinOrCreate in one request that joins the room and it will error if there aren't seats or for another problem?
We can then retry and the load balancer will go to another instance/room, and eventually the client will join a room.

Any thoughts?

[endel]

Hi @0xBreath,

The 2 requests are required to allow the client to know which server/address it must connect to (see docs here)

The way you described your deployment - with a regular load balancer, and without configuring Redis in any of the Colyseus processes - each process would have its own independent list of rooms. The only way to allow the client to connect to the correct server would be to use "sticky sessions" / "affinity cookie" in your load balancer.

I'd recommend following the scalability docs and configuring Redis + publicAddress + exposing each process to the public internet.

Cheers