State synchronization across server nodes.

[bwasia]

Does Colyseus synchronize state across servers? Suppose if there's two servers Server A and Server B running on ports 2567 and 2568 respectively. They both share the same server codebase.

Please refer to the code snippets below. index.ts is the server code, followed by console outputs from Server A and B.

Server A and B connects to a Redis docker container running on port 6379. As shown in the console outputs, the state is not shared.

index.ts:

import "dotenv/config.js";
const port = process.env.COLYSEUS_PORT
    ? Number(process.env.COLYSEUS_PORT)
    : 2567;
import { RedisPresence, Room, Server } from "colyseus";
import { MapSchema, Schema, type } from "@colyseus/schema";

console.log("COLYSEUS_PORT", process.env.COLYSEUS_PORT);
console.log("REDIS_HOST", process.env.REDIS_HOST);

const serverOptions = {
    presence: new RedisPresence({
        host: process.env.REDIS_HOST,
        port: process.env.REDIS_PORT ? Number(process.env.REDIS_PORT) : 6379,
    }),
};

class Player extends Schema {
    @type("string") sessionId: string = "";
    @type("number") x: number = 0;
    @type("number") y: number = 0;
}

class RoomState extends Schema {
    @type({ map: Player }) players = new MapSchema<Player>();
}

class TestRoom extends Room<RoomState> {
    maxClients = 4;
    onCreate(options: any) {
        console.log("TestRoom created!", options);

        this.setState(new RoomState());

        this.onMessage("move", this.onMove.bind(this));
    }
    onJoin(client: any, options: any) {
        const player = new Player();
        player.sessionId = client.sessionId;
        this.state.players.set(client.sessionId, player);
    }
    onMove(client: any, coors: {x: number, y: number}) {
        console.log("TestRoom received coords from", client.sessionId, ":", coors);
        const player = this.state.players.get(client.sessionId);
        if (player) {
            player.x = coors.x;
            player.y = coors.y;
        }
        // print out all players coords
        this.state.players.forEach((player) => {
            console.log(player.sessionId, player.x, player.y);
        });
    }
}

const gameServer = new Server(serverOptions);
gameServer.define("testroom", TestRoom);
gameServer.listen(port);

Server A Console Output

$ NODE_ENV=development COLYSEUS_PORT=2567 ts-node src/index.ts
COLYSEUS_PORT 2567
REDIS_HOST 127.0.0.1

       ___      _
      / __\___ | |_   _ ___  ___ _   _ ___
     / /  / _ \| | | | / __|/ _ \ | | / __|
    / /__| (_) | | |_| \__ \  __/ |_| \__ \
    \____/\___/|_|\__, |___/\___|\__,_|___/
                  |___/

Multiplayer Framework for Node.js · Open-source

💖 Sponsor Colyseus on GitHub → https://github.com/sponsors/endel
🌟 Give it a star on GitHub → https://github.com/colyseus/colyseus
☁️  Deploy and scale your project on Colyseus Cloud → https://cloud.colyseus.io

Error: seat reservation expired.
    at WebSocketServer.onConnection (C:\..\state-sync-test\server\node_modules\@colyseus\ws-transport\build\WebSocketTransport.js:102:15)
    at WebSocketServer.emit (node:events:513:28)
    at WebSocketServer.emit (node:domain:489:12)
    at WebSocketServer.completeUpgrade (C:\..\state-sync-test\server\node_modules\ws\lib\websocket-server.js:358:5)
    at WebSocketServer.handleUpgrade (C:\..\state-sync-test\server\node_modules\ws\lib\websocket-server.js:271:10)
    at Server.upgrade (C:\..\state-sync-test\server\node_modules\ws\lib\websocket-server.js:107:16)
    at Server.emit (node:events:513:28)
    at Server.emit (node:domain:489:12)
    at onParserExecuteCommon (node:_http_server:903:14)
    at onParserExecute (node:_http_server:797:3)
TestRoom created! {}
TestRoom received coords from KzVK_m1Ou : { x: 2, y: 3 }
KzVK_m1Ou 2 3
TestRoom received coords from KzVK_m1Ou : { x: 66, y: 66 }
KzVK_m1Ou 66 66

Server B Console Output

$ NODE_ENV=development COLYSEUS_PORT=2568 ts-node src/index.ts
COLYSEUS_PORT 2568
REDIS_HOST 127.0.0.1

       ___      _
      / __\___ | |_   _ ___  ___ _   _ ___
     / /  / _ \| | | | / __|/ _ \ | | / __|
    / /__| (_) | | |_| \__ \  __/ |_| \__ \
    \____/\___/|_|\__, |___/\___|\__,_|___/
                  |___/

Multiplayer Framework for Node.js · Open-source

💖 Sponsor Colyseus on GitHub → https://github.com/sponsors/endel
🌟 Give it a star on GitHub → https://github.com/colyseus/colyseus
☁️  Deploy and scale your project on Colyseus Cloud → https://cloud.colyseus.io

TestRoom created! {}
TestRoom received coords from Fmy5Mx9hT : { x: 4, y: 5 }
Fmy5Mx9hT 4 5
TestRoom received coords from Fmy5Mx9hT : { x: 6, y: 7 }
Fmy5Mx9hT 6 7
TestRoom received coords from Fmy5Mx9hT : { x: 77, y: 77 }
Fmy5Mx9hT 77 77

[endel]

Hi @bwasia, when using multiple processes you must also provide a "publicAddress" config option for the server: https://docs.colyseus.io/scalability/#recommendation-using-a-load-balancer

This is required so the client-side makes the request directly to the node that created the room and has the room in memory. Each room and each process is stateful in Colyseus.

Hope this helps!

[bwasia]

hi @endel

Thank you for your reply.

So, when you say "Each room and each process is stateful," you mean stateful for that node, right? Am I correct to assume this?

So, if a room mutates its state, will the nodes in the cluster be informed about the mutation only if the publicAddress option is set?

[eugene-egorov]

There is no any sharing state across processes. To sync some state across the rooms you can use redis

[bwasia]

Thank you!

[joshestein]

Hey @endel, thank you for a superb library!

I am wondering if there are any examples of syncing state across servers with Redis? I have read the documentation but am running into some difficulties getting things correctly setup.

[joshestein]

Hi @endel,

Would you mind explaining what the client-side code should look like for load balancing with NGINX and using redis + publicAddress?

[endel]

Hi @joshestein, the client SDK detects if the server has sent its publicAddress and connects directly to it: https://github.com/colyseus/colyseus.js/blob/afb7642e73631855ed74d76de02f4e8d203a45c9/src/Client.ts#L208

So there's no special configuration in the client, just in the server. Feel free to send me an email too!

[zack1991]

Hi @endel ,

Thanks for your support my issue get resolved, the issue was related with package-lock.json. I deleted the package-lock.json and node_modules folder and then run nmp install after that i restarted my server and my issue get resolved.

[zack1991]

Hi @endel ,
My server is running perfectly when using client.js to connect to server. But when i am trying to join room with unity version 0.15.3 i am getting exception, But when running colyseus with single instance then it is working perfect. If you don't mind may you please help what is the problem, i will be thank full for that.

Error Log :-
Unable to connect to the remote server at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <9577ac7a62ef43179789031239ba8798>:0
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <9577ac7a62ef43179789031239ba8798>:0
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <9577ac7a62ef43179789031239ba8798>:0
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <9577ac7a62ef43179789031239ba8798>:0
at System.Runtime.CompilerServices.TaskAwaiter1[TResult].GetResult () [0x00000] in <9577ac7a62ef43179789031239ba8798>:0 at Colyseus.ColyseusClient+d__251[T].MoveNext () [0x00200] in /Users/phonato-user/All Data/Bit Bucket/CamelCashCasino/Assets/Colyseus/Runtime/Colyseus/Models/ColyseusClient.cs:369

[endel]

Hi @zack1991 @VkDevPh, I've spent a few hours and couldn't reproduce this issue. (colyseus/colyseus-unity-sdk#215).

@VkDevPh also reported a "error CS0173: Type of conditional expression cannot be determined because there is no implicit conversion between 'method group' and '<null>'" (screenshot), which I also failed to reproduce.

Can you provide a reproduction scenario for these issues? (Unfortunately, I feel like wasting time when trying to reproduce issues like this without the full context.)