SocketIO load test always timesout

[manthan55]

Hi,

I have a Nodejs SocketIO server hosted in a Kubernetes cluster with 2 pods and I want to load test the same.

However, with the artillery test script below, I always get timeout errors.

socket-connect.yaml

config:
  target: "http://localhost:3001/"
  # target: "https://my-server.azurewebsites.net/"
  # target: "http://my-server-k8s-loadbalancer.com/"
  socketio:
    transports: ["websocket"]
  phases:
    - duration: 2
      arrivalRate: 2
scenarios:
  - name: Try to connect
    engine: socketio
    flow:
        - emit:
            channel: "join_chat_room"
            data:
                username: user-{{ $randomString() }}
        - think: 60

The result

Started phase 0, duration: 1s @ 19:19:39(+0530) 2021-02-23
Report @ 19:19:49(+0530) 2021-02-23
Elapsed time: 10 seconds
  Scenarios launched:  2
  Scenarios completed: 0
  Requests completed:  0
  Mean response/sec: NaN
  Response time (msec):
    min: NaN
    max: NaN
    median: NaN
    p95: NaN
    p99: NaN

Report @ 19:19:59(+0530) 2021-02-23
Elapsed time: 20 seconds
  Scenarios launched:  0
  Scenarios completed: 0
  Requests completed:  0
  Mean response/sec: NaN
  Response time (msec):
    min: NaN
    max: NaN
    median: NaN
    p95: NaN
    p99: NaN

Report @ 19:20:01(+0530) 2021-02-23
Elapsed time: 21 seconds
  Scenarios launched:  0
  Scenarios completed: 0
  Requests completed:  0
  Mean response/sec: NaN
  Response time (msec):
    min: NaN
    max: NaN
    median: NaN
    p95: NaN
    p99: NaN
  Errors:
    timeout: 2

All virtual users finished
Summary report @ 19:20:01(+0530) 2021-02-23
  Scenarios launched:  2
  Scenarios completed: 0
  Requests completed:  0
  Mean response/sec: NaN
  Response time (msec):
    min: NaN
    max: NaN
    median: NaN
    p95: NaN
    p99: NaN
  Scenario counts:
    Try to connect: 2 (100%)
  Errors:
    timeout: 2

I also have the same server running locally (localhost:3000) & on an Azure App Service and I am able to connect to all 3 hosted servers via a simple ReactJS client. Although, I always get timeout error regardless of either of the targets.

Server code

io.on("connection", (socket) => {
  console.log(`User connected : ${socket.id}`);
 
  socket.on("join_chat_room", (data) => {
    console.log("Joining room")
    socket.username = data.username;

    socket.broadcast.emit('user joined', {
      username: socket.username,
    });

  })
 
  socket.on("disconnect", (data) => {
    console.log(`User disconnected : ${socket.id}`);

    socket.broadcast.emit('user left', {
        username: socket.username
      });
  });
});

Any help would be really appreciated.

[hassy]

Is there a particular reason why transport is set to websocket? Is the ReactJS client also using websocket for transport?

What version of Socket.io is that on the server also?

[manthan55]

Hi,

I found the solution. Its a known issue #902.

I was able to solve the the above with the v3 version of SocketIO engine for Artillery found here.

@hassy Thanks for your time.

[hassy]

That's great @manthan55. I haven't had a chance to try out Socket.io v3 myself yet. Just curious, according to the official migration guide it would seem a v2 client (e.g. Artillery) should be able to connect to >=v3.1.0 Socket.io server, which is the version you seem to be using?

[manthan55]

Hey @hassy, I think you have been mistaken, the official migration guide clearly says a v2 client cannot connect to v3 (& vice versa).

It would be great to update Artillery docs to mention this since Artillery ships with SocketIO v2 engine as default.

[hassy]

Just below that it says:

As of Socket.IO 3.1.0, the v3 server is now able to communicate with v2 clients

We've updated the docs and will add v3 support into the official engine soon.

[manthan55]

Ah sorry, my bad, didn't read it thoroughly.

I read further and it seems like a v2 client can connect to v3 server if the allowEIO3 flag is set to true on the server, which wasn't set in my case as both client and server are v3.