Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cluster, net: backlog need pass to listen #4056

Closed
wants to merge 0 commits into from

Conversation

tangxinfa
Copy link

net.Server.listen's backlog option not take effect when use cluster, the backlog always 511(default value).

There is a two example, i set backlog to 64:

Without cluster:

var http       = require('http');

http.createServer(function(req, res) {
    res.writeHead(200);
    res.end("hello world\n");
}).listen(3000, "0.0.0.0", 64);

ss -ltn | grep 3000 output:

LISTEN     0      64           *:3000                     *:*                  

With cluster

var cluster    = require('cluster'),
    http       = require('http');


if (cluster.isMaster) {
    cluster.fork();
} else {
    http.createServer(function(req, res) {
        res.writeHead(200);
        res.end("hello world\n");
    }).listen(3000, "0.0.0.0", 64);
}

ss -ltn | grep 3000 output:

LISTEN     0      511          *:3000                     *:*                  

The backlog take effect is 511 not 64, this means node cluster module NOT take care of the backlog option when we call listen: server.listen(port[, hostname][, backlog][, callback])

@JungMinu
Copy link
Member

would you add some test?

@mscdex mscdex added cluster Issues and PRs related to the cluster subsystem. net Issues and PRs related to the net subsystem. labels Nov 28, 2015
@bnoordhuis
Copy link
Member

Can you update the documentation as well? If I read your patch right, the first backlog value wins when workers request different values. That should be documented.

else
this.server.listen(address); // UNIX socket path.
this.server.listen(address, backlog); // UNIX socket path.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do all of the variations of listen() accept a backlog parameter? It doesn't look like it's documented that way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks correct to me. Here is the relevant code in lib/net.js.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, the code checks out. The documentation is incomplete :-)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to server.listen's implementation, if second or third argument
is a Number, treat it as backlog size.

But server.listen(options[, callback]) already have a backlog in options
and has higher priority, it is better to avoid document it has another
standalone backlog parameter to avoid more confuse.

@bnoordhuis
Copy link
Member

@tangxinfa Left a comment. Note that GitHub doesn't send notifications when you add or change commits so please post a comment when you're done.

@jasnell
Copy link
Member

jasnell commented Dec 3, 2015

@tangxinfa .. haven't reviewed this but the PR would need to be cleaned up before it can land. It looks like you may have done a git merge instead of a rebase?

@tangxinfa
Copy link
Author

I cleaned up my repo, please checkout, and i also create another branch(https://github.com/tangxinfa/node/tree/cluster-support-backlog) combined three commits to one(tangxinfa@a1da62a) if you like:)

@jasnell
Copy link
Member

jasnell commented Dec 9, 2015

@tangxinfa .. thank you. Sorry I hadn't seen the additional commits, Github unfortunately does not notify when commits are added so at times we end up missing things. This LGTM if @bnoordhuis and @cjihrig are happy with it

@tangxinfa
Copy link
Author

@JungMinu
There is no easy way to write a test.

backlog is a hint to the implementation

man listen page says:

Implementations may impose a limit on backlog and silently reduce the specified value.

node doc says:

The actual length will
be determined by your OS through sysctl settings such as tcp_max_syn_backlog and
somaxconn on linux.

There is no standard api to retrieve the backlog option from a listened socket, but you can watch it from tools, test with the following example program(http-hello-cluster.js):

var cluster    = require('cluster'),
    http       = require('http');

if (cluster.isMaster) {
    cluster.fork().on('exit', function (worker) {
        console.log("worker #" + worker.id + " exit");
    });
    cluster.fork().on('exit', function (worker) {
        console.log("worker #" + worker.id + " exit");
    });
} else {
    var backlog = 64 + cluster.worker.id;
    http.createServer(function(req, res) {
        res.writeHead(200);
        res.end("hello world\n");
    }).listen(3000, "localhost", backlog).on('listening', function () {
        console.log("worker #" + cluster.worker.id + " backlog: " + backlog);
    });
}

"Send-Q" field from "ss -tln" command output

$ node ./http-hello-cluster.js &
worker #1 listen on port(3000) with backlog(65)
worker #2 listen on port(3000) with backlog(66)
$ ss -tln | grep 'Send-Q\|3000'
State      Recv-Q Send-Q Local Address:Port               Peer Address:Port              
LISTEN     0      65     127.0.0.1:3000                     *:*                       

"listen" system call from "strace" command output

$ strace node ./http-hello-cluster.js 2>&1 | grep listen
listen(12, 65)                          = 0
worker #2 listen on port(3000) with backlog(66)
worker #1 listen on port(3000) with backlog(65)

backlog is not store in the handle

So we can't do listen on slave with a backlog, retrieve the backlog from the handle after listened, and compare it to verify it works.

cjihrig pushed a commit that referenced this pull request Dec 16, 2015
The backlog parameter is supported by all variations of
net.Server.listen(), but wasn't consistently documented. This
commit brings the documentation into a more consistent state.

Refs: #4056
PR-URL: #4025
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
cjihrig pushed a commit that referenced this pull request Dec 16, 2015
The backlog parameter is supported by all variations of
net.Server.listen(), but wasn't consistently documented. This
commit brings the documentation into a more consistent state.

Refs: #4056
PR-URL: #4025
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
MylesBorins pushed a commit that referenced this pull request Dec 29, 2015
The backlog parameter is supported by all variations of
net.Server.listen(), but wasn't consistently documented. This
commit brings the documentation into a more consistent state.

Refs: #4056
PR-URL: #4025
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
MylesBorins pushed a commit that referenced this pull request Jan 19, 2016
The backlog parameter is supported by all variations of
net.Server.listen(), but wasn't consistently documented. This
commit brings the documentation into a more consistent state.

Refs: #4056
PR-URL: #4025
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
scovetta pushed a commit to scovetta/node that referenced this pull request Apr 2, 2016
The backlog parameter is supported by all variations of
net.Server.listen(), but wasn't consistently documented. This
commit brings the documentation into a more consistent state.

Refs: nodejs#4056
PR-URL: nodejs#4025
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
@estliberitas estliberitas force-pushed the master branch 2 times, most recently from 7da4fd4 to c7066fb Compare April 26, 2016 05:22
@jasnell jasnell added the stalled Issues and PRs that are stalled. label Apr 30, 2016
@cjihrig
Copy link
Contributor

cjihrig commented Jun 14, 2016

@tangxinfa are you still interested in this? If so, it needs a rebase.

@tangxinfa
Copy link
Author

@cjihrig
My repo's master branch is force rebase to upstream, all my commits moved to branch https://github.com/tangxinfa/node/tree/cluster-support-backlog

Please have a look, thanks:)

@cjihrig
Copy link
Contributor

cjihrig commented Jun 17, 2016

@tangxinfa you should be able to rebase on top of the same branch. You'll just have to resolve some merge conflicts. Alternatively, you could open a separate PR with your cluster-support-backlog branch.

@jfjm
Copy link

jfjm commented Jul 2, 2018

I'm seeing the same issue, any update on this?

oyyd added a commit to oyyd/node that referenced this pull request Jun 10, 2020
Currently, the master process would ignore `backlog` from worker
processes and use the default value instead. This commit will respect
the first `backlog` passed to the master process for a specific handle.

Refs: nodejs#4056
oyyd added a commit to oyyd/node that referenced this pull request Aug 18, 2020
Currently, the master process would ignore `backlog` from worker
processes and use the default value instead. This commit will respect
the first `backlog` passed to the master process for a specific handle.

Refs: nodejs#4056
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cluster Issues and PRs related to the cluster subsystem. net Issues and PRs related to the net subsystem. stalled Issues and PRs that are stalled.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants