From f9c053d46e6c444cbc6771ad31082514e10df514 Mon Sep 17 00:00:00 2001 From: Lance Ball Date: Mon, 9 Jan 2017 12:45:46 -0500 Subject: [PATCH 1/9] doc: update http.md for consistency The HTTP Keep-Alive text was inconsistent. These changes follow the following rules * When referring to flag used in code, it should always be written using backticks and in camel case. E.g. `keepAlive`. * When referring to the mechanism Keep-Alive functionality as described in the HTTP 1.1 RFC, it is written as 'HTTP Keep-Alive', without the use of backticks. * When referring to the request header, it should always use backticks and be written as `Connection: keep-alive`. This commit also includes some changes to how `http.Agent` is referenced. When `Agent` is used as a reference to an object or instance, backticks should always be used. Left somewhat unresolved is how to determine when to use "agent" vs. "`Agent`". At the moment, the API documentation typically uses "agent" when referring to a generic instance, and either `http.Agent` or `Agent` when referring to a specific instance. But it's not really 100% clear. I wonder if it would be best to just always use `Agent`. Ref: https://github.com/nodejs/node/pull/10614 Fixes: https://github.com/nodejs/node/issues/10567 --- doc/api/http.md | 56 ++++++++++++++++++++++++------------------------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/doc/api/http.md b/doc/api/http.md index fb90123330458c..8f5373470ad962 100644 --- a/doc/api/http.md +++ b/doc/api/http.md @@ -48,25 +48,25 @@ list like the following: added: v0.3.4 --> -The HTTP Agent is used for pooling sockets used in HTTP client +The HTTP `Agent` is used for pooling sockets used in HTTP client requests. -The HTTP Agent also defaults client requests to using +The HTTP `Agent` also defaults client requests to using `Connection: keep-alive`. If no pending HTTP requests are waiting on a socket to become free the socket is closed. This means that Node.js's -pool has the benefit of keep-alive when under load but still does not -require developers to manually close the HTTP clients using -KeepAlive. - -If you opt into using HTTP KeepAlive, you can create an Agent object -with that flag set to `true`. (See the [constructor options][].) -Then, the Agent will keep unused sockets in a pool for later use. They -will be explicitly marked so as to not keep the Node.js process running. -However, it is still a good idea to explicitly [`destroy()`][] KeepAlive -agents when they are no longer in use, so that the Sockets will be shut -down. - -Sockets are removed from the agent's pool when the socket emits either +pool has the benefit of HTTP Keep-Alive when under load but still does not +require developers to manually close HTTP clients when using +this option. + +If you opt into using HTTP Keep-Alive, you can create an `Agent` instance, +providing `{ keepAlive: true }` among the constructor options. (See the +[constructor options][].) The `Agent` will then keep unused sockets in a +pool for later use. They will be explicitly marked so as to not keep the +Node.js process running. However, it is still a good idea to explicitly +[`destroy()`][] HTTP Keep-Alive agents when they are no longer in use, so that +the Sockets will be shut down. + +Sockets are removed from the `Agent`'s pool when the socket emits either a `'close'` event or a special `'agentRemove'` event. This means that if you intend to keep one HTTP request open for a long time and don't want it to stay in the pool you can do something along the lines of: @@ -136,7 +136,7 @@ added: v0.11.4 Produces a socket/stream to be used for HTTP requests. By default, this function is the same as [`net.createConnection()`][]. However, -custom Agents may override this method in case greater flexibility is desired. +custom agents may override this method in case greater flexibility is desired. A socket/stream can be supplied in one of two ways: by returning the socket/stream from this function, or by passing the socket/stream to `callback`. @@ -151,7 +151,7 @@ added: v0.11.4 Destroy any sockets that are currently in use by the agent. It is usually not necessary to do this. However, if you are using an -agent with KeepAlive enabled, then it is best to explicitly shut down +agent with HTTP Keep-Alive enabled, then it is best to explicitly shut down the agent when you know that it will no longer be used. Otherwise, sockets may hang open for quite a long time before the server terminates them. @@ -164,7 +164,7 @@ added: v0.11.4 * {Object} An object which contains arrays of sockets currently awaiting use by -the Agent when HTTP KeepAlive is used. Do not modify. +the agent when HTTP Keep-Alive is used. Do not modify. ### agent.getName(options) -The HTTP `Agent` is used for pooling sockets used in HTTP client -requests. - -The HTTP `Agent` also defaults client requests to using -`Connection: keep-alive`. If no pending HTTP requests are waiting on a -socket to become free the socket is closed. This means that Node.js's -pool has the benefit of HTTP Keep-Alive when under load but still does not -require developers to manually close HTTP clients when using -this option. - -If you opt into using HTTP Keep-Alive, you can create an `Agent` instance, -providing `{ keepAlive: true }` among the [constructor options][]. -The agent will then keep unused sockets in a pool for later use. The -unused sockets will be explicitly marked so as to not keep the -Node.js process running. However, it is still a good idea to explicitly -[`destroy()`][] HTTP Keep-Alive agents when they are no longer in use, so that -the sockets will be shut down. +An `Agent` is responsible for managing connection persistence +and reuse for HTTP clients. It maintains a queue of pending requests +for a given host and port, reusing a single socket connection for each +until the queue is empty, at which time the socket is destroyed. +When a socket connection is no longer writable, for example, if a server +closes the connection, the socket is destroyed. + +Unless an `Agent` is expressly provided in the [`http.request()`] +options, all HTTP client requests will use the default +[`http.globalAgent`]. + +In addition to managing connection persistence for queued HTTP +requests, an `Agent` can be used for pooling sockets across connections. +When providing `{ keepAlive: true }` among the [constructor options], +the `Agent` will not destroy sockets when the request queue has been +emptied. Rather, it will pool these sockets for later requests on the +same host and port. + +By providing `{ keepAlive: true }` as a constructor option to the the +HTTP `Agent`, sockets will not only be pooled, but the HTTP header +`Connection: keep-alive` will be sent by default for all client +requests. This also results in periodic TCP `SO_KEEPALIVE` requests +from the client to the server. + +When a socket is closed, whether by the user, or alternatively +by the server, it is removed from the pool. Any unused sockets in the pool +will be marked so as not to keep the Node.js process running. +It is good practice, however, to [`destroy()`][] HTTP Keep-Alive +agents when they are no longer in use, so that the sockets will be shut down. Sockets are removed from an agent's pool when the socket emits either -a `'close'` event or a special `'agentRemove'` event. This means that if +a `'close'` event or an `'agentRemove'` event. This means that if you intend to keep one HTTP request open for a long time and don't want it to stay in the pool you can do something along the lines of: @@ -79,7 +91,11 @@ http.get(options, (res) => { }); ``` -Alternatively, you could just opt out of pooling entirely using +You may also use an agent for an individual request. By providing +`{agent: false}` as an option to the `http.get()` or `http.request()` +functions, a one-time use `Agent` with default options and no connection +pooling will be used for the client connection. + `agent:false`: ```js @@ -102,9 +118,9 @@ added: v0.3.4 Can have the following fields: * `keepAlive` {Boolean} Keep sockets around in a pool to be used by other requests in the future. Default = `false` - * `keepAliveMsecs` {Integer} When using HTTP KeepAlive, how often - to send TCP KeepAlive packets over sockets being kept alive. - Default = `1000`. Only relevant if `keepAlive` is set to `true`. + * `keepAliveMsecs` {Integer} When using the `keepAlive` option, how + often to send TCP `SO_KEEPALIVE` `ACK` packets. Ignored when the + `keepAlive` option is false or undefined. Default = `1000`. * `maxSockets` {Number} Maximum number of sockets to allow per host. Default = `Infinity`. * `maxFreeSockets` {Number} Maximum number of sockets to leave open @@ -114,7 +130,7 @@ added: v0.3.4 The default [`http.globalAgent`][] that is used by [`http.request()`][] has all of these values set to their respective defaults. -To configure any of them, you must create your own [`http.Agent`][] object. +To configure any of them, you must create your own [`http.Agent`][] instance. ```js const http = require('http'); From e92d4ae34f10ada711b1045fbca0a53e990af0ea Mon Sep 17 00:00:00 2001 From: Lance Ball Date: Tue, 17 Jan 2017 16:33:23 -0500 Subject: [PATCH 4/9] doc: minor wording change/clarification to http.md --- doc/api/http.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/api/http.md b/doc/api/http.md index 13a9514c5e8de6..4eb3cf5196f48a 100644 --- a/doc/api/http.md +++ b/doc/api/http.md @@ -60,11 +60,11 @@ options, all HTTP client requests will use the default [`http.globalAgent`]. In addition to managing connection persistence for queued HTTP -requests, an `Agent` can be used for pooling sockets across connections. -When providing `{ keepAlive: true }` among the [constructor options], -the `Agent` will not destroy sockets when the request queue has been -emptied. Rather, it will pool these sockets for later requests on the -same host and port. +requests, an `Agent` can be used for pooling sockets across multiple +client requests. When providing `{ keepAlive: true }` among the +[constructor options], the `Agent` will not destroy sockets when +the request queue has been emptied. Rather, it will pool these +sockets for later requests on the same host and port. By providing `{ keepAlive: true }` as a constructor option to the the HTTP `Agent`, sockets will not only be pooled, but the HTTP header @@ -76,7 +76,7 @@ When a socket is closed, whether by the user, or alternatively by the server, it is removed from the pool. Any unused sockets in the pool will be marked so as not to keep the Node.js process running. It is good practice, however, to [`destroy()`][] HTTP Keep-Alive -agents when they are no longer in use, so that the sockets will be shut down. +agents when they are no longer iqn use, so that the sockets will be shut down. Sockets are removed from an agent's pool when the socket emits either a `'close'` event or an `'agentRemove'` event. This means that if From 4d7148bf070a042d430186fe0f4a8c7abff645a8 Mon Sep 17 00:00:00 2001 From: Lance Ball Date: Wed, 18 Jan 2017 17:59:07 +0100 Subject: [PATCH 5/9] doc: clarify that pooled sockets are unref()ed. --- doc/api/http.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/doc/api/http.md b/doc/api/http.md index 4eb3cf5196f48a..ad8788bf577286 100644 --- a/doc/api/http.md +++ b/doc/api/http.md @@ -74,9 +74,10 @@ from the client to the server. When a socket is closed, whether by the user, or alternatively by the server, it is removed from the pool. Any unused sockets in the pool -will be marked so as not to keep the Node.js process running. -It is good practice, however, to [`destroy()`][] HTTP Keep-Alive -agents when they are no longer iqn use, so that the sockets will be shut down. +will be marked so as not to keep the Node.js process running +(see [socket.unref()]). It is good practice, however, to [`destroy()`][] +a client `Agent` when it is no longer in use, so that the sockets +will be shut down. Sockets are removed from an agent's pool when the socket emits either a `'close'` event or an `'agentRemove'` event. This means that if @@ -1665,3 +1666,4 @@ There are a few special headers that should be noted. [constructor options]: #http_new_agent_options [Readable Stream]: stream.html#stream_class_stream_readable [Writable Stream]: stream.html#stream_class_stream_writable +[socket.unref()]: net.html#socket_unref \ No newline at end of file From bf0b7d171bea2c50b16bdf32e1e5cb3362e297ab Mon Sep 17 00:00:00 2001 From: Lance Ball Date: Wed, 18 Jan 2017 18:05:01 +0100 Subject: [PATCH 6/9] doc: fix wording for agent option in http.request --- doc/api/http.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/doc/api/http.md b/doc/api/http.md index ad8788bf577286..30dcef4842eae0 100644 --- a/doc/api/http.md +++ b/doc/api/http.md @@ -1537,12 +1537,10 @@ added: v0.3.6 * `headers` {Object} An object containing request headers. * `auth` {String} Basic authentication i.e. `'user:password'` to compute an Authorization header. - * `agent` {http.Agent|Boolean} Controls [`Agent`][] behavior. When an `Agent` - is used request will default to `Connection: keep-alive`. Possible values: + * `agent` {http.Agent|Boolean} Controls [`Agent`][] behavior. Possible values: * `undefined` (default): use [`http.globalAgent`][] for this host and port. * `Agent` object: explicitly use the passed in `Agent`. - * `false`: opts out of connection pooling with an `Agent`, defaults request to - `Connection: close`. + * `false`: causes a new `Agent` with default values to be used. * `createConnection` {Function} A function that produces a socket/stream to use for the request when the `agent` option is not used. This can be used to avoid creating a custom `Agent` class just to override the default From dab4842936aac23943d2eda0060258bacdbbc0b0 Mon Sep 17 00:00:00 2001 From: Lance Ball Date: Wed, 18 Jan 2017 18:11:30 +0100 Subject: [PATCH 7/9] doc: fix link to net.html#net_socket_unref --- doc/api/http.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api/http.md b/doc/api/http.md index 30dcef4842eae0..a86623eb538a61 100644 --- a/doc/api/http.md +++ b/doc/api/http.md @@ -1664,4 +1664,4 @@ There are a few special headers that should be noted. [constructor options]: #http_new_agent_options [Readable Stream]: stream.html#stream_class_stream_readable [Writable Stream]: stream.html#stream_class_stream_writable -[socket.unref()]: net.html#socket_unref \ No newline at end of file +[socket.unref()]: net.html#net_socket_unref \ No newline at end of file From 89727143f14463b12148221cfafa3be21f8c10b1 Mon Sep 17 00:00:00 2001 From: Lance Ball Date: Wed, 18 Jan 2017 23:12:56 +0100 Subject: [PATCH 8/9] doc: minor adjustments to keepAlive usage --- doc/api/http.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/doc/api/http.md b/doc/api/http.md index a86623eb538a61..5066e31c7bf857 100644 --- a/doc/api/http.md +++ b/doc/api/http.md @@ -64,7 +64,7 @@ requests, an `Agent` can be used for pooling sockets across multiple client requests. When providing `{ keepAlive: true }` among the [constructor options], the `Agent` will not destroy sockets when the request queue has been emptied. Rather, it will pool these -sockets for later requests on the same host and port. +sockets for later requests to the same host. By providing `{ keepAlive: true }` as a constructor option to the the HTTP `Agent`, sockets will not only be pooled, but the HTTP header @@ -72,12 +72,12 @@ HTTP `Agent`, sockets will not only be pooled, but the HTTP header requests. This also results in periodic TCP `SO_KEEPALIVE` requests from the client to the server. -When a socket is closed, whether by the user, or alternatively -by the server, it is removed from the pool. Any unused sockets in the pool -will be marked so as not to keep the Node.js process running +When a connection is closedby the client or the server, it is removed +from the pool. Any unused sockets in the pool +will be unrefed so as not to keep the Node.js process running (see [socket.unref()]). It is good practice, however, to [`destroy()`][] -a client `Agent` when it is no longer in use, so that the sockets -will be shut down. +a client `Agent` when it is no longer in use, because unused sockets +consume OS resources. Sockets are removed from an agent's pool when the socket emits either a `'close'` event or an `'agentRemove'` event. This means that if @@ -117,8 +117,9 @@ added: v0.3.4 * `options` {Object} Set of configurable options to set on the agent. Can have the following fields: - * `keepAlive` {Boolean} Keep sockets around in a pool to be used by - other requests in the future. Default = `false` + * `keepAlive` {Boolean} Keep sockets around even when there are no + outstanding requests, so it can be used for future requests without + having to reestablish an HTTP connection. Default = `false` * `keepAliveMsecs` {Integer} When using the `keepAlive` option, how often to send TCP `SO_KEEPALIVE` `ACK` packets. Ignored when the `keepAlive` option is false or undefined. Default = `1000`. @@ -168,7 +169,7 @@ added: v0.11.4 Destroy any sockets that are currently in use by the agent. It is usually not necessary to do this. However, if you are using an -agent with HTTP Keep-Alive enabled, then it is best to explicitly shut down +agent with `keepAlive` enabled, then it is best to explicitly shut down the agent when you know that it will no longer be used. Otherwise, sockets may hang open for quite a long time before the server terminates them. @@ -181,7 +182,7 @@ added: v0.11.4 * {Object} An object which contains arrays of sockets currently awaiting use by -the agent when HTTP Keep-Alive is used. Do not modify. +the agent when `keepAlive` is enabled. Do not modify. ### agent.getName(options)