Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

events: optimize eventEmitter.once() #914

Closed
wants to merge 1,732 commits into from
Closed

Conversation

mscdex
Copy link
Contributor

@mscdex mscdex commented Feb 21, 2015

This basically removes the extra closure previously created by
once() and instead uses an object to denote a one-time handler.
The result is ~340% performance increase when adding one-time
handlers, ~48% performance increase when emitting events with
mixed persistent and one-time handlers, and ~10% performance
increase when emitting with multiple arguments with mixed handlers,
all with negligible negative performance impact to other EventEmitter
functionality.

Also, removeListener() was optimized a bit by eliminating handler
arrays of length 1 so that the leftover handler is set on
_events[type].

The benchmarks from #912 were used when measuring the changes of this PR.

handlers.onceCount = (existingHandler.once ? 1 : 0);
dest._events.error = handlers;
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might pose a problem – Readable and Duplex streams coming out of pegged versions of readable-stream on npm won't have this logic, and so won't preserve the once-ness of existing error handlers after these revisions.

For example:

var Readable = require('readable-stream').Readable;
var rs = new Readable;

var Writable = require('stream').Writable;
var ws = new Writable;
writable.once('error', function() {
  // important code to only perform once
});
rs.pipe(ws);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that'd be a problem. Gotta start somewhere though :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chrisdickinson If we simultaneously merge this same change into iojs/readable-stream everything should still work for both node.js and io.js right?

@mscdex mscdex added the events Issues and PRs related to the events subsystem / EventEmitter. label Mar 22, 2015
@brendanashworth
Copy link
Contributor

Reiterating on this, is this PR still in the discussion stage?

@mscdex
Copy link
Contributor Author

mscdex commented Apr 4, 2015

@brendanashworth Since this modifies streams code, there might be a potential issue with module authors who have pegged their readable-stream dependencies to a specific version.

@ChALkeR
Copy link
Member

ChALkeR commented May 28, 2015

@mscdex As I understand this, this PR breaks unattaching and reattaching an event that should be fired once. That is used, for example, by https://github.com/timoxley/overshadow-listeners/blob/master/index.js.

The problem here is that EventEmitter.prototype.listeners returns different results before and after this PR. Testcase:

const EventEmmiter = require('events');
var ee = new EventEmmiter();
ee.once('foo', function() {});
console.log(ee.listeners('foo')[0].toString());

To be compatible, EventEmitter.prototype.listeners should use old-style wraps around «once» callbacks. Note: you should also double-check EventEmitter.prototype.removeListener after that.

If not corrected, this would be a semver-major.

@ChALkeR
Copy link
Member

ChALkeR commented May 28, 2015

Another testcase:

const EventEmmiter = require('events');
var ee = new EventEmmiter();

ee.once('foo', function() { console.log('foo!'); });
var bar = ee.listeners('foo')[0];
ee.removeAllListeners('foo');
ee.on('foo', bar);

ee.emit('foo');
ee.emit('foo');

@mscdex
Copy link
Contributor Author

mscdex commented May 28, 2015

@ChALkeR Yes that would be backwards-incompatible for that particular use case. However IMHO the new behavior is probably what most people would expect. All the user knows is that they added a function to be called once and if they then copied that function and added it to be called more than once, you would expect it to be called more than once.

@ChALkeR
Copy link
Member

ChALkeR commented May 28, 2015

But that still makes it a semver-major.

@mscdex
Copy link
Contributor Author

mscdex commented May 28, 2015

@ChALkeR Right, I'm not disagreeing there.

@ChALkeR
Copy link
Member

ChALkeR commented May 28, 2015

And the new implementation returns exactly the same callback for

const EventEmmiter = require('events');
var ee = new EventEmmiter();
ee.once('foo', function() {});
var callback = ee.listeners('foo')[0];

and

const EventEmmiter = require('events');
var ee = new EventEmmiter();
ee.on('foo', function() {});
var callback = ee.listeners('foo')[0];

Now there is no way to distinguish «once» and normal callbacks when listing the listeners.
I wouldn't say that this is expected. Though I wouldn't say that current #914 (comment) result is expected too.

Maybe there should a new API that properly distinguishes «once» and normal callbacks.
For example:

  • EventEmmiter.prototype.listenersRegular returns only regular («on») callbacks,
  • EventEmmiter.prototype.listenersOnce returns only «once» callbacks,
  • EventEmmiter.prototype.listeners keeps compatibility by wrapping «once» callbacks.

This could work. Though EventEmitter.prototype.removeListener might need a closer look and could also cause a semver-major.

@mscdex
Copy link
Contributor Author

mscdex commented May 28, 2015

@ChALkeR Why would you need to distinguish between one-time event handlers and not when receiving the current list of event handlers for an event?

@ChALkeR
Copy link
Member

ChALkeR commented May 28, 2015

@mscdex
Copy link
Contributor Author

mscdex commented May 28, 2015

@ChALkeR I already saw that. IMHO that's inappropriately relying on an implementation detail.

I could also argue that your original test case does not make sense (without knowing EventEmitter internals) when run without the changes in this PR. "I added my listener with on(), why isn't it firing more than once?"

@ChALkeR
Copy link
Member

ChALkeR commented May 28, 2015

@mscdex Are you talking about #914 (comment)? If yes, I already mentioned in #914 (comment).

@brendanashworth brendanashworth added the semver-major PRs that contain breaking changes and should be released in the next major version. label Jul 15, 2015
@Fishrock123
Copy link
Contributor

@mscdex did you still want to do this or can we close it?

@mscdex
Copy link
Contributor Author

mscdex commented Oct 19, 2015

@Fishrock123 I think the main problem is the npm modules that are pegged at particular versions of readable-stream. We'd need those modules to either manually update the pegged version (once the readable-stream changes would be incorporated simultaneously with node) or to change their version specifier (e.g. to use semver's ^) so that the new changes would be picked up automatically.

@jasnell jasnell added the stalled Issues and PRs that are stalled. label Nov 16, 2015
Myles Borins and others added 5 commits January 7, 2016 16:48
Adds Myles Borins and his public key to the README

PR-URL: nodejs#4578
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Adds Evan Lucas and his public key to the README for releases

PR-URL: nodejs#4579
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
It avoids the creation of unnecessary handles. This issue is causing
intermitent failures in `test-cluster-disconnect-race` on `FreeBSD`
and `OS X`.

The problem is that the `worker2.disconnect` is being called on the
master before the `queryServer` is handled, causing the worker to
be deleted, then the Server handle is created afterwards. Later on,
when `removeWorker` is called from the `exit` handler, there are no
workers left, but one handle, thus the `AssertionError`.

Add a new `test/sequential/test-cluster-disconnect-leak` based on
`test-cluster-disconnect-race` that creates lots of workers and fails
consistently without this patch.

PR-URL: nodejs#4465
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Rich Trott <rtrott@gmail.com>
fix description about the latest LTS release download page
to make it clear

PR-URL: nodejs#4583
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
`V4MAPPED` isn't supported by Android either (as of 6.0)

PR-URL: nodejs#4580
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Trott and others added 26 commits February 2, 2016 15:10
PR-URL: nodejs#4988
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Fixes: nodejs#5031
PR-URL: nodejs#5044
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#4985
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Stop using the deprecated `GetHiddenValue()` and `SetHiddenValue()`
methods, start using `GetPrivate()` and `SetPrivate()` instead.

This commit turns some of the entries in the per-isolate string table
into private symbols.

PR-URL: nodejs#5045
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
PR-URL: nodejs#5008
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
This commit adds a new method for TLS sockets that returns the
negotiated protocol version.

PR-URL: nodejs#4995
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
getCipher() actually includes the protocol version that the cipher was
first supported and *not* the negotiated protocol of the current
connection.

PR-URL: nodejs#4995
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
fix a reference to a non-existent API, `hash.final()`.
It should be `hash.digest()`.

PR-URL: nodejs#5050
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
PR-URL: nodejs#5047
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#5047
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#5068
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
From time to time this test is failing in OS X because at least one of
the connections takes quite a long time (around 5 seconds) causing some
of the timers may fire before the test exited. To solve this, wait for
all the connections to be established before setting the timeouts and
unrefing the sockets.

PR-URL: nodejs#4772
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#4996
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Put links in a lexical order. Add missing links. Remove duplicates.

PR-URL: nodejs#5072
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: nodejs#5057
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Add links to `process.arch` and `process.platform`.

PR-URL: nodejs#5006
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
PR-URL: nodejs#4904
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Roman Reiss <me@silverwind.io>
Sort links in lexical order

PR-URL: nodejs#5076
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Enable `space-unary-ops` in `.eslintrc`. This prohibits things like:

    i ++        // use `i++` instead
    typeof(foo) // use `typeof foo` or `typeof (foo)` instead

Ref: nodejs#4772 (comment)
PR-URL: nodejs#5063
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: James M Snell <jasnell@gmail.com>
The comment stating it was deprecated was added in 2011 via
4ef8f06. It is time to
actually deprecate it.

PR-URL: nodejs#5049
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
Add fromArrayLike() to handle logic of copying in values from array-like
argument.

PR-URL: nodejs#4948
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
`fs.read` supports a deprecated string interface version, which is
not documented. It was intended to be deprecated in this commit in 2010
nodejs@c93e0aa
This patch issues a deprecation message saying the usage of this
interface is deprecated.

PR-URL: nodejs#4525
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
There was a very subtle change in behavior introduced with 27def4f

In the past if querystring.parse was given Infinity for maxKeys,
everything worked as expected.

Check to see is maxKeys is Infinity before forwarding the value to
String.prototype.split which causes this regression

PR-URL: nodejs#5066
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Currently a debug context is created for various calls to util.

If the node debugger is being run the main context is the debug
context. In this case node_contextify was freeing the debug context
and causing everything to explode.

This change moves around the logic and no longer frees the context.

There is a concern about the dangling pointer

The regression test was adapted from code submitted by @3y3 in nodejs#4815

Fixes: nodejs#4440
Fixes: nodejs#4815
Fixes: nodejs#4597
Fixes: nodejs#4952

PR-URL: nodejs#4815

Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Rich Trott <rtrott@gmail.com>
A few tests have started failing on Raspberry Pi devices in CI.
https://ci.nodejs.org/job/node-test-binary-arm/943/

PR-URL: nodejs#5082
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
Ref: nodejs#4830
Ref: nodejs#3635
Ref: nodejs#4526
This commit improves once() performance by storing the event handler
directly instead of creating a wrapper function every time.

These changes bring ~150% increase in performance when simply adding
once() event handlers, ~220% increase in the included ee-emit-once
benchmark, and a ~50% increase in the included ee-add-remove-once
benchmark.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
events Issues and PRs related to the events subsystem / EventEmitter. semver-major PRs that contain breaking changes and should be released in the next major version. stalled Issues and PRs that are stalled.
Projects
None yet
Development

Successfully merging this pull request may close these issues.