Fix Automattic/mongoose#3481; flush callbacks on error

[vkarpov15]

Hi Christian,

Re: Automattic/mongoose#3481. This issue is hard to repro consistently and really only happens if you get very unlucky, which is why I didn't include a test (figured you might have some ideas on that front). The general idea is that if you're connected to a standalone mongod and happen to have a write going when the mongod dies, the server will swallow the connection pool error, try to reconnect, and leave a callback hanging. Here's the debug output:

[DEBUG-Server:26752] 1447957927937 executing command [{"ns":"test.$cmd","cmd":{"insert":"numbers","documents":[{"number":901,"_id":"564e15a74f310c8068c3f288","__v":0}],"ordered":false,"writeConcern":{"w":1}},"options":{}}] against localhost:27017 { type: 'debug',
  message: 'executing command [{"ns":"test.$cmd","cmd":{"insert":"numbers","documents":[{"number":901,"_id":"564e15a74f310c8068c3f288","__v":0}],"ordered":false,"writeConcern":{"w":1}},"options":{}}] against localhost:27017',
  className: 'Server',
  pid: 26752,
  date: 1447957927937 }
[DEBUG-Server:26752] 1447957927938 cmd [{"insert":"numbers","documents":[{"number":901,"_id":"564e15a74f310c8068c3f288","__v":0}],"ordered":false,"writeConcern":{"w":1}}] about to be executed on connection with id 262 at localhost:27017 { type: 'debug',
  message: 'cmd [{"insert":"numbers","documents":[{"number":901,"_id":"564e15a74f310c8068c3f288","__v":0}],"ordered":false,"writeConcern":{"w":1}}] about to be executed on connection with id 262 at localhost:27017',
  className: 'Server',
  pid: 26752,
  date: 1447957927938 }
[DEBUG-Connection:26752] 1447957927938 writing buffer [a10000008603000000000000d407000000000000746573742e24636d640000000000ffffffff] to localhost:27017 { type: 'debug',
  message: 'writing buffer [a10000008603000000000000d407000000000000746573742e24636d640000000000ffffffff] to localhost:27017',
  className: 'Connection',
  pid: 26752,
  date: 1447957927938 }
[DEBUG-Connection:26752] 1447957927938 writing buffer [7b00000002696e7365727400080000006e756d626572730004646f63756d656e747300330000000330002b000000106e756d6265720085030000075f696400564e15a74f310c8068c3f288105f5f7600000000000000086f7264657265640000037772697465436f6e6365726e000c000000107700010000000000] to localhost:27017 { type: 'debug',
  message: 'writing buffer [7b00000002696e7365727400080000006e756d626572730004646f63756d656e747300330000000330002b000000106e756d6265720085030000075f696400564e15a74f310c8068c3f288105f5f7600000000000000086f7264657265640000037772697465436f6e6365726e000c000000107700010000000000] to localhost:27017',
  className: 'Connection',
  pid: 26752,
  date: 1447957927938 }
[DEBUG-Connection:26752] 1447957927940 connection 262 for [localhost:27017] errored out with [{"code":"EPIPE","errno":"EPIPE","syscall":"write"}] { type: 'debug',
  message: 'connection 262 for [localhost:27017] errored out with [{"code":"EPIPE","errno":"EPIPE","syscall":"write"}]',
  className: 'Connection',
  pid: 26752,
  date: 1447957927940 }
[DEBUG-Pool:26752] 1447957927943 pool [undefined] errored out [{"name":"MongoError","message":"write EPIPE"}] with connection [{"id":262,"host":"localhost","port":27017}] { type: 'debug',
  message: 'pool [undefined] errored out [{"name":"MongoError","message":"write EPIPE"}] with connection [{"id":262,"host":"localhost","port":27017}]',
  className: 'Pool',
  pid: 26752,
  date: 1447957927943 }

Once this happens and the mongod comes back, the pool reconnects just fine, but the callback still hangs. Let me know if you need more info or have any suggestions.

[christkv]

this is exactly the kind of situation I build the mock server lib for :)

[vkarpov15]

The mock lib would be very useful - would love to see how to use it. Repro-ing this from the mongoose integration layer was an adventure.