Ran into issues. Would love a quick primer on the arch, so that I can attempt to solve it.

[DhavalW]

Hi John,

First off, A big thank you for attempting this clustering plugin. It's obviously not foolproof yet, but should serve as a great starting point for any future work.

I'm currently working on this http://www.st8flo.com. Attempted to use deepstream behind the scenes as a signalling server, and tried using your plugin to cluster 2 nodes, but ran into a bunch of issues that I'd like to try solving.

Would love it if you can give me a primer on the architecture of deepstream & the clustering mechanism that you've used

If I manage to solve anything, will be happy to contribute that code back to your module.

---

The issues i ran into

1> auto discovery using node-discover seems to be broken.
The error i get is - "Cannot read property clusterNode of undefined".

So instead, I clustered the 2 nodes manually - both set as peers of each other :

2> There's a possible multiplication of state / messaging.

For a distributed indexing / db built on top of st8flo - A query that should've resulted in a 100 items, results in 400,

but only when this clustering plugin is used.

Without clustering it works as expected.

Networking Architecture for cluster test-
2 Deepstream signalling nodes clustered using this plugin (one on Mac, another on linux)

• 1 independent worker machine attached to each signalling node. Worker machines do the actual indexing and storage. Deepstream only acts as an RPC co-ordinator.

Networking architecture normally -
Single deepstream node that acts as signal / RPC server

• Multiple worker machines that connect to it and do the actual indexing / storage.

3> If accidentally disconnected, a signalling node can't reconnect or re-sync state.

Environment -
Deepstream - 3.1.1
Deepstream cluster - 0.6.2
Nodejs - 9.10.1
OS - Mac El Capitan & Linux Ubuntu 14.04

Unfortunately, I can't reproduce this for you to test, since st8flo isn't published yet.
But I can try to figure it out myself, if you help me understand how it works under the hood.

Any pointers / primers ?
Thanks in advance.

[wehriam]

I don’t have much time to help at this point, but if you create a pull request with tests illustrating the issues you’re encountering we can likely resolve them quickly. John Wehr email: johnwehr@gmail.com phone: 850-321-7070

…

________________________________ From: DhavalW <notifications@github.com> Sent: Wednesday, April 4, 2018 4:02:23 AM To: wehriam/deepstream.io-cluster Cc: Subscribed Subject: [wehriam/deepstream.io-cluster] Ran into issues. Would love a quick primer on the arch, so that I can attempt to solve it. (#1) Hi John, First off, A big thank you for attempting this clustering plugin. It's obviously not foolproof yet, but should serve as a great starting point for any future work. I'm currently working on this http://www.st8flo.com. Attempted to use deepstream behind the scenes as a signalling server, and tried using your plugin to cluster 2 nodes, but ran into a bunch of issues that I'd like to try solving. Would love it if you can give me a primer on the architecture of deepstream & the clustering mechanism that you've used If I manage to solve anything, will be happy to contribute that code back to your module.

________________________________ The issues i ran into 1> auto discovery using node-discover seems to be broken. The error i get is - "Cannot read property clusterNode of undefined". So instead, I clustered the 2 nodes manually - both set as peers of each other : 2> There's a possible multiplication of state / messaging. For a distributed indexing / db built on top of st8flo - A query that should've resulted in a 100 items, results in 400, but only when this clustering plugin is used. Without clustering it works as expected. Networking Architecture for cluster test- 2 Deepstream signalling nodes clustered using this plugin (one on Mac, another on linux) * 1 independent worker machine attached to each signalling node. Worker machines do the actual indexing and storage. Deepstream only acts as an RPC co-ordinator. Networking architecture normally - Single deepstream node that acts as signal / RPC server * Multiple worker machines that connect to it and do the actual indexing / storage. 3> If accidentally disconnected, a signalling node can't reconnect or re-sync state. Environment - Deepstream - 3.1.1 Deepstream cluster - 0.6.2 Nodejs - 9.10.1 OS - Mac El Capitan & Linux Ubuntu 14.04 Unfortunately, I can't reproduce this for you to test, since st8flo isn't published yet. But I can try to figure it out myself, if you help me understand how it works under the hood. Any pointers / primers ? Thanks in advance. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAE-SqG5W6Rx5dwxL9nMN8aa8WXUF1qDks5tlH4OgaJpZM4TGVwx>.