-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Experimental v2/channel and consumer group #1661
base: main
Are you sure you want to change the base?
Commits on Mar 6, 2024
-
Add a client side channel group that listens for active channels and subscribes/unsubscribes as the set of active changes. Add a consumer group based on presence set Add typing information for ChannelGroups Fix tests for v2 ChannelGroups is a default realtime module types: add channel group typings consumergroup: use modulo-based hashing scheme The `hashring` package is node-only as it depends on the native `crypto` package. Replaced with a simple modulo hash scheme for now. Fixes the case where the channel is already attached and the channel is obtained with new rewind options that require a re-attach. Updates the consumer group partitioning test to more robustly assert that channels are partitioned across consumers. channelgroup: make get sync, subscribe async This matches the pattern used by Channels, which is sync to obtain the channel and async on subscribe in order to await attachment. In the channel group case, awaiting the subscribe awaits the joining of the consumer group. consumergroup: make consumerId required field consumergroup: make hashring a required field format: apply prettier formatting rules jsdoc: add consumer group docs channelgroups: add active channel name option channelgroup: add explicit join method Exposes the join method on the channel group, which is analogous to attach on the channel. Uses this in tests for more robust assertions. Additionally avoids re-attaching to the consumer group channel if already attached, and obtains presence membership synchronously in the join. consumergroup: use subscribe over on The `on` method was not reliable across clients, despite being documented in https://ably.com/docs/presence-occupancy/presence?lang=java#synced, so use subscribe instead. channelgroup: fix assigned channel processing We need to keep the total set of active channels around as updating the assigned channel set when the membership changes requires computing the new assignments from the complete channel set, not the previously set of assigned channels. consumergroup: include consumerId in logs consumergroups: add test for consumer group resize test: remove explicit join from test lint: apply formatting and cleanup test: replace var with let test: remove unnecessary outer try-catch test: add prefix to channels Avoid channel name collisions causing tests to fail from concurrent test runs in CI. channelgroup: detach from channel on un-assignment channelgroup: add unsubscribe listener method test: fix rebalance test waits for consumers channelgroup: add leave method test: remove dangling console logs test: test consumer group scale down event test: prefix consumer group channel Similar to the active channel, we need to avoid conflicts. consumergroup: fix current member tracking We store the current active set of members in the hashring. test: fix done condition w/ at-least-once delivery Messages can be delivered more than once during a consumer group rescaling event, so deduplicate the results when checking the end condition. channelgroups: use Utils.inspectError in logs channelgroups: do not share channel object The Channels object used by the ChannelGroup for internal channel operations no longer shares the same object exposed on the client via the .channels() method. This is to ensure that independent usage of an individual channel that happens to be included in a channel group is not impacted by its usage in the channel group. test: tidy up leave test Now that we can correctly handle a channel group and channel being used independently from the same client, this tidies up the leave test to remove the additional client previously needed. test: rename waitForConsumers for clarity test: rename waitForConsumers for clarity channelgroups: do not share channel object The Channels object used by the ChannelGroup for internal channel operations no longer shares the same object exposed on the client via the .channels() method. This is to ensure that independent usage of an individual channel that happens to be included in a channel group is not impacted by its usage in the channel group. channelgroups: add module integration modules: update ChannelGroups module definitions channelgroup: add temp rewind channel group option channelgroup: unsubscribe channel after timeout In order to avoid keeping the channel alive, we add a configurable timeout after which the channel will be unsubscribed if no messages are received. This is to avoid keeping the channel active. This can lead to missed messages if the a message is published after the client unsubscribes and before the channel becomes inactive. This is an acceptable edge case for the client-side simulation, especially with the default 1h timeout. deps: remove unused hashring types pkg utils: remove arrIndexOf polyfill consumergroup: rename hashring to locator test: use async style tests for channel groups Replaces the use of the `done()` callback with an async function style test. This allows us to await channel publish results and more easily handle race conditions in tests. channelgroup: use qualifier options Previously we relied on a new BaseRealtime instance with it's own Channels object to separate usage of channels in the ChannelGroup from independent external usage of those channels from the regular client.channels.get() method. This led to various problems with shared Auth state such as nonces in token requests which caused connections to terminate and tests to fail. A simpler solution is to avoid creating a new client instance and instead share the Channel pool, but force the library to treat channels used from the ChannelGroup independently (with their own attachment) by setting dummy options in the qualifier, which is used as the key in the channel map. This implementation does not support channels in the channel group which already have a qualifier. This is acceptable for the experimental client-side simulation of the feature.
Configuration menu - View commit details
-
Copy full SHA for bcf965f - Browse repository at this point
Copy the full SHA bcf965fView commit details
Commits on Mar 8, 2024
-
Configuration menu - View commit details
-
Copy full SHA for dc3fb7e - Browse repository at this point
Copy the full SHA dc3fb7eView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1ccee4a - Browse repository at this point
Copy the full SHA 1ccee4aView commit details -
Configuration menu - View commit details
-
Copy full SHA for fb78dd7 - Browse repository at this point
Copy the full SHA fb78dd7View commit details
Commits on Mar 12, 2024
-
channelgroup: deduplicate presence set by clientId
Each client will be entered in the presence set once per connection, so we need to deduplicate the presence set by clientId.
Configuration menu - View commit details
-
Copy full SHA for d3b7272 - Browse repository at this point
Copy the full SHA d3b7272View commit details
Commits on Apr 8, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 3a5caae - Browse repository at this point
Copy the full SHA 3a5caaeView commit details
Commits on Apr 9, 2024
-
Merge pull request #1725 from ably/experimental/channel-group-event-e…
…mitter channelgroup: emit channel assigned/active events
Configuration menu - View commit details
-
Copy full SHA for 05bd66e - Browse repository at this point
Copy the full SHA 05bd66eView commit details
Commits on Apr 10, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 2dab4c7 - Browse repository at this point
Copy the full SHA 2dab4c7View commit details
Commits on Apr 11, 2024
-
Merge pull request #1728 from ably/experimental/channel-group-member-…
…events consumergroup: emit membership updated event
Configuration menu - View commit details
-
Copy full SHA for 25bdfc1 - Browse repository at this point
Copy the full SHA 25bdfc1View commit details
Commits on Apr 26, 2024
-
channelgroup: only emit active channels on changed
Since the set of active channels are surfaced on $ably:active is a superset of those specified by a particular channel group expression, this commit ensures we only emit an active.updated event when the set of channels matching the consumer group expression have actually changed.
Configuration menu - View commit details
-
Copy full SHA for 029f5f5 - Browse repository at this point
Copy the full SHA 029f5f5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7462806 - Browse repository at this point
Copy the full SHA 7462806View commit details