Replace the existing plugins mechanism

A summary of the discussion on #1492:

- the modular API should be the only way to pass optional functionality
  to the SDK

- this means we need to replace the existing ClientOptions.plugins
  mechanism, which is currently used to pass a Vcdiff decoder

- since the modular variant of the SDK only exists for web at the
  moment, we will bundle Vcdiff decoding into all other platforms (in
  which bundle size is not much of a concern)

- on web, if you want deltas, you have to use the modular variant of the
  SDK

So, we remove the ClientOptions.plugins mechanism and introduce a
tree-shakable Vcdiff module, which bundles the vcdiff-decoder library
(meaning that users no longer need to directly import this library).

Note that this means that, currently, it is no longer possible to use
deltas inside a Web Worker. We’ll address this in #1514.

The README example of configuring a channel to use deltas is copied from
the README of the vcdiff-decoder library.

(Once ably-js v2 is released, we should update the instructions in the
vcdiff-decoder library’s README to make it clear they only apply to v1.
I’ve raised #1513 for this.)

Resolves #1492.

README.md

@@ -200,7 +200,39 @@ channel.subscribe('myEvent', function (message) {
 
 Subscribing to a channel in delta mode enables [delta compression](https://www.ably.com/docs/realtime/channels/channel-parameters/deltas). This is a way for a client to subscribe to a channel so that message payloads sent contain only the difference (ie the delta) between the present message and the previous message on the channel.
 
-Configuring a channel for deltas is detailed in the [@ably-forks/vcdiff-decoder documentation](https://github.com/ably-forks/vcdiff-decoder#usage).
+To subscribe to a channel in delta mode, you must:
+
+1. Create a client that supports deltas (this only applies when running in a browser);
+2. Configure the channel to operate in delta mode.
+
+#### Creating a client that supports deltas
+
+This section only applies when running in a browser. The Realtime client on all other platforms includes delta support.
+
+To use delta functionality in the browser, you must use the [modular variant of the library](#modular-tree-shakable-variant) and create a client that includes the `Vcdiff` module:
+
+  ```javascript
+  import { BaseRealtime, WebSocketTransport, FetchRequest, Vcdiff } from 'ably/modules';
+
+  const options = { key: 'YOUR_ABLY_KEY' };
+  const client = new BaseRealtime(options, {
+    WebSocketTransport,
+    FetchRequest,
+    Vcdiff
+  });
+  ```
+
+#### Configuring a channel to operate in delta mode
+
+To configure a channel to operate in delta mode, specify channel parameters of `{ delta: 'vcdiff' }` when fetching the channel:
+
+```javascript
+const channel = realtime.channels.get('your-ably-channel', {
+    params: {
+        delta: 'vcdiff'
+    }
+});
+```
 
 Beyond specifying channel options, the rest is transparent and requires no further changes to your application. The `message.data` instances that are delivered to your listening function continue to contain the values that were originally published.
 
@@ -466,13 +498,6 @@ const nextPage = await statsPage.next();        // retrieves the next page as Pa
 const time = await client.time(); // time is in ms since epoch
 ```
 
-## Delta Plugin
-
-From version 1.2 this client library supports subscription to a stream of Vcdiff formatted delta messages from the Ably service. For certain applications this can bring significant data efficiency savings.
-This is an optional feature so our
-
-See the [@ably-forks/vcdiff-decoder documentation](https://github.com/ably-forks/vcdiff-decoder#usage) for setup and usage examples.
-
 ## Support, feedback and troubleshooting
 
 Please visit http://support.ably.com/ for access to our knowledgebase and to ask for any assistance.

ably.d.ts

@@ -565,16 +565,6 @@ declare namespace Types {
      * @defaultValue 10s
      */
     realtimeRequestTimeout?: number;
-
-    /**
-     * A map between a plugin type and a plugin object.
-     */
-    plugins?: {
-      /**
-       * A plugin capable of decoding `vcdiff`-encoded messages. For more information on how to configure a channel to use delta encoding, see the [documentation for the `@ably-forks/vcdiff-decoder` package](https://github.com/ably-forks/vcdiff-decoder#usage).
-       */
-      vcdiff?: any;
-    };
   }
 
   /**

modules.d.ts

@@ -162,6 +162,20 @@ export declare const FetchRequest: unknown;
  */
 export declare const MessageInteractions: unknown;
 
+/**
+ * Provides a {@link BaseRealtime} instance with the ability to use [delta compression](https://www.ably.com/docs/realtime/channels/channel-parameters/deltas).
+ *
+ * To create a client that includes this module, include it in the `ModulesMap` that you pass to the {@link BaseRealtime.constructor}:
+ *
+ * ```javascript
+ * import { BaseRealtime, WebSocketTransport, FetchRequest, Vcdiff } from 'ably/modules';
+ * const realtime = new BaseRealtime(options, { WebSocketTransport, FetchRequest, Vcdiff });
+ * ```
+ *
+ * For information on how to configure a channel to use delta encoding, see [the documentation in the `README`](https://github.com/ably/ably-js/blob/main/README.md#configuring-a-channel-to-operate-in-delta-mode).
+ */
+export declare const Vcdiff: unknown;
+
 /**
  * Pass a `ModulesMap` to { @link BaseRest.constructor | the constructor of BaseRest } or {@link BaseRealtime.constructor | that of BaseRealtime} to specify which functionality should be made available to that client.
  */
@@ -215,6 +229,11 @@ export interface ModulesMap {
    * See {@link MessageInteractions | documentation for the `MessageInteractions` module}.
    */
   MessageInteractions?: typeof MessageInteractions;
+
+  /**
+   * See {@link Vcdiff | documentation for the `Vcdiff` module}.
+   */
+  Vcdiff?: typeof Vcdiff;
 }
 
 /**

scripts/moduleReport.ts

@@ -17,6 +17,7 @@ const moduleNames = [
   'XHRRequest',
   'FetchRequest',
   'MessageInteractions',
+  'Vcdiff',
 ];
 
 // List of all free-standing functions exported by the library along with the

src/common/lib/client/baserealtime.ts

@@ -11,13 +11,15 @@ import ClientOptions from '../../types/ClientOptions';
 import * as API from '../../../../ably';
 import { ModulesMap, RealtimePresenceModule } from './modulesmap';
 import { TransportNames } from 'common/constants/TransportName';
-import { TransportImplementations } from 'common/platform';
+import Platform, { TransportImplementations } from 'common/platform';
+import { VcdiffDecoder } from '../types/message';
 
 /**
  `BaseRealtime` is an export of the tree-shakable version of the SDK, and acts as the base class for the `DefaultRealtime` class exported by the non tree-shakable version.
  */
 class BaseRealtime extends BaseClient {
   readonly _RealtimePresence: RealtimePresenceModule | null;
+  readonly _decodeVcdiff: VcdiffDecoder | null;
   // Extra transport implementations available to this client, in addition to those in Platform.Transports.bundledImplementations
   readonly _additionalTransportImplementations: TransportImplementations;
   _channels: any;
@@ -28,6 +30,7 @@ class BaseRealtime extends BaseClient {
     Logger.logAction(Logger.LOG_MINOR, 'Realtime()', '');
     this._additionalTransportImplementations = BaseRealtime.transportImplementationsFromModules(modules);
     this._RealtimePresence = modules.RealtimePresence ?? null;
+    this._decodeVcdiff = (modules.Vcdiff ?? (Platform.Vcdiff.supported && Platform.Vcdiff.bundledDecode)) || null;
     this.connection = new Connection(this, this.options);
     this._channels = new Channels(this);
     if (options.autoConnect !== false) this.connect();

src/common/lib/client/modulesmap.ts

@@ -10,6 +10,7 @@ import {
   fromValues as presenceMessageFromValues,
   fromValuesArray as presenceMessagesFromValuesArray,
 } from '../types/presencemessage';
+import { VcdiffDecoder } from '../types/message';
 
 export interface PresenceMessageModule {
   presenceMessageFromValues: typeof presenceMessageFromValues;
@@ -31,6 +32,7 @@ export interface ModulesMap {
   XHRRequest?: typeof XHRRequest;
   FetchRequest?: typeof fetchRequest;
   MessageInteractions?: typeof FilteredSubscriptions;
+  Vcdiff?: VcdiffDecoder;
 }
 
 export const allCommonModules: ModulesMap = { Rest };

src/common/lib/client/realtimechannel.ts

@@ -122,7 +122,7 @@ class RealtimeChannel extends EventEmitter {
     this._attachResume = false;
     this._decodingContext = {
       channelOptions: this.channelOptions,
-      plugins: client.options.plugins || {},
+      decodeVcdiff: client._decodeVcdiff ?? undefined,
       baseEncodedPreviousPayload: undefined,
     };
     this._lastPayload = {

src/common/lib/types/message.ts

@@ -22,21 +22,18 @@ export type CipherOptions = {
   };
 };
 
+export type VcdiffDecoder = (delta: Uint8Array, source: Uint8Array) => Uint8Array;
+
 export type EncodingDecodingContext = {
   channelOptions: ChannelOptions;
-  plugins: {
-    vcdiff?: {
-      decode: (delta: Uint8Array, source: Uint8Array) => Uint8Array;
-    };
-  };
+  decodeVcdiff?: VcdiffDecoder;
   baseEncodedPreviousPayload?: Buffer | BrowserBufferlike;
 };
 
 function normaliseContext(context: CipherOptions | EncodingDecodingContext | ChannelOptions): EncodingDecodingContext {
   if (!context || !(context as EncodingDecodingContext).channelOptions) {
     return {
       channelOptions: context as ChannelOptions,
-      plugins: {},
       baseEncodedPreviousPayload: undefined,
     };
   }
@@ -216,8 +213,12 @@ export async function decode(
               throw new Error('Unable to decrypt message; not an encrypted channel');
             }
           case 'vcdiff':
-            if (!context.plugins || !context.plugins.vcdiff) {
-              throw new ErrorInfo('Missing Vcdiff decoder (https://github.com/ably-forks/vcdiff-decoder)', 40019, 400);
+            if (!context.decodeVcdiff) {
+              if (Platform.Vcdiff.supported) {
+                Utils.throwMissingModuleError('Vcdiff');
+              } else {
+                throw new ErrorInfo(Platform.Vcdiff.errorMessage, 40019, 400);
+              }
             }
             if (typeof Uint8Array === 'undefined') {
               throw new ErrorInfo(
@@ -236,7 +237,7 @@ export async function decode(
               const deltaBaseBuffer = Platform.BufferUtils.toBuffer(deltaBase as Buffer);
               data = Platform.BufferUtils.toBuffer(data);
 
-              data = Platform.BufferUtils.arrayBufferViewToBuffer(context.plugins.vcdiff.decode(data, deltaBaseBuffer));
+              data = Platform.BufferUtils.arrayBufferViewToBuffer(context.decodeVcdiff(data, deltaBaseBuffer));
               lastPayload = data;
             } catch (e) {
               throw new ErrorInfo('Vcdiff delta decode failed with ' + e, 40018, 400);

src/common/platform.ts

@@ -8,6 +8,7 @@ import * as WebBufferUtils from '../platform/web/lib/util/bufferutils';
 import * as NodeBufferUtils from '../platform/nodejs/lib/util/bufferutils';
 import { IUntypedCryptoStatic } from '../common/types/ICryptoStatic';
 import TransportName from './constants/TransportName';
+import { VcdiffDecoder } from './lib/types/message';
 
 type Bufferlike = WebBufferUtils.Bufferlike | NodeBufferUtils.Bufferlike;
 type BufferUtilsOutput = WebBufferUtils.Output | NodeBufferUtils.Output;
@@ -39,4 +40,10 @@ export default class Platform {
   };
   static Defaults: IDefaults;
   static WebStorage: IWebStorage | null;
+  static Vcdiff:
+    | { supported: false; errorMessage: string /* explains why this platform does not support vcdiff */ }
+    | {
+        supported: true;
+        bundledDecode: VcdiffDecoder | null /* { supported: true, bundledDecode: null } means that the decode implementation can be provided via ModulesMap */;
+      };
 }

src/platform/nativescript/index.ts

@@ -4,6 +4,7 @@ import { DefaultRealtime } from '../../common/lib/client/defaultrealtime';
 import Platform from '../../common/platform';
 import ErrorInfo from '../../common/lib/types/errorinfo';
 import { fromDeserializedIncludingDependencies as protocolMessageFromDeserialized } from '../../common/lib/types/protocolmessage';
+import { decode as decodeVcdiff } from '@ably/vcdiff-decoder';
 
 // Platform Specific
 import BufferUtils from '../web/lib/util/bufferutils';
@@ -30,6 +31,7 @@ Platform.Http = Http;
 Platform.Config = Config;
 Platform.Transports = Transports;
 Platform.WebStorage = WebStorage;
+Platform.Vcdiff = { supported: true, bundledDecode: decodeVcdiff };
 
 for (const clientClass of [DefaultRest, DefaultRealtime]) {
   clientClass.Crypto = Crypto;

src/platform/nodejs/index.ts

@@ -4,6 +4,7 @@ import { DefaultRealtime } from '../../common/lib/client/defaultrealtime';
 import Platform from '../../common/platform';
 import ErrorInfo from '../../common/lib/types/errorinfo';
 import { fromDeserializedIncludingDependencies as protocolMessageFromDeserialized } from '../../common/lib/types/protocolmessage';
+import { decode as decodeVcdiff } from '@ably/vcdiff-decoder';
 
 // Platform Specific
 import BufferUtils from './lib/util/bufferutils';
@@ -26,6 +27,7 @@ Platform.Http = Http;
 Platform.Config = Config;
 Platform.Transports = Transports;
 Platform.WebStorage = null;
+Platform.Vcdiff = { supported: true, bundledDecode: decodeVcdiff };
 
 for (const clientClass of [DefaultRest, DefaultRealtime]) {
   clientClass.Crypto = Crypto;

src/platform/react-native/index.ts

@@ -4,6 +4,7 @@ import { DefaultRealtime } from '../../common/lib/client/defaultrealtime';
 import Platform from '../../common/platform';
 import ErrorInfo from '../../common/lib/types/errorinfo';
 import { fromDeserializedIncludingDependencies as protocolMessageFromDeserialized } from '../../common/lib/types/protocolmessage';
+import { decode as decodeVcdiff } from '@ably/vcdiff-decoder';
 
 // Platform Specific
 import BufferUtils from '../web/lib/util/bufferutils';
@@ -30,6 +31,7 @@ Platform.Http = Http;
 Platform.Config = Config;
 Platform.Transports = Transports;
 Platform.WebStorage = WebStorage;
+Platform.Vcdiff = { supported: true, bundledDecode: decodeVcdiff };
 
 for (const clientClass of [DefaultRest, DefaultRealtime]) {
   clientClass.Crypto = Crypto;

src/platform/web/index.ts

@@ -28,6 +28,11 @@ Platform.Http = Http;
 Platform.Config = Config;
 Platform.Transports = Transports;
 Platform.WebStorage = WebStorage;
+// To use vcdiff on web you must use the modular variant of the library
+Platform.Vcdiff = {
+  supported: false,
+  errorMessage: 'For vcdiff functionality in the browser, you must use the modular variant of ably-js',
+};
 
 for (const clientClass of [DefaultRest, DefaultRealtime]) {
   clientClass.Crypto = Crypto;

src/platform/web/modules.ts

@@ -22,6 +22,7 @@ Platform.Http = Http;
 Platform.Config = Config;
 Platform.Transports = ModulesTransports;
 Platform.WebStorage = WebStorage;
+Platform.Vcdiff = { supported: true, bundledDecode: null };
 
 Http.bundledRequestImplementations = modulesBundledRequestImplementations;
 
@@ -49,6 +50,7 @@ export * from './modules/msgpack';
 export * from './modules/realtimepresence';
 export * from './modules/transports';
 export * from './modules/http';
+export * from './modules/vcdiff';
 export { Rest } from '../../common/lib/client/rest';
 export { FilteredSubscriptions as MessageInteractions } from '../../common/lib/client/filteredsubscriptions';
 export { BaseRest, BaseRealtime, ErrorInfo };

src/platform/web/modules/vcdiff.ts

@@ -0,0 +1 @@
+export { decode as Vcdiff } from '@ably/vcdiff-decoder';

test/browser/modules.test.js

@@ -20,9 +20,10 @@ import {
   FetchRequest,
   XHRRequest,
   MessageInteractions,
+  Vcdiff,
 } from '../../build/modules/index.js';
 
-function registerAblyModulesTests(helper) {
+function registerAblyModulesTests(helper, registerDeltaTests) {
   describe('browser/modules', function () {
     this.timeout(10 * 1000);
     const expect = chai.expect;
@@ -705,14 +706,43 @@ function registerAblyModulesTests(helper) {
         });
       });
     });
+
+    // Tests for the Vcdiff module
+    //
+    // Note: Unlike the other tests in this file, which only test how the
+    // absence or presence of a module affects the client, assuming that the
+    // underlying functionality is tested in detail in the test suite for the
+    // default variant of the library, the tests for the Vcdiff module actually
+    // test the library’s delta encoding functionality. This is because on web,
+    // delta encoding functionality is only available in the modular variant of
+    // the library.
+    (() => {
+      const config = {
+        createRealtimeWithDeltaPlugin: (options) => {
+          return new BaseRealtime(options, {
+            WebSocketTransport,
+            FetchRequest,
+            Vcdiff,
+          });
+        },
+        createRealtimeWithoutDeltaPlugin: (options) => {
+          return new BaseRealtime(options, {
+            WebSocketTransport,
+            FetchRequest,
+          });
+        },
+      };
+
+      registerDeltaTests('Vcdiff', config);
+    })();
   });
 }
 
 // This function is called by browser_setup.js once `require` is available
 window.registerAblyModulesTests = async () => {
   return new Promise((resolve) => {
-    require(['shared_helper'], (helper) => {
-      registerAblyModulesTests(helper);
+    require(['shared_helper', 'delta_tests'], (helper, registerDeltaTests) => {
+      registerAblyModulesTests(helper, registerDeltaTests);
       resolve();
     });
   });

test/common/globals/named_dependencies.js

@@ -3,10 +3,6 @@ define(function () {
   return (module.exports = {
     // Ably modules
     ably: { browser: 'build/ably', node: 'build/ably-node' },
-    'vcdiff-decoder': {
-      browser: 'node_modules/@ably/vcdiff-decoder/dist/vcdiff-decoder',
-      node: 'node_modules/@ably/vcdiff-decoder',
-    },
 
     // test modules
     globals: { browser: 'test/common/globals/environment', node: 'test/common/globals/environment' },