Skip to content

Commit

Permalink
Strip less-important logging from modular variant
Browse files Browse the repository at this point in the history
In order to reduce bundle size further, we’ve decided to strip all
logging from the modular variant of the SDK, except for errors and
certain network events. (We also considered providing a separate
tree-shakable module with all of this logging code so that a user of the
modular variant of the library can opt in to it, but decided against it
for now; we might add it in later. This does mean that there is
currently no version of the SDK that allows you to use both deltas and
verbose logging on web.)

I couldn’t find any out-of-the-box esbuild functionality that let us do
this. The only stuff I could find related to stripping code was:

- the `pure` option, but that code only gets stripped if you minify the
  code (and even in that case I couldn’t actually get it to be stripped,
  perhaps would have been able to with further trying though), but
  minifying our generated modules bundle causes the bundle size of those
  who use it (as tested by our modulereport script) to increase
  considerably (for reasons I’m not sure of)

- the `drop` option, but that only lets you remove calls to `console` or
  `debugger`

So instead I’ve implemented it as an esbuild plugin.

Resolves #1526
  • Loading branch information
lawrence-forooghian committed Jan 15, 2024
1 parent 6edac83 commit 0dda975
Show file tree
Hide file tree
Showing 14 changed files with 189 additions and 31 deletions.
1 change: 1 addition & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ module.exports = {
"typedoc/generated",
"react",
"Gruntfile.js",
"grunt",
],
settings: {
jsdoc: {
Expand Down
3 changes: 2 additions & 1 deletion Gruntfile.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ var esbuild = require('esbuild');
var umdWrapper = require('esbuild-plugin-umd-wrapper');
var banner = require('./src/fragments/license');
var process = require('process');
var stripLogsPlugin = require('./grunt/esbuild/strip-logs').default;

module.exports = function (grunt) {
grunt.loadNpmTasks('grunt-contrib-concat');
Expand Down Expand Up @@ -133,7 +134,7 @@ module.exports = function (grunt) {
entryPoints: ['src/platform/web/modules.ts'],
outfile: 'build/modules/index.js',
format: 'esm',
plugins: [],
plugins: [stripLogsPlugin],
};
}

Expand Down
9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,14 @@ You must provide:

`BaseRealtime` offers the same API as the `Realtime` class described in the rest of this `README`. This means that you can develop an application using the default variant of the SDK and switch to the modular version when you wish to optimize your bundle size.

For more information, see the [generated documentation](https://sdk.ably.com/builds/ably/ably-js/main/typedoc/modules/modules.html) (this link points to the documentation for the `main` branch).
In order to further reduce bundle size, the modular variant of the SDK performs less logging than the default variant. It only logs:

- messages that have a `logLevel` of 1 (that is, errors)
- a small number of other network events

If you need more verbose logging, use the default variant of the SDK.

For more information about the modular variant of the SDK, see the [generated documentation](https://sdk.ably.com/builds/ably/ably-js/main/typedoc/modules/modules.html) (this link points to the documentation for the `main` branch).

### TypeScript

Expand Down
111 changes: 111 additions & 0 deletions grunt/esbuild/strip-logs.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
var path = require('path');
var fs = require('fs');
var babel = {
types: require('@babel/types'),
parser: require('@babel/parser'),
traverse: require('@babel/traverse'),
generator: require('@babel/generator'),
};

// This function is copied from
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions#escaping
function escapeRegExp(string) {
return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string
}

// This esbuild plugin strips all log messages from the modular variant of
// the library, except for error-level logs and other logging statements
// explicitly marked as not to be stripped.
const stripLogsPlugin = {
name: 'stripLogs',
setup(build) {
let foundLogToStrip = false;
let foundErrorLog = false;
let foundNoStripLog = false;

const filter = new RegExp(`^${escapeRegExp(path.join(__dirname, '..', '..', 'src') + path.sep)}.*\\.[tj]s$`);
build.onLoad({ filter }, async (args) => {
const contents = (await fs.promises.readFile(args.path)).toString();
const lines = contents.split('\n');
const ast = babel.parser.parse(contents, { sourceType: 'module', plugins: ['typescript'] });
const errors = [];

babel.traverse.default(ast, {
enter(path) {
if (
path.isCallExpression() &&
babel.types.isMemberExpression(path.node.callee) &&
babel.types.isIdentifier(path.node.callee.object, { name: 'Logger' })
) {
if (babel.types.isIdentifier(path.node.callee.property, { name: 'logAction' })) {
const firstArgument = path.node.arguments[0];

if (
babel.types.isMemberExpression(firstArgument) &&
babel.types.isIdentifier(firstArgument.object, { name: 'Logger' }) &&
firstArgument.property.name.startsWith('LOG_')
) {
if (firstArgument.property.name === 'LOG_ERROR') {
// `path` is a call to `Logger.logAction(Logger.LOG_ERROR, ...)`; preserve it.
foundErrorLog = true;
} else {
// `path` is a call to `Logger.logAction(Logger.LOG_*, ...) for some other log level; strip it.
foundLogToStrip = true;
path.remove();
}
} else {
// `path` is a call to `Logger.logAction(...)` with some argument other than a `Logger.LOG_*` expression; raise an error because we can’t determine whether to strip it.
errors.push({
location: {
file: args.path,
column: firstArgument.loc.start.column,
line: firstArgument.loc.start.line,
lineText: lines[firstArgument.loc.start.line - 1],
},
text: `First argument passed to Logger.logAction() must be Logger.LOG_*, got \`${
babel.generator.default(firstArgument).code
}\``,
});
}
} else if (babel.types.isIdentifier(path.node.callee.property, { name: 'logActionNoStrip' })) {
// `path` is a call to `Logger.logActionNoStrip(...)`; preserve it.
foundNoStripLog = true;
}
}
},
});

return { contents: babel.generator.default(ast).code, loader: 'ts', errors };
});

build.onEnd(() => {
const errorMessages = [];

// Perform a sense check to make sure that we found some logging
// calls to strip (to protect us against accidentally changing the
// internal logging API in such a way that would cause us to no
// longer strip any calls).

if (!foundLogToStrip) {
errorMessages.push('Did not find any Logger.logAction(...) calls to strip');
}

// Perform a sense check to make sure that we found some logging
// calls to preserve (to protect us against accidentally changing the
// internal logging API in such a way that would cause us to
// accidentally strip all logging calls).

if (!foundErrorLog) {
errorMessages.push('Did not find any Logger.logAction(Logger.LOG_ERROR, ...) calls to preserve');
}

if (!foundNoStripLog) {
errorMessages.push('Did not find any Logger.logActionNoStrip(...) calls to preserve');
}

return { errors: errorMessages.map((text) => ({ text })) };
});
},
};

exports.default = stripLogsPlugin;
18 changes: 18 additions & 0 deletions modules.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,15 @@ export interface ModulesMap {
* A client that offers a simple stateless API to interact directly with Ably's REST API.
*
* `BaseRest` is the equivalent, in the modular variant of the Ably Client Library SDK, of the [`Rest`](../../default/classes/Rest.html) class in the default variant of the SDK. The difference is that its constructor allows you to decide exactly which functionality the client should include. This allows unused functionality to be tree-shaken, reducing bundle size.
*
* > **Note**
* >
* > In order to further reduce bundle size, `BaseRest` performs less logging than the `Rest` class exported by the default variant of the SDK. It only logs:
* >
* > - messages that have a {@link Types.ClientOptions.logLevel | `logLevel`} of 1 (that is, errors)
* > - a small number of other network events
* >
* > If you need more verbose logging, use the default variant of the SDK.
*/
export declare class BaseRest extends AbstractRest {
/**
Expand All @@ -292,6 +301,15 @@ export declare class BaseRest extends AbstractRest {
* A client that extends the functionality of {@link BaseRest} and provides additional realtime-specific features.
*
* `BaseRealtime` is the equivalent, in the modular variant of the Ably Client Library SDK, of the [`Realtime`](../../default/classes/Realtime.html) class in the default variant of the SDK. The difference is that its constructor allows you to decide exactly which functionality the client should include. This allows unused functionality to be tree-shaken, reducing bundle size.
*
* > **Note**
* >
* > In order to further reduce bundle size, `BaseRealtime` performs less logging than the `Realtime` class exported by the default variant of the SDK. It only logs:
* >
* > - messages that have a {@link Types.ClientOptions.logLevel | `logLevel`} of 1 (that is, errors)
* > - a small number of other network events
* >
* > If you need more verbose logging, use the default variant of the SDK.
*/
export declare class BaseRealtime extends AbstractRealtime {
/**
Expand Down
15 changes: 9 additions & 6 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@
"devDependencies": {
"@ably/vcdiff-decoder": "1.0.6",
"@arethetypeswrong/cli": "^0.13.1",
"@babel/generator": "^7.23.6",
"@babel/parser": "^7.23.6",
"@babel/traverse": "^7.23.7",
"@testing-library/react": "^13.3.0",
"@types/jmespath": "^0.15.2",
"@types/node": "^18.0.0",
Expand Down Expand Up @@ -132,8 +135,8 @@
"lint": "eslint .",
"lint:fix": "eslint --fix .",
"prepare": "npm run build",
"format": "prettier --write --ignore-path .gitignore --ignore-path .prettierignore src test ably.d.ts modules.d.ts webpack.config.js Gruntfile.js scripts/*.[jt]s docs/chrome-mv3.md",
"format:check": "prettier --check --ignore-path .gitignore --ignore-path .prettierignore src test ably.d.ts modules.d.ts webpack.config.js Gruntfile.js scripts/*.[jt]s",
"format": "prettier --write --ignore-path .gitignore --ignore-path .prettierignore src test ably.d.ts modules.d.ts webpack.config.js Gruntfile.js scripts/*.[jt]s docs/chrome-mv3.md grunt",
"format:check": "prettier --check --ignore-path .gitignore --ignore-path .prettierignore src test ably.d.ts modules.d.ts webpack.config.js Gruntfile.js scripts/*.[jt]s grunt",
"sourcemap": "source-map-explorer build/ably.min.js",
"modulereport": "tsc --noEmit scripts/moduleReport.ts && esr scripts/moduleReport.ts",
"docs": "typedoc"
Expand Down
2 changes: 1 addition & 1 deletion scripts/moduleReport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import * as path from 'path';
import { explore } from 'source-map-explorer';

// The maximum size we allow for a minimal useful Realtime bundle (i.e. one that can subscribe to a channel)
const minimalUsefulRealtimeBundleSizeThresholdKiB = 108;
const minimalUsefulRealtimeBundleSizeThresholdKiB = 93;

// List of all modules accepted in ModulesMap
const moduleNames = [
Expand Down
13 changes: 7 additions & 6 deletions src/common/lib/client/realtimechannel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -750,12 +750,13 @@ class RealtimeChannel extends EventEmitter {
this.errorReason = reason;
}
const change = new ChannelStateChange(this.state, state, resumed, hasBacklog, reason);
const logLevel = state === 'failed' ? Logger.LOG_ERROR : Logger.LOG_MAJOR;
Logger.logAction(
logLevel,
'Channel state for channel "' + this.name + '"',
state + (reason ? '; reason: ' + reason : '')
);
const action = 'Channel state for channel "' + this.name + '"';
const message = state + (reason ? '; reason: ' + reason : '');
if (state === 'failed') {
Logger.logAction(Logger.LOG_ERROR, action, message);
} else {
Logger.logAction(Logger.LOG_MAJOR, action, message);
}

if (state !== 'attaching' && state !== 'suspended') {
this.retryCount = 0;
Expand Down
8 changes: 4 additions & 4 deletions src/common/lib/client/resource.ts
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,13 @@ function logResponseHandler<T>(
): ResourceCallback {
return (err, body, headers, unpacked, statusCode) => {
if (err) {
Logger.logAction(
Logger.logActionNoStrip(
Logger.LOG_MICRO,
'Resource.' + method + '()',
'Received Error; ' + urlFromPathAndParams(path, params) + '; Error: ' + Utils.inspectError(err)
);
} else {
Logger.logAction(
Logger.logActionNoStrip(
Logger.LOG_MICRO,
'Resource.' + method + '()',
'Received; ' +
Expand Down Expand Up @@ -215,7 +215,7 @@ class Resource {

function doRequest(this: any, headers: Record<string, string>, params: Record<string, any>) {
if (Logger.shouldLog(Logger.LOG_MICRO)) {
Logger.logAction(
Logger.logActionNoStrip(
Logger.LOG_MICRO,
'Resource.' + method + '()',
'Sending; ' + urlFromPathAndParams(path, params)
Expand All @@ -238,7 +238,7 @@ class Resource {
);
}
}
Logger.logAction(
Logger.logActionNoStrip(
Logger.LOG_MICRO,
'Resource.' + method + '()',
'Sending; ' + urlFromPathAndParams(path, params) + '; Body: ' + decodedBody
Expand Down
13 changes: 7 additions & 6 deletions src/common/lib/transport/connectionmanager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1187,12 +1187,13 @@ class ConnectionManager extends EventEmitter {
}

enactStateChange(stateChange: ConnectionStateChange): void {
const logLevel = stateChange.current === 'failed' ? Logger.LOG_ERROR : Logger.LOG_MAJOR;
Logger.logAction(
logLevel,
'Connection state',
stateChange.current + (stateChange.reason ? '; reason: ' + stateChange.reason : '')
);
const action = 'Connection state';
const message = stateChange.current + (stateChange.reason ? '; reason: ' + stateChange.reason : '');
if (stateChange.current === 'failed') {
Logger.logAction(Logger.LOG_ERROR, action, message);
} else {
Logger.logAction(Logger.LOG_MAJOR, action, message);
}
Logger.logAction(
Logger.LOG_MINOR,
'ConnectionManager.enactStateChange',
Expand Down
2 changes: 1 addition & 1 deletion src/common/lib/transport/protocol.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ class Protocol extends EventEmitter {
this.messageQueue.push(pendingMessage);
}
if (Logger.shouldLog(Logger.LOG_MICRO)) {
Logger.logAction(
Logger.logActionNoStrip(
Logger.LOG_MICRO,
'Protocol.send()',
'sending msg; ' +
Expand Down
4 changes: 2 additions & 2 deletions src/common/lib/transport/transport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ abstract class Transport extends EventEmitter {

onProtocolMessage(message: ProtocolMessage): void {
if (Logger.shouldLog(Logger.LOG_MICRO)) {
Logger.logAction(
Logger.logActionNoStrip(
Logger.LOG_MICRO,
'Transport.onProtocolMessage()',
'received on ' +
Expand All @@ -132,7 +132,7 @@ abstract class Transport extends EventEmitter {

switch (message.action) {
case actions.HEARTBEAT:
Logger.logAction(
Logger.logActionNoStrip(
Logger.LOG_MICRO,
'Transport.onProtocolMessage()',
this.shortName + ' heartbeat; connectionId = ' + this.connectionManager.connectionId
Expand Down
14 changes: 13 additions & 1 deletion src/common/lib/util/logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,11 +98,23 @@ class Logger {
}

/* public static functions */
/**
* In the modular variant of the SDK, the `stripLogs` esbuild plugin strips out all calls to this method (when invoked as `Logger.logAction(...)`) except when called with level `Logger.LOG_ERROR`.
*
* The aforementioned plugin expects `level` to be an expression of the form `Logger.LOG_*`; that is, you can’t dynamically specify the log level.
*/
static logAction = (level: LogLevels, action: string, message?: string) => {
this.logActionNoStrip(level, action, message);
};

/**
* Calls to this method are never stripped by the `stripLogs` esbuild plugin. Use it for log statements that you wish to always be included in the modular variant of the SDK.
*/
static logActionNoStrip(level: LogLevels, action: string, message?: string) {
if (Logger.shouldLog(level)) {
(level === LogLevels.Error ? Logger.logErrorHandler : Logger.logHandler)('Ably: ' + action + ': ' + message);
}
};
}

/* Where a logging operation is expensive, such as serialisation of data, use shouldLog will prevent
the object being serialised if the log level will not output the message */
Expand Down

0 comments on commit 0dda975

Please sign in to comment.