qlobber

Node.js globbing for amqp-like topics.

Note: Version 5.0.0 adds async and worker thread support when used on Node 12+.

Example:

var assert = require('assert');
var Qlobber = require('qlobber').Qlobber;
var matcher = new Qlobber();
matcher.add('foo.*', 'it matched!');
assert.deepEqual(matcher.match('foo.bar'), ['it matched!']);
assert(matcher.test('foo.bar', 'it matched!'));

The API is described here.

qlobber is implemented using a trie, as described in the RabbitMQ blog posts here and here.

Installation

npm install qlobber

Another Example

A more advanced example using topics from the RabbitMQ topic tutorial:

var assert = require('assert');
var Qlobber = require('qlobber').Qlobber;
var matcher = new Qlobber();
matcher.add('*.orange.*', 'Q1');
matcher.add('*.*.rabbit', 'Q2');
matcher.add('lazy.#', 'Q2');
assert.deepEqual(['quick.orange.rabbit',
                  'lazy.orange.elephant',
                  'quick.orange.fox',
                  'lazy.brown.fox',
                  'lazy.pink.rabbit',
                  'quick.brown.fox',
                  'orange',
                  'quick.orange.male.rabbit',
                  'lazy.orange.male.rabbit'].map(function (topic)
                  {
                      return matcher.match(topic).sort();
                  }),
                 [['Q1', 'Q2'],
                  ['Q1', 'Q2'],
                  ['Q1'],
                  ['Q2'],
                  ['Q2', 'Q2'],
                  [],
                  [],
                  [],
                  ['Q2']]);

Async Example

Same as the first example but using await:

const assert = require('assert');
const { Qlobber } = require('qlobber').set_native(require('qlobber-native'));
const matcher = new Qlobber.nativeString();

(async () => {
    await matcher.addP('foo.*', 'it matched!');
    assert.deepEqual(await matcher.matchP('foo.bar'), ['it matched!']);
    assert(await matcher.testP('foo.bar', 'it matched!'));
})();

Worker Thread Example

Same again but the matching is done on a separate thread:

const { Qlobber } = require('qlobber').set_native(require('qlobber-native'));
const {
    Worker, isMainThread, parentPort, workerData
} = require('worker_threads');

if (isMainThread) {
    const matcher = new Qlobber.nativeString();
    matcher.add('foo.*', 'it matched!');
    const worker = new Worker(__filename, {
        workerData: matcher.state_address
    });
    worker.on('message', msg => {
        const assert = require('assert');
        assert.deepEqual(msg, [['it matched!'], true]);
    });
} else {
    const matcher = new Qlobber.nativeString(workerData);
    parentPort.postMessage([
        matcher.match('foo.bar'),
        matcher.test('foo.bar', 'it matched!')
    ]);
}

Licence

MIT

Tests

qlobber passes the RabbitMQ topic tests (I converted them from Erlang to Javascript).

To run the tests:

npm test

Lint

grunt lint

Code Coverage

npm run coverage

c8 results are available here.

Coveralls page is here.

Benchmarks

grunt bench

qlobber is also benchmarked in ascoltatori.

Native Qlobbers

The Javascript Qlobbers don't support asynchronous calls and worker threads because Javascript values can't be shared between threads.

In order to support asynchronous calls and worker threads, a native C++ implementation is available in the qlobber-native module.

Add qlobber-native as a dependency to your project and then add it to qlobber like this:

require('qlobber').set_native(require('qlobber-native'));

Note that set_native returns qlobber's exports so you can do something like this:

const { Qlobber } = require('qlobber').set_native(require('qlobber-native'));

Note that qlobber-native requires Gnu C++ version 9+ and Boost 1.70+, including the boost_context runtime library.

Once's you've added it to qlobber, the following classes will be available alongside the Javascript classes:

Qlobber.nativeString
Qlobber.nativeNumber
QlobberDedup.nativeString
QlobberDedup.nativeNumber
QlobberTrue.native

They can only hold values of a single type (currently strings or numbers).

Asynchronous calls

The native classes support the same API as the Javascript classes but have the following additional methods:

addP
removeP
matchP
match_iterP
testP
clearP
visitP
get_restorerP

They correspond to their namesakes but return Promises. Note that match_iterP and visitP return async iterators.

API

Source: lib/qlobber.js

Qlobber
Qlobber.prototype.add
Qlobber.prototype.remove
Qlobber.prototype.match
Qlobber.prototype.match_iter
Qlobber.prototype.test
Qlobber.prototype.test_values
Qlobber.prototype.clear
Qlobber.prototype.visit
Qlobber.prototype.get_restorer
QlobberDedup
QlobberDedup.prototype.test_values
QlobberDedup.prototype.match
QlobberTrue
QlobberTrue.prototype.test_values
QlobberTrue.prototype.match
VisitorStream
RestorerStream
set_native

Qlobber([options])

Creates a new qlobber.

Parameters:

{Object} [options] Configures the qlobber. Use the following properties:
- {String} separator The character to use for separating words in topics. Defaults to '.'. MQTT uses '/' as the separator, for example.
- {String} wildcard_one The character to use for matching exactly one non-empty word in a topic. Defaults to '*'. MQTT uses '+', for example.
- {String} wildcard_some The character to use for matching zero or more words in a topic. Defaults to '#'. MQTT uses '#' too.
- {Boolean} match_empty_levels If true then wilcard_one also matches an empty word in a topic. Defaults to false.
- {Boolean|Map} cache_adds Whether to cache topics when adding topic matchers. This will make adding multiple matchers for the same topic faster at the cost of extra memory usage. Defaults to false. If you supply a Map then it will be used to cache the topics (use this to enumerate all the topics in the qlobber).
- {Integer} cache_splits How many topic.split results to cache. When you pass in a topic, it has to be split on the separator. Caching the results will make using the same topics multiple times faster at the cost of extra memory usage. Defaults to 0 (no caching). The number of split results cached is limited by the value you pass here.
- {Integer} max_words Maximum number of words to allow in a topic. Defaults to 100.
- {Integer} max_wildcard_somes Maximum number of wildcard_some words in a topic. Defaults to 3.

_{Go: TOC}

Qlobber.prototype.add(topic, val)

Add a topic matcher to the qlobber.

Note you can match more than one value against a topic by calling add multiple times with the same topic and different values.

Parameters:

{String} topic The topic to match against.
{Any} val The value to return if the topic is matched.

Return:

{Qlobber} The qlobber (for chaining).

_{Go: TOC | Qlobber.prototype}

Qlobber.prototype.remove(topic, [val])

Remove a topic matcher from the qlobber.

Parameters:

{String} topic The topic that's being matched against.
{Any} [val] The value that's being matched. If you don't specify val then all matchers for topic are removed.

Return:

{Qlobber} The qlobber (for chaining).

_{Go: TOC | Qlobber.prototype}

Qlobber.prototype.match(topic)

Match a topic.

Parameters:

{String} topic The topic to match against.

Return:

{Array} List of values that matched the topic. This may contain duplicates. Use a QlobberDedup if you don't want duplicates.

_{Go: TOC | Qlobber.prototype}

Qlobber.prototype.match_iter()

Match a topic, returning the matches one at a time.

Return:

{Iterator} An iterator on the values that match the topic. There may be duplicate values, even if you use a QlobberDedup.

_{Go: TOC | Qlobber.prototype}

Qlobber.prototype.test(topic, val)

Test whether a topic match contains a value. Faster than calling match and searching the result for the value. Values are tested using test_values.

Parameters:

{String} topic The topic to match against.
{Any} val The value being tested for.

Return:

{Boolean} Whether matching against topic contains val.

_{Go: TOC | Qlobber.prototype}

Qlobber.prototype.test_values(vals, val)

Test whether values found in a match contain a value passed to test. You can override this to provide a custom implementation. Defaults to using indexOf.

Parameters:

{Array} vals The values found while matching.
{Any} val The value being tested for.

Return:

{Boolean} Whether vals contains val.

_{Go: TOC | Qlobber.prototype}

Qlobber.prototype.clear()

Reset the qlobber.

Removes all topic matchers from the qlobber.

Return:

{Qlobber} The qlobber (for chaining).

_{Go: TOC | Qlobber.prototype}

Qlobber.prototype.visit()

Visit each node in the qlobber's trie in turn.

Return:

{Iterator} An iterator on the trie. The iterator returns objects which, if fed (in the same order) to the function returned by get_restorer on a different qlobber, will build that qlobber's trie to the same state. The objects can be serialized using JSON.stringify, if the values you store in the qlobber are also serializable.

_{Go: TOC | Qlobber.prototype}

Qlobber.prototype.get_restorer([options])

Get a function which can restore the qlobber's trie to a state you retrieved by calling visit on this or another qlobber.

Parameters:

{Object} [options] Options for restoring the trie.
- {Boolean} cache_adds Whether to cache topics when rebuilding the trie. This only applies if you also passed cache_adds as true in the constructor.

Return:

{Function} Function to call in order to rebuild the qlobber's trie. You should call this repeatedly with the objects you received from a call to visit. If you serialized the objects, remember to deserialize them first (e.g. with JSON.parse)!

_{Go: TOC | Qlobber.prototype}

QlobberDedup([options])

Creates a new de-duplicating qlobber.

Inherits from Qlobber.

Parameters:

{Object} [options] Same options as Qlobber.

_{Go: TOC}

QlobberDedup.prototype.test_values(vals, val)

Test whether values found in a match contain a value passed to test. You can override this to provide a custom implementation. Defaults to using has.

Parameters:

{Set} vals The values found while matching (ES6 Set).
{Any} val The value being tested for.

Return:

{Boolean} Whether vals contains val.

_{Go: TOC | QlobberDedup.prototype}

QlobberDedup.prototype.match(topic)

Match a topic.

Parameters:

{String} topic The topic to match against.

Return:

{Set} ES6 Set of values that matched the topic.

_{Go: TOC | QlobberDedup.prototype}

QlobberTrue([options])

Creates a new qlobber which only stores the value true.

Whatever value you add to this qlobber (even undefined), a single, de-duplicated true will be stored. Use this qlobber if you only need to test whether topics match, not about the values they match to.

Inherits from Qlobber.

Parameters:

{Object} [options] Same options as Qlobber.

_{Go: TOC}

QlobberTrue.prototype.test_values()

This override of test_values always returns true. When you call test on a QlobberTrue instance, the value you pass is ignored since it only cares whether a topic is matched.

Return:

{Boolean} Always true.

_{Go: TOC | QlobberTrue.prototype}

QlobberTrue.prototype.match(topic)

Match a topic.

Since QlobberTrue only cares whether a topic is matched and not about values it matches to, this override of match just calls test (with value undefined).

Parameters:

{String} topic The topic to match against.

Return:

{Boolean} Whether the QlobberTrue instance matches the topic.

_{Go: TOC | QlobberTrue.prototype}

VisitorStream(qlobber)

Creates a new Readable stream, in object mode, which calls visit on a qlobber to generate its data.

You could pipe this to a JSONStream.stringify stream, for instance, to serialize the qlobber to JSON. See this test for an example.

Inherits from Readable.

Parameters:

{Qlobber} qlobber The qlobber to call visit on.

_{Go: TOC}

RestorerStream(qlobber)

Creates a new Writable stream, in object mode, which passes data written to it into the function returned by calling get_restorer on a qlobber.

You could pipe a JSONStream.parse stream to this, for instance, to deserialize the qlobber from JSON. See this test for an example.

Inherits from Writable.

Parameters:

{Qlobber} qlobber The qlobber to call get_restorer on.

_{Go: TOC}

set_native(qlobber_native)

Add qlobber-native to qlobber.

Parameters:

{Object} qlobber_native The qlobber-native module, obtained using require('qlobber-native').

Return:

{Object} The qlobber exports with the following native classes added:

Qlobber.nativeString
Qlobber.nativeNumber
QlobberDedup.nativeString
QlobberDedup.nativeNumber
QlobberTrue.native

_{Go: TOC}

—generated by apidox—

Name		Name	Last commit message	Last commit date
Latest commit History 273 Commits
.github/workflows		.github/workflows
aedes		aedes
bench		bench
coverage		coverage
examples		examples
lib		lib
native		native
test		test
.gitignore		.gitignore
.npmignore		.npmignore
Gruntfile.js		Gruntfile.js
LICENCE		LICENCE
README.md		README.md
index.js		index.js
package-lock.json		package-lock.json
package.json		package.json

License

davedoesdev/qlobber

Folders and files

Latest commit

History

Repository files navigation

qlobber

Installation

Another Example

Async Example

Worker Thread Example

Licence

Tests

Lint

Code Coverage

Benchmarks

Native Qlobbers

Asynchronous calls

API

Qlobber([options])

Qlobber.prototype.add(topic, val)

Qlobber.prototype.remove(topic, [val])

Qlobber.prototype.match(topic)

Qlobber.prototype.match_iter()

Qlobber.prototype.test(topic, val)

Qlobber.prototype.test_values(vals, val)

Qlobber.prototype.clear()

Qlobber.prototype.visit()

Qlobber.prototype.get_restorer([options])

QlobberDedup([options])

QlobberDedup.prototype.test_values(vals, val)

QlobberDedup.prototype.match(topic)

QlobberTrue([options])

QlobberTrue.prototype.test_values()

QlobberTrue.prototype.match(topic)

VisitorStream(qlobber)

RestorerStream(qlobber)

set_native(qlobber_native)

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Contributors 3

Languages

Packages