Think of this connector generator is a kind of create-react-app for Hull connectors. It helps you go from 0 to fully setup environment in about 30 seconds.
It generates a fully working connector setup that you can immediately boot and use to receive Hull data streams and process them. It has sane defaults, with
- prettier
- eslint
- babel
- webpack
- flow
- mocha / sinon / chai / nock (for server testing)
- jest (for client testing)
Plus a bunch of hull-specific tooling to make it easy to simulate traffic:
At the same time, it doesn't force you into any specific setup. It's just an express app at the core.
If you need to build a Dashboard UI, React hot-reload and Webpack are pre-configured, you just have to start writing your code.
We use builder-Init to generate the boilerplate app. Installing and generating a connector takes 2 lines:
~/test ❯❯❯ yarn global add builder builder-init
~/test ❯❯❯ builder-init hull-connector;
hull-connector-1.0.44.tgz
[builder-init] Preparing templates for: hull-connector
? Human-readable Name (e.g., 'Intercom') Test
? Package / GitHub project name hull-test
? GitHub organization name hull-ships
? Package description Synchronize Data with Test
? License organization (e.g., you or your company) hull-ships
? Destination directory to write hull-test
[builder-init] Wrote files:
- hull-test/.babelrc
- hull-test/.builderrc
- hull-test/.editorconfig
- hull-test/.eslintignore
- hull-test/.flowconfig
- hull-test/jest.config.json
- hull-test/manifest.json
- hull-test/newrelic.js
- [....]
[builder-init] New hull-connector project is ready at: hull-test
~/test ❯❯❯ cd hull-test
~/test ❯❯❯ yarn
~/test ❯❯❯ yarn dev //Start development server
~/test ❯❯❯ yarn build //Build project
~/test ❯❯❯ yarn start //Start production server
When your connector is ready (as soon as you have ran yarn in it's folder), you can start using it as follows:
| command | description | Notes |
| -- |--- |-- |-- |
| yarn dev | Start in development mode |
| yarn ngrok | Start a ngrok server | be sure to have your account setup to choose the subdomain
| yarn build | Build client and server assets |
| yarn start | Start in production mode |
| yarn build:client | Build Client-files from src to dist | Uses Webpack & Babel
| yarn build:server | Build Server assets from server to lib | Uses Babel
| yarn clean | Remove build files |
| yarn prettier | Prettifies source files |
| yarn flow | Checks Flow annotations |
| yarn lint | Lint and surface errors |
| yarn test:electrode | Starts Hull's Repl, Electrode |
| yarn test | Run Server & Client tests |
| yarn test:client | Jest client tests |
| yarn test:units | Unit Tests |
| yarn test:units:watch | Starts a mocha --watch server | so you can quickly work on unit tests |
| yarn test:specs | Runs integration tests | Use minihull |
| yarn test:specs:watch | Starts a mocha --watch server | so you can quickly work on unit tests |
Electrode is an easy-to-use REPL you can use to send any kind of traffic to connectors, while being in a real hull environment, and see what the connector sends to Hull in response.
- It takes a real connector ID & Secret to boot
- It fetches the actual settings for this ID & Secret and uses them on the connector.
- It accepts User or Account notification payloads,
- It displays what the connetor sends back to Hull
Getting an example payload is very easy: just install a Processor connector, search for the User you want and copy the payload on the left column.
~/hull-test ❯❯❯ yarn test:electrode --port 8083
# yarn run v1.3.2
# $ node ./tests/electrode
? SHIP_ID my-ship-id
? SHIP_SECRET my-ship-secret
? SHIP_ORG my-org.hullapp.io
[...]
"schedules": [
{
"url": "/status",
"type": "interval",
"value": "5",
"next_run_time": 1517693700,
"last_run_time": null
}
]
},
"resources": {}
}
╔═════════════════════════════════════════════════════════════╗
║ ║
║ ║
║ Connector Started on port 8083 with instance my-ship-id ║
║ Organization: my-org.hullapp.io ║
║ ║
║ ║
╚═════════════════════════════════════════════════════════════╝
{"port":8083,"level":"info","message":"connector.server.listen"}
hull ❯ _
hull ❯ .load file.json user # loads `file.json` as `user`
hull ❯ send(user) # sends user as a User Notification
# What's available out of the box :
# colorize -> colorize console dumps with console.log(colorize(object));
# segments -> segments on currently booted Hull organization
# minihull -> hull server
# hull -> hull client
# connector -> connector object
# organization -> org name
# moment -> moment.js
# lo -> lodash (renamed to avoid conflicts)
# urijs
#
# Of course you can require your own modules:
hull ❯ const boxen = require('boxen');Mockr is a testing addition to make it easy to simulate calls and settings and write assertions on the connector's responses to Hull
Boilerplate comes with mocha/chai/sinon/nock already setup for server tests.
You can then simply require hull-connector-dev/lib/mockr package and start writing assertions.
It sets up some mocks and minihull, which is a stripped down version of hull that's able to send messages to connectors and offer expectations on what the connector should send to the Firehose.
import { expect } from "chai";
import mockr from "hull-connector-dev/lib/mockr";
// Your server's entry point, with the same format as the one `hull-connector` bundles.
// Options will be passed to it.
import server from "../../server/server";
describe("Test Group", () => {
// Start the mocks. they will run `beforeEach` and `afterEach` cleanups for you,
// Start a development server
const mocks = mockr({
server
beforeEach,
afterEach,
port: 8000,
segments: [{ id: "1", name: "A" }], // Segments that should exist on the server
});
it("should behave properly", async () => {
const myNock = mocks
.nock("https://api.myremote.test.com")
.get("/test")
.query({ foo: "bar" })
.reply(200, [{ email: "foo@foo.bar", id: "foobar" }]);
// Optional, if you want to stub more things that your connector
// will access during it's flow.
mocks.minihull.stubApp("/api/v1/search/user_reports").respond({
pagination: { total: 0 },
aggregations: {
without_email: { doc_count: 0 },
by_source: { buckets: [] },
},
});
// Send a `user:update` call to the connector.
const { batch, logs } = await mocks.minihull.userUpdate(
{
// Connector Settings
connector: {
id: "123456789012345678901234",
private_settings: {
api_key: "123",
handle_accounts: true,
prospect_enabled: true,
prospect_segments: ["1"],
prospect_filter_titles: ["foo"],
prospect_limit_count: 2,
},
},
// Message payload
messages: [
{
user: { id: "abc", "traits/clearbit/source": "reveal" },
account: { id: "ACCOUNTID", domain: "domain.com" },
segments: [{ id: "1" }],
},
],
}
);
// const { batch, logs } = await mocks.minihull.accountUpdate is supported too
// This is what the Firehose receives.
const [first, second, third, fourth] = batch;
expect(batch.length).to.equal(4);
expect(logs[1].message).to.equal("outgoing.user.start");
myNock.done();
});
});If you use Atom, i strongly recommend you install mocha-test-runner
which will let you run test by hitting ctrl-alt-m
yarn gen-dev;
npm link ; cd dev ; npm link
# Bump version in package.json
npm run publish:both
- Will monitor, bundle and build all
.jsfiles in thesrcfolder for Client-side code - Will build the
serverfolder aslibfor Server-side code