Skip to content

Commit

Permalink
Merge branch 'master' into ai-assistant-improvements
Browse files Browse the repository at this point in the history
* master: (21 commits)
  feat: Add queue mode setup to benchmarks (no-changelog) (#10608)
  feat: Add n8n postgres setup to benchmarks (no-changelog) (#10604)
  fix(API): Update express-openapi-validator to resolve AIKIDO-2024-10229 (#10612)
  fix: Fix edge case in log in (no-changelog) (#10610)
  feat: Add local orchestration of benchmarks (no-changelog) (#10589)
  ci: Run nightly benchmark against nightly n8n image (no-changelog) (#10588)
  fix: Reduce variability in benchmarks (no-changelog) (#10606)
  docs: Add missing changelog entry (#10609)
  refactor(editor): Convert ResourceLocator to composition API (no-changelog) (#10526)
  feat(editor): Update new canvas node handle label rendering mechanism and design (no-changelog) (#10611)
  refactor(editor): Convert credential related components to composition API (no-changelog) (#10530)
  fix(HTTP Request Node): Sanitize authorization headers (#10607)
  refactor: Use `NodeConnectionType` consistently across the code base (no-changelog) (#10595)
  fix(editor): Hide execution buttons in readonly mode in new canvas (no-changelog) (#10603)
  fix(editor): Prevent keyboard shortcuts when ndv is open in new canvas (no-changelog) (#10601)
  fix(editor): Add confirmation toast when changing user role (#10592)
  feat(editor): Add support for changing sticky notes color in new canvas (no-changelog) (#10593)
  ci: Fix `forceConsistentCasingInFileNames` for aliased paths (no-changelog) (#10598)
  feat(editor): Allow sticky notes alongside fallback nodes in new canvas (no-changelog) (#10583)
  ci: Push nightly images to ghcr (no-changelog) (#10580)
  ...
  • Loading branch information
MiloradFilipovic committed Aug 30, 2024
2 parents c79d985 + 7e9d186 commit 0c53737
Show file tree
Hide file tree
Showing 826 changed files with 5,860 additions and 4,429 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/benchmark-nightly.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Run Nightly Benchmark
run-name: Benchmark ${{ inputs.n8n_tag }}
run-name: Benchmark ${{ inputs.n8n_tag || 'nightly' }}

on:
schedule:
Expand Down Expand Up @@ -60,10 +60,10 @@ jobs:

- name: Run the benchmark with debug logging
if: github.event.inputs.debug == 'true'
run: pnpm run-in-cloud --debug
run: pnpm run-in-cloud --n8nTag ${{ inputs.n8n_tag || 'nightly' }} --benchmarkTag ${{ inputs.benchmark_tag || 'latest' }} --debug
working-directory: packages/@n8n/benchmark

- name: Run the benchmark
if: github.event.inputs.debug != 'true'
run: pnpm run-in-cloud
run: pnpm run-in-cloud --n8nTag ${{ inputs.n8n_tag || 'nightly' }} --benchmarkTag ${{ inputs.benchmark_tag || 'latest' }}
working-directory: packages/@n8n/benchmark
17 changes: 16 additions & 1 deletion .github/workflows/docker-images-nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
[[ "${{github.event.inputs.merge-master}}" == "true" ]] && git remote add upstream https://github.com/n8n-io/n8n.git -f; git merge upstream/master --allow-unrelated-histories || echo ""
shell: bash

- name: Build and push
- name: Build and push to DockerHub
uses: docker/build-push-action@v5.1.0
with:
context: .
Expand All @@ -78,6 +78,21 @@ jobs:
cache-to: type=gha,mode=max
tags: ${{ secrets.DOCKER_USERNAME }}/n8n:${{ github.event.inputs.tag || 'nightly' }}

- name: Login to GitHub Container Registry
if: github.event.inputs.tag == 'nightly'
uses: docker/login-action@v3.0.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Push image to GHCR
if: github.event.inputs.tag == 'nightly'
run: |
docker buildx imagetools create \
--tag ghcr.io/${{ github.repository_owner }}/n8n:nightly \
${{ secrets.DOCKER_USERNAME }}/n8n:nightly
- name: Call Success URL - optionally
run: |
[[ "${{github.event.inputs.success-url}}" != "" ]] && curl -v ${{github.event.inputs.success-url}} || echo ""
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@

* **core:** Make execution queries faster ([#9817](https://github.com/n8n-io/n8n/issues/9817)) ([dc7dc99](https://github.com/n8n-io/n8n/commit/dc7dc995d5e2ea8fbd0dcb54cfa8aa93ecb437c9))

### Other
* **Add user journey link to [n8n.io](https://n8n.io)** ([#10331](https://github.com/n8n-io/n8n/pull/10331))


# [1.56.0](https://github.com/n8n-io/n8n/compare/n8n@1.55.0...n8n@1.56.0) (2024-08-21)
Expand Down
55 changes: 37 additions & 18 deletions packages/@n8n/benchmark/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,38 @@
# n8n benchmarking tool

Tool for executing benchmarks against an n8n instance.
Tool for executing benchmarks against an n8n instance. The tool consists of these components:

## Running locally with Docker
## Directory structure

```text
packages/@n8n/benchmark
├── scenarios Benchmark scenarios
├── src Source code for the n8n-benchmark cli
├── Dockerfile Dockerfile for the n8n-benchmark cli
├── scripts Orchestration scripts
```

## Running the entire benchmark suite

The benchmark suite consists of [benchmark scenarios](#benchmark-scenarios) and different [n8n setups](#n8n-setups).

### locally

```sh
pnpm run-locally
```

### In the cloud

```sh
pnpm run-in-cloud
```

## Running the `n8n-benchmark` cli

The `n8n-benchmark` cli is a node.js program that runs one or more scenarios against a single n8n instance.

### Locally with Docker

Build the Docker image:

Expand All @@ -23,7 +53,7 @@ docker run \
n8n-benchmark
```

## Running locally without Docker
### Locally without Docker

Requirements:

Expand All @@ -35,23 +65,8 @@ pnpm build

# Run tests against http://localhost:5678 with specified email and password
N8N_USER_EMAIL=user@n8n.io N8N_USER_PASSWORD=password ./bin/n8n-benchmark run

# If you installed k6 using brew, you might have to specify it explicitly
K6_PATH=/opt/homebrew/bin/k6 N8N_USER_EMAIL=user@n8n.io N8N_USER_PASSWORD=password ./bin/n8n-benchmark run
```

## Running in the cloud

There's a script to run the performance tests in a cloud environment. The script provisions a cloud environment, sets up n8n in the environment, runs the tests and destroys the environment.

```sh
pnpm run-in-cloud
```

## Configuration

The configuration options the cli accepts can be seen from [config.ts](./src/config/config.ts)

## Benchmark scenarios

A benchmark scenario defines one or multiple steps to execute and measure. It consists of:
Expand All @@ -61,3 +76,7 @@ A benchmark scenario defines one or multiple steps to execute and measure. It co
- A [`k6`](https://grafana.com/docs/k6/latest/using-k6/http-requests/) script which executes the steps and receives `API_BASE_URL` environment variable in runtime.

Available scenarios are located in [`./scenarios`](./scenarios/).

## n8n setups

A n8n setup defines a single n8n runtime configuration using Docker compose. Different n8n setups are located in [`./scripts/n8nSetups`](./scripts/n8nSetups).
4 changes: 2 additions & 2 deletions packages/@n8n/benchmark/infra/modules/benchmark-vm/vars.tf
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ variable "ssh_public_key" {

variable "vm_size" {
description = "VM Size"
# 4 vCPUs, 16 GiB memory
default = "Standard_DC4s_v2"
# 8 vCPUs, 32 GiB memory
default = "Standard_DC8_v2"
}

variable "tags" {
Expand Down
4 changes: 3 additions & 1 deletion packages/@n8n/benchmark/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@
"start": "./bin/n8n-benchmark",
"test": "echo \"Error: no test specified\" && exit 1",
"typecheck": "tsc --noEmit",
"run-in-cloud": "zx scripts/runInCloud.mjs",
"benchmark": "zx scripts/run.mjs",
"benchmark-in-cloud": "pnpm benchmark --env cloud",
"benchmark-locally": "pnpm benchmark --env local",
"destroy-cloud-env": "zx scripts/destroyCloudEnv.mjs",
"watch": "concurrently \"tsc -w -p tsconfig.build.json\" \"tsc-alias -w -p tsconfig.build.json\""
},
Expand Down
45 changes: 45 additions & 0 deletions packages/@n8n/benchmark/scripts/clients/dockerComposeClient.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import { which } from 'zx';

export class DockerComposeClient {
/**
*
* @param {{ $: Shell; verbose?: boolean }} opts
*/
constructor({ $ }) {
this.$$ = $;
}

async $(...args) {
await this.resolveExecutableIfNeeded();

if (this.isCompose) {
return await this.$$`docker-compose ${args}`;
} else {
return await this.$$`docker compose ${args}`;
}
}

async resolveExecutableIfNeeded() {
if (this.isResolved) {
return;
}

// The VM deployment doesn't have `docker compose` available,
// so try to resolve the `docker-compose` first
const compose = await which('docker-compose', { nothrow: true });
if (compose) {
this.isResolved = true;
this.isCompose = true;
return;
}

const docker = await which('docker', { nothrow: true });
if (docker) {
this.isResolved = true;
this.isCompose = false;
return;
}

throw new Error('Could not resolve docker-compose or docker');
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@ const paths = {
};

export class TerraformClient {
constructor({ privateKeyPath, isVerbose = false }) {
this.privateKeyPath = privateKeyPath;
constructor({ isVerbose = false }) {
this.isVerbose = isVerbose;
this.$$ = $({
cwd: paths.infraCodeDir,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
services:
postgres:
image: postgres:16
restart: always
environment:
- POSTGRES_DB=n8n
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
n8n:
image: ghcr.io/n8n-io/n8n:${N8N_VERSION:-latest}
environment:
- N8N_DIAGNOSTICS_ENABLED=false
- N8N_USER_FOLDER=/n8n
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PASSWORD=password
ports:
- 5678:5678
volumes:
- ${RUN_DIR}:/n8n
depends_on:
- postgres
benchmark:
image: ghcr.io/n8n-io/n8n-benchmark:${N8N_BENCHMARK_VERSION:-latest}
depends_on:
- n8n
environment:
- N8N_BASE_URL=http://n8n:5678
- K6_API_TOKEN=${K6_API_TOKEN}
73 changes: 73 additions & 0 deletions packages/@n8n/benchmark/scripts/n8nSetups/queue/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
services:
redis:
image: redis:6-alpine
ports:
- 6379:6379
postgres:
image: postgres:16
restart: always
environment:
- POSTGRES_DB=n8n
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
n8n_worker1:
image: ghcr.io/n8n-io/n8n:${N8N_VERSION:-latest}
environment:
- N8N_DIAGNOSTICS_ENABLED=false
- N8N_USER_FOLDER=/n8n/worker1
- N8N_ENCRYPTION_KEY=very-secret-encryption-key
- EXECUTIONS_MODE=queue
- QUEUE_BULL_REDIS_HOST=redis
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PASSWORD=password
command: worker
volumes:
- ${RUN_DIR}:/n8n
depends_on:
- postgres
- redis
n8n_worker2:
image: ghcr.io/n8n-io/n8n:${N8N_VERSION:-latest}
environment:
- N8N_DIAGNOSTICS_ENABLED=false
- N8N_USER_FOLDER=/n8n/worker2
- N8N_ENCRYPTION_KEY=very-secret-encryption-key
- EXECUTIONS_MODE=queue
- QUEUE_BULL_REDIS_HOST=redis
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PASSWORD=password
command: worker
volumes:
- ${RUN_DIR}:/n8n
depends_on:
- postgres
- redis
n8n:
image: ghcr.io/n8n-io/n8n:${N8N_VERSION:-latest}
environment:
- N8N_DIAGNOSTICS_ENABLED=false
- N8N_USER_FOLDER=/n8n/main
- N8N_ENCRYPTION_KEY=very-secret-encryption-key
- EXECUTIONS_MODE=queue
- QUEUE_BULL_REDIS_HOST=redis
- DB_TYPE=postgresdb
- DB_POSTGRESDB_HOST=postgres
- DB_POSTGRESDB_PASSWORD=password
ports:
- 5678:5678
volumes:
- ${RUN_DIR}:/n8n
depends_on:
- postgres
- redis
- n8n_worker1
- n8n_worker2
benchmark:
image: ghcr.io/n8n-io/n8n-benchmark:${N8N_BENCHMARK_VERSION:-latest}
depends_on:
- n8n
environment:
- N8N_BASE_URL=http://n8n:5678
- K6_API_TOKEN=${K6_API_TOKEN}
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ services:
ports:
- 5678:5678
volumes:
- /n8n:/n8n
- ${RUN_DIR}:/n8n
benchmark:
image: ghcr.io/n8n-io/n8n-benchmark:${N8N_BENCHMARK_VERSION:-latest}
depends_on:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ services:
ports:
- 5678:5678
volumes:
- /n8n:/n8n
- ${RUN_DIR}:/n8n
benchmark:
image: ghcr.io/n8n-io/n8n-benchmark:${N8N_BENCHMARK_VERSION:-latest}
depends_on:
Expand Down
Loading

0 comments on commit 0c53737

Please sign in to comment.