-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(core): Add support for implicit schema in postgres migrations #5233
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
41 changes: 10 additions & 31 deletions
41
packages/cli/src/databases/migrations/postgresdb/1587669153312-InitialMigration.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,62 +1,41 @@ | ||
| import { MigrationInterface, QueryRunner } from 'typeorm'; | ||
|
|
||
| import config from '@/config'; | ||
| import { getTablePrefix } from '@db/utils/migrationHelpers'; | ||
|
|
||
| export class InitialMigration1587669153312 implements MigrationInterface { | ||
| name = 'InitialMigration1587669153312'; | ||
|
|
||
| async up(queryRunner: QueryRunner): Promise<void> { | ||
| let tablePrefix = config.getEnv('database.tablePrefix'); | ||
| const tablePrefixIndex = tablePrefix; | ||
| const schema = config.getEnv('database.postgresdb.schema'); | ||
| if (schema) { | ||
| tablePrefix = schema + '.' + tablePrefix; | ||
| } | ||
|
|
||
| await queryRunner.query(`SET search_path TO ${schema};`); | ||
| const tablePrefix = getTablePrefix(); | ||
|
|
||
| await queryRunner.query( | ||
| `CREATE TABLE IF NOT EXISTS ${tablePrefix}credentials_entity ("id" SERIAL NOT NULL, "name" character varying(128) NOT NULL, "data" text NOT NULL, "type" character varying(32) NOT NULL, "nodesAccess" json NOT NULL, "createdAt" TIMESTAMP NOT NULL, "updatedAt" TIMESTAMP NOT NULL, CONSTRAINT PK_${tablePrefixIndex}814c3d3c36e8a27fa8edb761b0e PRIMARY KEY ("id"))`, | ||
| `CREATE TABLE IF NOT EXISTS ${tablePrefix}credentials_entity ("id" SERIAL NOT NULL, "name" character varying(128) NOT NULL, "data" text NOT NULL, "type" character varying(32) NOT NULL, "nodesAccess" json NOT NULL, "createdAt" TIMESTAMP NOT NULL, "updatedAt" TIMESTAMP NOT NULL, CONSTRAINT PK_${tablePrefix}814c3d3c36e8a27fa8edb761b0e PRIMARY KEY ("id"))`, | ||
| undefined, | ||
| ); | ||
| await queryRunner.query( | ||
| `CREATE INDEX IF NOT EXISTS IDX_${tablePrefixIndex}07fde106c0b471d8cc80a64fc8 ON ${tablePrefix}credentials_entity (type) `, | ||
| `CREATE INDEX IF NOT EXISTS IDX_${tablePrefix}07fde106c0b471d8cc80a64fc8 ON ${tablePrefix}credentials_entity (type) `, | ||
| undefined, | ||
| ); | ||
| await queryRunner.query( | ||
| `CREATE TABLE IF NOT EXISTS ${tablePrefix}execution_entity ("id" SERIAL NOT NULL, "data" text NOT NULL, "finished" boolean NOT NULL, "mode" character varying NOT NULL, "retryOf" character varying, "retrySuccessId" character varying, "startedAt" TIMESTAMP NOT NULL, "stoppedAt" TIMESTAMP NOT NULL, "workflowData" json NOT NULL, "workflowId" character varying, CONSTRAINT PK_${tablePrefixIndex}e3e63bbf986767844bbe1166d4e PRIMARY KEY ("id"))`, | ||
| `CREATE TABLE IF NOT EXISTS ${tablePrefix}execution_entity ("id" SERIAL NOT NULL, "data" text NOT NULL, "finished" boolean NOT NULL, "mode" character varying NOT NULL, "retryOf" character varying, "retrySuccessId" character varying, "startedAt" TIMESTAMP NOT NULL, "stoppedAt" TIMESTAMP NOT NULL, "workflowData" json NOT NULL, "workflowId" character varying, CONSTRAINT PK_${tablePrefix}e3e63bbf986767844bbe1166d4e PRIMARY KEY ("id"))`, | ||
| undefined, | ||
| ); | ||
| await queryRunner.query( | ||
| `CREATE INDEX IF NOT EXISTS IDX_${tablePrefixIndex}c4d999a5e90784e8caccf5589d ON ${tablePrefix}execution_entity ("workflowId") `, | ||
| `CREATE INDEX IF NOT EXISTS IDX_${tablePrefix}c4d999a5e90784e8caccf5589d ON ${tablePrefix}execution_entity ("workflowId") `, | ||
| undefined, | ||
| ); | ||
| await queryRunner.query( | ||
| `CREATE TABLE IF NOT EXISTS ${tablePrefix}workflow_entity ("id" SERIAL NOT NULL, "name" character varying(128) NOT NULL, "active" boolean NOT NULL, "nodes" json NOT NULL, "connections" json NOT NULL, "createdAt" TIMESTAMP NOT NULL, "updatedAt" TIMESTAMP NOT NULL, "settings" json, "staticData" json, CONSTRAINT PK_${tablePrefixIndex}eded7d72664448da7745d551207 PRIMARY KEY ("id"))`, | ||
| `CREATE TABLE IF NOT EXISTS ${tablePrefix}workflow_entity ("id" SERIAL NOT NULL, "name" character varying(128) NOT NULL, "active" boolean NOT NULL, "nodes" json NOT NULL, "connections" json NOT NULL, "createdAt" TIMESTAMP NOT NULL, "updatedAt" TIMESTAMP NOT NULL, "settings" json, "staticData" json, CONSTRAINT PK_${tablePrefix}eded7d72664448da7745d551207 PRIMARY KEY ("id"))`, | ||
| undefined, | ||
| ); | ||
| } | ||
|
|
||
| async down(queryRunner: QueryRunner): Promise<void> { | ||
| let tablePrefix = config.getEnv('database.tablePrefix'); | ||
| const tablePrefixIndex = tablePrefix; | ||
| const schema = config.getEnv('database.postgresdb.schema'); | ||
| if (schema) { | ||
| tablePrefix = schema + '.' + tablePrefix; | ||
| } | ||
|
|
||
| await queryRunner.query(`SET search_path TO ${schema};`); | ||
| let tablePrefix = getTablePrefix(); | ||
|
|
||
| await queryRunner.query(`DROP TABLE ${tablePrefix}workflow_entity`, undefined); | ||
| await queryRunner.query( | ||
| `DROP INDEX IDX_${tablePrefixIndex}c4d999a5e90784e8caccf5589d`, | ||
| undefined, | ||
| ); | ||
| await queryRunner.query(`DROP INDEX IDX_${tablePrefix}c4d999a5e90784e8caccf5589d`, undefined); | ||
| await queryRunner.query(`DROP TABLE ${tablePrefix}execution_entity`, undefined); | ||
| await queryRunner.query( | ||
| `DROP INDEX IDX_${tablePrefixIndex}07fde106c0b471d8cc80a64fc8`, | ||
| undefined, | ||
| ); | ||
| await queryRunner.query(`DROP INDEX IDX_${tablePrefix}07fde106c0b471d8cc80a64fc8`, undefined); | ||
| await queryRunner.query(`DROP TABLE ${tablePrefix}credentials_entity`, undefined); | ||
| } | ||
| } |
22 changes: 4 additions & 18 deletions
22
packages/cli/src/databases/migrations/postgresdb/1589476000887-WebhookModel.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,33 +1,19 @@ | ||
| import { MigrationInterface, QueryRunner } from 'typeorm'; | ||
|
|
||
| import config from '@/config'; | ||
| import { getTablePrefix } from '@db/utils/migrationHelpers'; | ||
|
|
||
| export class WebhookModel1589476000887 implements MigrationInterface { | ||
| name = 'WebhookModel1589476000887'; | ||
|
|
||
| async up(queryRunner: QueryRunner): Promise<void> { | ||
| let tablePrefix = config.getEnv('database.tablePrefix'); | ||
| const tablePrefixIndex = tablePrefix; | ||
| const schema = config.getEnv('database.postgresdb.schema'); | ||
| if (schema) { | ||
| tablePrefix = schema + '.' + tablePrefix; | ||
| } | ||
|
|
||
| await queryRunner.query(`SET search_path TO ${schema};`); | ||
|
|
||
| const tablePrefix = getTablePrefix(); | ||
| await queryRunner.query( | ||
| `CREATE TABLE IF NOT EXISTS ${tablePrefix}webhook_entity ("workflowId" integer NOT NULL, "webhookPath" character varying NOT NULL, "method" character varying NOT NULL, "node" character varying NOT NULL, CONSTRAINT "PK_${tablePrefixIndex}b21ace2e13596ccd87dc9bf4ea6" PRIMARY KEY ("webhookPath", "method"))`, | ||
| `CREATE TABLE IF NOT EXISTS ${tablePrefix}webhook_entity ("workflowId" integer NOT NULL, "webhookPath" character varying NOT NULL, "method" character varying NOT NULL, "node" character varying NOT NULL, CONSTRAINT "PK_${tablePrefix}b21ace2e13596ccd87dc9bf4ea6" PRIMARY KEY ("webhookPath", "method"))`, | ||
| undefined, | ||
| ); | ||
| } | ||
|
|
||
| async down(queryRunner: QueryRunner): Promise<void> { | ||
| let tablePrefix = config.getEnv('database.tablePrefix'); | ||
| const schema = config.getEnv('database.postgresdb.schema'); | ||
| if (schema) { | ||
| tablePrefix = schema + '.' + tablePrefix; | ||
| } | ||
| await queryRunner.query(`SET search_path TO ${schema};`); | ||
| const tablePrefix = getTablePrefix(); | ||
| await queryRunner.query(`DROP TABLE ${tablePrefix}webhook_entity`, undefined); | ||
| } | ||
| } |
26 changes: 5 additions & 21 deletions
26
packages/cli/src/databases/migrations/postgresdb/1594828256133-CreateIndexStoppedAt.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,34 +1,18 @@ | ||
| import { MigrationInterface, QueryRunner } from 'typeorm'; | ||
|
|
||
| import config from '@/config'; | ||
| import { getTablePrefix } from '@/databases/utils/migrationHelpers'; | ||
|
|
||
| export class CreateIndexStoppedAt1594828256133 implements MigrationInterface { | ||
| name = 'CreateIndexStoppedAt1594828256133'; | ||
|
|
||
| async up(queryRunner: QueryRunner): Promise<void> { | ||
| let tablePrefix = config.getEnv('database.tablePrefix'); | ||
| const tablePrefixPure = tablePrefix; | ||
| const schema = config.getEnv('database.postgresdb.schema'); | ||
| if (schema) { | ||
| tablePrefix = schema + '.' + tablePrefix; | ||
| } | ||
|
|
||
| await queryRunner.query(`SET search_path TO ${schema};`); | ||
|
|
||
| const tablePrefix = getTablePrefix(); | ||
| await queryRunner.query( | ||
| `CREATE INDEX IF NOT EXISTS IDX_${tablePrefixPure}33228da131bb1112247cf52a42 ON ${tablePrefix}execution_entity ("stoppedAt") `, | ||
| `CREATE INDEX IF NOT EXISTS IDX_${tablePrefix}33228da131bb1112247cf52a42 ON ${tablePrefix}execution_entity ("stoppedAt") `, | ||
| ); | ||
| } | ||
|
|
||
| async down(queryRunner: QueryRunner): Promise<void> { | ||
| let tablePrefix = config.getEnv('database.tablePrefix'); | ||
|
|
||
| const tablePrefixPure = tablePrefix; | ||
| const schema = config.getEnv('database.postgresdb.schema'); | ||
| if (schema) { | ||
| tablePrefix = schema + '.' + tablePrefix; | ||
| } | ||
| await queryRunner.query(`SET search_path TO ${schema};`); | ||
| await queryRunner.query(`DROP INDEX IDX_${tablePrefixPure}33228da131bb1112247cf52a42`); | ||
| const tablePrefix = getTablePrefix(); | ||
| await queryRunner.query(`DROP INDEX IDX_${tablePrefix}33228da131bb1112247cf52a42`); | ||
| } | ||
| } |
12 changes: 2 additions & 10 deletions
12
packages/cli/src/databases/migrations/postgresdb/1607431743768-MakeStoppedAtNullable.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
25 changes: 5 additions & 20 deletions
25
packages/cli/src/databases/migrations/postgresdb/1611144599516-AddWebhookId.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea here was to allow db init to happen "async" while other things were happening; not a huge gain I believe, but are there any reasons to remove this behavior?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it wasn't working as expected when i was testing it. it was somehow only waiting until the db connection was ready, but not until all the migrations were done executing.
but, I can revert it, and test it again.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it really provides any major gains, but it's just something to watch out for if we see people saying n8n is taking a bit longer to start. I don't believe we'll be seeing those issues frequently since it's a very simple thing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah. and since nodes are lazy loaded now, there shouldn't be much of a difference in startup times.