Migration failure moving from 0.124.0 to 0.183.0

[SpeedHighway]

Describe the bug
Attempting to upgrade my n8n release from 0.124.0 to the newest version appears to attempt SQL migration, but then immediately fails trying to reach a folder that does not, and never would have, existed.

The following text displays:

Initializing n8n process
Migrations in progress, please do NOT stop the process.
query is slow: CREATE TABLE "temporary_execution_entity" ("id" integer PRIMARY KEY AUTOINCREMENT NOT NULL, "data" text NOT NULL, "finished" boolean NOT NULL, "mode" varchar NOT NULL, "retryOf" varchar, "retrySuccessId" varchar, "startedAt" datetime NOT NULL, "stoppedAt" datetime, "workflowData" text NOT NULL, "workflowId" varchar, "waitTill" DATETIME)
execution time: 25884
 »   Error: There was an error: EPERM: operation not permitted, scandir 'C:\Documents and Settings'

To Reproduce
Steps to reproduce the behavior:

1. Have an existing 0.124.0 version of n8n installed, and in-use.
2. Shut down n8n
3. Run powershell in admin mode.
4. Upgrade to newest version of npm for the version of node installed. (npm install -g npm@8.12.2)
5. Upgrade n8n (npm update -g n8n)
6. Attempt to run n8n (n8n from the command line)

Expected behavior
n8n successfully starts, and I can navigate my existing workflows.

Environment (please complete the following information):

• OS: Windows 10 Pro 21H1
• n8n Version: 0.183.0
• Node.js Version: 16.3.0
• Database system: SQLite
• Operation mode: own (I don't know what this means, as I've not needed to know)

Additional context
Running a 2nd time results in the migration removing the temp table, and still failing with the same error, but a 3rd time shows the same results as in the description:

Initializing n8n process
Migrations in progress, please do NOT stop the process.
query is slow: DROP TABLE IF EXISTS "temporary_execution_entity"
execution time: 3785
 »   Error: There was an error: EPERM: operation not permitted, scandir 'C:\Documents and Settings'

I also tried running this from my user folder (though it didn't matter where I ran it from previously) and it fails to access the Application Data folder within.

[netroy]

@SpeedHighway Do you by any chance have the older version of n8n still running?
If yes, please try to shutdown or kill that process first.
The operation not permitted error sounds like the database.sqlite file might be locked by a running process.

[SpeedHighway]

@netroy No, definitely not. And the operation not permitted claims to be having errors with trying to read things that have nothing to do with the project whatsoever. The powershell window that was running n8n is the same one I used to upgrade, so it was shut down first. (Also, I've installed globally, so it would have fought me if I'd tried to update while it was in-use.)

I don't know why it would be trying these arbitrary locations for reading anyways, as the .n8n pieces are all in C:\Users\UserName\.n8n. "Documents and Settings" and "Application Data" are both folder names in Windows XP, if I recall. (Those folders are now Documents - almost never in C:\ - and AppData)

Additionally, I found the error.log file, just now, and it has a stack trace:

2022-06-22T17:37:22.588Z Error: There was an error: EPERM: operation not permitted, scandir 'C:/Documents and Settings'
2022-06-22T17:37:22.588Z     at Object.error (T:/Users/UserName/AppData/Roaming/nvm/v16.3.0/node_modules/n8n/node_modules/@oclif/errors/lib/index.js:26:15)
2022-06-22T17:37:22.588Z     at Start.error (T:/Users/UserName/AppData/Roaming/nvm/v16.3.0/node_modules/n8n/node_modules/@oclif/command/lib/command.js:60:23)
2022-06-22T17:37:22.588Z     at T:/Users/UserName/AppData/Roaming/nvm/v16.3.0/node_modules/n8n/dist/commands/start.js:235:22
2022-06-22T17:37:22.588Z     at async Start.run (T:/Users/UserName/AppData/Roaming/nvm/v16.3.0/node_modules/n8n/dist/commands/start.js:69:9)
2022-06-22T17:37:22.588Z     at async Start._run (T:/Users/UserName/AppData/Roaming/nvm/v16.3.0/node_modules/n8n/node_modules/@oclif/command/lib/command.js:43:20)
2022-06-22T17:37:22.588Z     at async Config.runCommand (T:/Users/UserName/AppData/Roaming/nvm/v16.3.0/node_modules/n8n/node_modules/@oclif/config/lib/config.js:173:24)
2022-06-22T17:37:22.588Z     at async Main.run (T:/Users/UserName/AppData/Roaming/nvm/v16.3.0/node_modules/n8n/node_modules/@oclif/command/lib/main.js:27:9)
2022-06-22T17:37:22.588Z     at async Main._run (T:/Users/UserName/AppData/Roaming/nvm/v16.3.0/node_modules/n8n/node_modules/@oclif/command/lib/command.js:43:20)

[netroy]

@SpeedHighway can you please run this on your console node -e 'console.log(process.platform, process.env.USERPROFILE, process.env.N8N_USER_FOLDER)'?

Hopefully the output will be win32 C:\Users\UserName undefined.

[SpeedHighway]

Yes, the output was exactly that (with my actual username, of course) :)

[netroy]

in that case I'm not sure what could be causing the code to read from those directories.

but to unblock you, I'd suggest you try exporting the data, and then re-importing on a fresh n8n instance.

Step 1: export the data

npx n8n@0.124.0 export:credentials --backup --output credentials
npx n8n@0.124.0 export:workflow --backup --output workflows

Step 2: rename C:\Users\UserName\.n8n to C:\Users\UserName\.n8n.backup (so that the old folder can still be used for further investigation)

Step 3: import the data

npx n8n@0.183.0 import:credentials --separate --input=credentials
npx n8n@0.183.0 import:workflow --separate --input=workflows

rm -rf credentials/ workflows/

[SpeedHighway]

@netroy Ty for the suggestion. I've attempted the commands and while the exports appear to have worked, the credentials refuse to import.

PS C:\Users\UserName\n8nTemp> npx n8n@0.124.0 export:credentials --backup --output credentials
Need to install the following packages:
  n8n@0.124.0
Ok to proceed? (y) y
npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated formidable@1.2.6: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
Migrations in progress, please do NOT stop the process.
Migrations finished.
Successfully exported 3 credentials.
PS C:\Users\UserName\n8nTemp> npx n8n@0.124.0 export:workflow --backup --output workflows
Successfully exported 1 workflows.

PS C:\Users\UserName\n8nTemp> npx n8n@0.183.0 import:credentials --separate --input=credentials
Need to install the following packages:
  n8n@0.183.0
Ok to proceed? (y) y
npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated formidable@1.2.6: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated core-js@2.6.12: core-js@<3.4 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
Migrations in progress, please do NOT stop the process.
UserSettings were generated and saved to: C:\Users\UserName\.n8n\config
Successfully imported 0 credentials.
PS C:\Users\UserName\n8nTemp> npx n8n@0.183.0 import:workflow --separate --input=workflows
Successfully imported 1 workflow.
PS C:\Users\UserName\n8nTemp> npx n8n@0.183.0 import:credentials --separate --input=credentials
Successfully imported 0 credentials.

(I did check the json files in the credentials folder and there's 3 and they seems pretty straightfoward)

[netroy]

looks like the credentials importing code wasn't adapting to windows paths, and hence not finding any of the JSON files.

I've opened a PR to fix this. Hopefully this will help unblock you.

[netroy]

@SpeedHighway Can you please try again with n8n@0.188.0 ?

[SpeedHighway]

@netroy I suspected that this solution might have resolved both the import and the migration, so I tested just running n8n first. Migration succeeded this time due to the ability to properly import the credentials.

[SpeedHighway]

I also just tried the import command to verify that it works. Aside from the fact that npx doesn't want to use the (npm) globally installed n8n, it did successfully import (into the new n8n that it installed).