Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(core): Fix missing execution ID in webhook-based workflow producing binary data #7244

Merged
merged 2 commits into from
Sep 25, 2023

Conversation

ivov
Copy link
Contributor

@ivov ivov commented Sep 25, 2023

Story: https://linear.app/n8n/issue/PAY-839

This is a longstanding bug, fixed now so that the S3 backend for binary data can use execution IDs as part of the filename.

To reproduce:

  1. Set up a workflow with a POST Webhook node that accepts binary data.
  2. Activate the workflow and call it sending a binary file, e.g. curl -X POST -F "file=@/path/to/binary/file/test.jpg" http://localhost:5678/webhook/uuid
  3. Check ~/.n8n/binaryData. The binary data and metadata files will be missing the execution ID, e.g. 11869055-83c4-4493-876a-9092c4708b9b instead of 39011869055-83c4-4493-876a-9092c4708b9b.

@github-actions
Copy link
Contributor

Great PR! Please pay attention to the following items before merging:

Files matching packages/**:

  • If fixing bug, added test to cover scenario.
  • If addressing forum or Github issue, added link to description.

Files matching packages/**/*.ts:

  • Added unit tests to cover new or updated functionality.

Make sure to check off this list before asking for review.

@n8n-assistant n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team labels Sep 25, 2023
@codecov
Copy link

codecov bot commented Sep 25, 2023

Codecov Report

Patch coverage: 62.06% and no project coverage change.

Comparison is base (6d6e248) 32.83% compared to head (c83b3be) 32.84%.
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #7244   +/-   ##
=======================================
  Coverage   32.83%   32.84%           
=======================================
  Files        3352     3353    +1     
  Lines      199978   200007   +29     
  Branches    21924    21925    +1     
=======================================
+ Hits        65664    65683   +19     
- Misses     133202   133211    +9     
- Partials     1112     1113    +1     
Files Changed Coverage Δ
packages/core/src/BinaryData/BinaryData.service.ts 40.67% <0.00%> (-1.43%) ⬇️
packages/core/src/BinaryData/FileSystem.manager.ts 51.31% <20.00%> (-2.21%) ⬇️
packages/cli/src/WorkflowExecuteAdditionalData.ts 13.19% <33.33%> (+0.18%) ⬆️
...src/executionLifecycleHooks/restoreBinaryDataId.ts 94.11% <94.11%> (ø)

... and 4 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

const { runData } = run.data.resultData;

const promises = Object.keys(runData).map(async (nodeName) => {
const binaryDataId = runData[nodeName]?.[0]?.data?.main?.[0]?.[0].binary?.data.id;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😱

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:)

@cypress
Copy link

cypress bot commented Sep 25, 2023

1 flaky test on run #2261 ↗︎

0 240 3 0 Flakiness 1

Details:

🌳 pay-839 🖥️ browsers:node18.12.0-chrome107 🤖 ivov 🗃️ e2e/*
Project: n8n Commit: c83b3bee91
Status: Passed Duration: 07:19 💡
Started: Sep 25, 2023 10:21 AM Ended: Sep 25, 2023 10:29 AM
Flakiness  cypress/e2e/28-debug.cy.ts • 1 flaky test

View Output Video

Test Artifacts
Debug > should be able to debug executions Output Screenshots Video

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.

@github-actions
Copy link
Contributor

✅ All Cypress E2E specs passed

@ivov ivov merged commit 33991e9 into master Sep 25, 2023
20 checks passed
@ivov ivov deleted the pay-839 branch September 25, 2023 10:30
This was referenced Sep 27, 2023
netroy added a commit that referenced this pull request Sep 28, 2023
# [1.9.0](https://github.com/n8n-io/n8n/compare/n8n@1.8.0...n8n@1.9.0)
(2023-09-28)


### Bug Fixes

* **Airtable Node:** Attachments field type fix
([#7227](#7227))
([2af967c](2af967c))
* **core:** Change WorkflowHistory nodes/connections columns to be json
([#7282](#7282))
([a80abad](a80abad))
* **core:** Fix binary data manager check on pruning
([#7251](#7251))
([484035e](484035e))
* **core:** Fix missing execution ID in webhook-based workflow producing
binary data ([#7244](#7244))
([33991e9](33991e9))
* **core:** Handle filename* with quotes in Content-Disposition header
([#7229](#7229))
([67b985f](67b985f))
* **core:** Make DNS resolution order configurable
([#7272](#7272))
([5b3121c](5b3121c))
* **core:** Make senderId required for all command messages
([#7252](#7252))
([4b01428](4b01428))
* **core:** Prevent executions from displaying Running status
incorrectly ([#7261](#7261))
([861cac5](861cac5))
* **core:** Use consistent timezone-aware timestamps in postgres
([#6948](#6948))
([0132514](0132514)),
closes [#2178](#2178)
[#2810](#2810)
[#3855](#3855)
[#2813](#2813)
* **editor:** Add debug feature docs link
([#7240](#7240))
([4614e1e](4614e1e))
* **editor:** Fix SQL editor issue
([#7236](#7236))
([647fc6c](647fc6c))
* **editor:** Ensure new Set node is on top of search list
([#7215](#7215))
([2491ccf](2491ccf))
* **editor:** Forbid password reset when cloud account is limited in the
number of users [7188](#7188)
([303bc8e](303bc8e))
* **HTTP Request Node:** Add suggestion how to fix '429 - too many
requests' errors ([#7293](#7293))
([0bc33b1](0bc33b1))
* **Item Lists Node:** Concatenate operation pairedItems fix
([#7286](#7286))
([cde23a1](cde23a1))
* **Respond to Webhook Node:** JSON output from expression fix
([#7294](#7294))
([8bc369d](8bc369d))


### Features

* Add onboarding flow
([#7212](#7212))
([01e9340](01e9340))
* **core:** Add secrets provider reload and refactor
([#7277](#7277))
([53a7502](53a7502))
* **core:** Add Tournament as the new default expression evaluator
([#6964](#6964))
([bf74f09](bf74f09))
* **core:** Initial workflow history API
([#7234](#7234))
([0083a9e](0083a9e))
* **core:** Introduce object store service
([#7225](#7225))
([fa84545](fa84545))
* **editor:** Add user cloud ID to telemetry
[#7232](#7232)
([60c152d](60c152d))
* **editor:** Rework banners framework and add email confirmation banner
([#7205](#7205))
([b0e98b5](b0e98b5))
* **MISP Node:** Update credential to support HTTP Request node
([#7268](#7268))
([e4c302c](e4c302c))


### Performance Improvements

* **core:** Skip unneeded calls on every pruning cycle
([#7260](#7260))
([db01164](db01164))

Co-authored-by: netroy <netroy@users.noreply.github.com>
@janober
Copy link
Member

janober commented Sep 28, 2023

Got released with n8n@1.9.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team Released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants