Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: ensure lmdb correct binaries are available for lambdas #253

Merged
merged 4 commits into from
Feb 3, 2022

Conversation

ascorbic
Copy link
Contributor

@ascorbic ascorbic commented Feb 1, 2022

When gatsby build is run, it generates a "query engine", a process that includes bundling the code but also relocating native binaries into an "assets" directory inside the build cache. The assets are identified using a webpack loader that has various heuristics to identify native binaries, because there is no standard for this in Node. The datastore used by Gatsby is lmdb-js, which uses node-gyp-build for loading the correct native binary at runtime. The loader handles moduels that use node-gyp-build by calling this path() method at build time, relocating the defined binaries, and replacing the import with this path. The problem is that this then identifies the binary for the build environment, not the runtime. In our default build environment these do not match, because the Node ABI versions differ, meaning that the wrong binary is included, and the wrong path written to the bundle.

This PR handles these as a special case, and manually relocates the correct lmdb binary for linux-x86/abi83, which matches the default lambda runtime. It then patches the bundle to ensure the loader path matches. It handles two different cases for this: the existing lmdb-store package, and the lmdb package that Gatsby will be migrating to soon.

Because this kind of hard-coded workaround is fragile, this PR also ensures that deploy previews run with a node version that triggers this behaviour, and uses Gatsby's next version so problems are identified early.

To test, check the build logs for the deploy preview to see that Node 16 is being used (i.e. not Node 14 as is used in Lambdas). Then check an SSR page in the deploy preview to ensure it renders.

@netlify
Copy link

netlify bot commented Feb 1, 2022

✔️ Deploy Preview for netlify-plugin-gatsby-demo canceled.

🔨 Explore the source changes: 6a74dc6

🔍 Inspect the deploy log: https://app.netlify.com/sites/netlify-plugin-gatsby-demo/deploys/61fbd78c3429d00007dda364

@ascorbic ascorbic force-pushed the mk/copy-binaries branch 6 times, most recently from 6e74003 to 5789a41 Compare February 1, 2022 17:38
@ascorbic ascorbic marked this pull request as ready for review February 2, 2022 09:22
@ascorbic ascorbic requested a review from tiffafoo February 2, 2022 09:27
tiffafoo
tiffafoo previously approved these changes Feb 2, 2022
@kodiakhq kodiakhq bot merged commit c6ea2e2 into main Feb 3, 2022
@kodiakhq kodiakhq bot deleted the mk/copy-binaries branch February 3, 2022 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants