mongodb and cannot be bundled with worker presets

[manniL]

Environment

• Operating System: Darwin
• Node Version: v18.13.0
• Nitro Version: edge-2.2.2-27941588.be1ac07
• Package Manager: pnpm@7.27.0

Reproduction

https://github.com/manniL/rm-nitro-externals-mongodb

1. Clone Repo
2. pnpm i
3. pnpm build
4. See error

Describe the bug

When building, the following error occurs:

Error: Cannot resolve "@mongodb-js/zstd" from "/path/to/project/node_modules/.pnpm/mongodb@5.0.1/node_modules/mongodb/lib/deps.js" and externals are not allowed!

Additional context

Similar to #947

Logs

No response

[pi0]

Thanks for making the reproduction.

I have a fix for the particular first issue we encounter. However, the mongodb library is not the most popular to be bundler friendly to be fully packed by rollup. It has many implicit often optional dependencies in tree such as aws-sdk, aws4 and native ones and snappy. And mongo is not alone... Even vue needed weeks in Nuxt side to be bundle friendly.

We might finally require a preset of configurations or special packed distro of MongoDB.

Are you trying to deploy it on a worker target?

[manniL]

Thanks for making the reproduction.
It has many implicit often optional dependencies in tree such as aws-sdk, aws4 and native ones and snappy.

Yes indeed, but I think they are optional, at least according to deps.ts here.

We might finally require a preset of configurations or special packed distro of MongoDB.

Sounds reasonable indeed.

Are you trying to deploy it on a worker target?

Yup, CF workers 🙈

[dargmuesli]

Just out of curiosity:

Even vue needed weeks in Nuxt side to be bundle friendly.

What was needed to make Vue bundle friendly?

[pi0]

Vue 3: https://github.com/nuxt/nuxt/blob/c45b84203731bf469cb22979570e03a4f7202a80/packages/nuxt/src/core/nitro.ts#L127

Vue 2:
https://github.com/nuxt/bridge/blob/ad2d6d2167792b2ca41a97acfb20b516617074de/packages/bridge/src/nitro.ts#L111

Also couple of indirect mocks enabled by unenv/nodeless to reduce size for workers:

https://github.com/unjs/unenv/blob/main/src/presets/nodeless.ts

[manniL]

After further research I wonder if mongodb on CF workers would even work w/o nitro (and without e.g. using the MongoDB Atlas Data API) 🙈

Because of TCP/IP you might need Cloudflare tunnel like for mysql.

Further read: https://developers.cloudflare.com/workers/learning/integrations/databases/

Maybe also worth checking e.g. https://github.com/denodrivers/mongo

[TylorMayfield]

I'm having the same issue on "vercel-edge"

[itpropro]

@manniL I don't think mongodb will work, as of the missing raw support for TCP/UDP that you mentioned. Even the official Mongo documentation only references the Realm SDK for Workers.
For node.js target, the mongodb package works and is bundled fine in my tests.
Maybe for now we could add a section about "libraries that don't work with certain targets" or something similar to the docs?

[pi0]

For node.js target, the mongodb package works and is bundled fine in my tests.

It works with externals tracer indeed (still lots of duplicate deps as you know 🙈). It would probably worth to make it better anyway at least for Node.js target. (if possible)

Maybe for now we could add a section about "libraries that don't work with certain targets" or something similar to the docs?

Sound like a nice idea for docs! Maybe in Deployment>Overview>Workers and mention workers limits + possible current worker targets? (deno, cloudflare, vercel edge, netlify edge -- they all share similar limits and specs)

[flozero]

I hit the same problem here. Does it mean we can't use mongo db with their server less approach ?

What is the recommandation here please :s

[manniL]

I hit the same problem here. Does it mean we can't use mongo db with their server less approach ?

What is the recommandation here please :s

Serverless is fine ✅
Non-node environment like CF workers is not ❌
* For that, you'd use realm.js / realm-web instead
* My "real-life" example

[flozero]

I changed for supabase personnally

[itpropro]

I added your example to #953 @manniL

[eladcandroid]

@TylorMayfield Did you manage to solve it with vercel-edge preset?

[yabuking84]

Having the same issue, setting NITRO_PRESET to remove edge workers (ex. setting it to just vercel instead of vercel-edge) creates the bundle but I would really like to use the edge workers. Any workarounds? I see using Realm SDK was suggested.

[pi0]

Some engines such as pg recently support edge-workers using native connect() and seems mongo is not yet so even if we fix any possible bundling issues it won't be working.

For mongoDB and incompatible databases, you often need an HTTP gateway. Check this example with cloudflare+atlas/realm: https://www.mongodb.com/developer/products/atlas/cloudflare-worker-rest-api/