Implementing turbo prune command for pnpm workspaces

[Aarush-Goyal]

What version of Turborepo are you using?

1.0.24

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Linux

Describe the Bug

I was running the command turbo prune --scope=web for a pnpm workspace using turborepo

And it returns this error message:

 ERROR  this command is not yet implemented for nodejs-pnpm

Expected Behavior

It should have trimmed the workspace with relation to the scope declared which would make it easier for me to build the container image for that particular app.

To Reproduce

You can create a simple turborepo that uses pnpm as the package manager like the pnpm example https://github.com/vercel/turborepo/tree/main/examples/with-pnpm

and try to run the command:

turbo prune --scope=web

I am running the pnpm version 6.24.4 node version v17.3.0 turbo version 1.0.24

[ormauda]

I face the same problem with pnpm 6.26.1 , node 12.18.2 and turbo 1.0.24.
It would be awesome to use this flag.

In the meantime, I'd love to update the docs stating that this feature is missing for pnpm if that's fine.

[AlonMiz]

can be based on this solution
https://www.npmjs.com/package/pnpm-isolate-workspace

[Aarush-Goyal]

Thanks @AlonMiz and @ivoilic for your contributions.

[iiian]

Can't wait to have this feature.

[Aarush-Goyal]

@jaredpalmer can you please put this issue at a higher priority? It would make working with pnpm and turbo very easy.

[iiian]

DISCLAIMER: This might be terrible advice.

In the meantime while we wait for pnpm-based turbo prune, you can use pnpm locally for blazing fast package management at dev-time while still using turbo prune for docker builds, if that is something you're interested in doing. Steps I'm taking to achieve this:

1. Fork your root package.json file. Maybe something like package.docker-build.json.
2. Update your "packageManager" property in package.json to target your pnpm@x.y.z (eg 6.32.4).
3. Create a pnpm-workspace.yaml as explained in the pnpm documentation.
4. Update your .dockerignore, ymmv. Make sure you add package.json and package.docker-build.json, because the next step 👇 .
5. In your Dockerfile, COPY package.docker-build.json package.json at the proper moment in your context transfer.

Potential Drawbacks:
the cost of synchronizing a top level yarn.lock file, depending on how you're building your project, may make this all more hassle than it is really worth. But the nice thing about workspaces is that the two top-level package.jsons are isolated from dependency changes in subworkspaces.

[Manubi]

Push. Would be nice so see this added soon as more and more people are changing to pnpm.

[nathanhammond]

This is something that is on our list however we have not prioritized doing this work at this time. The existing implementation is narrowly focused on an individual Yarn use case and is not easily generalizable, so it's not a trivial task.

[Manubi]

Is there a way around in the meantime? :)

[nathanhammond]

@Manubi I haven't explicitly tested it, but in general the thing proposed here should work: #534 (comment)

(Use Yarn to generate docker images, and whatever package manager you want for general use.)

[ivoilic]

@Manubi This is what I'm using for now:

RUN mkdir -p out/json && find . -type f -name package.json -not -path '*/node_modules/*' | xargs -i cp --parents {} ./out/json
RUN mkdir out/full
COPY . ./out/full```

[Manubi]

@ivoilic can you maybe post the full docker file? :)
I just recently started with turbo pnpm and docker... would be appreciated!

thanks

[sebastiangug]

can be based on this solution https://www.npmjs.com/package/pnpm-isolate-workspace

this doesn't appear to currently work due to various type errors neither on linux or windows

[sethgw]

I need this so bad 👍

[weyert]

I am using with great succes: https://pnpm.io/cli/deploy

[michael-land]

Hey @weyert, is there any example repo ?

[zigang93]

my solution for nextjs, hope can help someone:
this is just a temp solution, need wait for turbo prune --scope=web --docker support at pnpm
your apps/web Dockerfile

# 1. Generate a sparse/partial monorepo with a pruned lockfile for a target workspace. https://turborepo.org/docs/reference/command-line-reference#turbo-prune---scopetarget

# Refer to this original https://github.com/vercel/turborepo/blob/main/examples/with-docker/apps/web/Dockerfile

FROM node:alpine AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat

# Set working directory
WORKDIR /app

# pnpm cannot support turbo prune yet. so need manual create isolated folder https://github.com/vercel/turborepo/issues/534
# RUN pnpm global add turbo
# COPY . .
# RUN turbo prune --scope=web --docker

# Need to change to correct app folder name for each flow [ deps, builder, runner ]
# set your APP_NAME
ENV APP_NAME=web

# Create apps folder
RUN mkdir -p apps

# Copy related packages and apps, warning: this solution can't copy package that only needed for the build.
COPY ./packages /app/packages
COPY ./apps/$APP_NAME /app/apps/$APP_NAME

# Copy nextjs env, comment out if you don't have .env
COPY ./apps/$APP_NAME/.env.staging.sample /app/apps/$APP_NAME/.env.production

# Copy clean root files to root folder
COPY ./turbo.json /app/turbo.json
COPY ./.gitignore /app/.gitignore
COPY ./.npmrc /app/.npmrc
# Add lockfile and package.json's of isolated subworkspace
COPY ./package.json /app/package.json
COPY ./pnpm-lock.yaml /app/pnpm-lock.yaml
COPY ./pnpm-workspace.yaml /app/pnpm-workspace.yaml

# 2. Rebuild the source code only when needed
FROM node:alpine AS builder
WORKDIR /app

# Build the project and its dependencies
ENV NEXT_TELEMETRY_DISABLED 1

# set your APP_NAME
ENV APP_NAME=web

COPY --from=deps /app/ .
RUN corepack enable
RUN pnpm install
RUN pnpm turbo run build --filter=$APP_NAME...

# 3. Production image, copy all the files and run next
FROM node:alpine AS runner

WORKDIR /app

ENV NEXT_TELEMETRY_DISABLED 1
ENV NODE_ENV=production

# set your APP_NAME
ENV APP_NAME=web

RUN addgroup -g 1001 -S nodejs
RUN adduser -S nextjs -u 1001

# comment out below if you don't have public folder
COPY --from=builder /app/apps/$APP_NAME/public public

COPY --from=builder --chown=nextjs:nodejs /app/apps/$APP_NAME/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/apps/$APP_NAME/.next/static ./.next/static

USER nextjs

EXPOSE 3000

ENV PORT 3000

CMD node apps/$APP_NAME/server.js

remember enable standalone and set root path at next.config.js

const withTM = require('next-transpile-modules')(['ui'])
const path = require('path')

module.exports = withTM({
  reactStrictMode: true,
  output: 'standalone',
  experimental: {
    outputFileTracingRoot: path.join(__dirname, '../../'),
  },
})

[chris-olszewski]

With #1819 prune is now supported for repos using pnpm

[zigang93]

@chris-olszewski you forget to update the docs

[nathanhammond]

Will update the docs when it is released. 🎉

[ScreamZ]

Looks like it is :-) Can you update doc ? ahah

[trm217]

@nathanhammond Have the docs been updated for pnpm examples?
I have been trying to get turbo prune working with pnpm in a docker container for the better part of a day, but I have no change of getting it working.
Would really appreciate it if there were more examples & docs for using turbo with pnpm.

[nathanhammond]

@trm217 There is basically no material difference between each of the package managers. Remove the lockfile, create pnpm-workspace.yml, change * to workspace:*, and then do a global s/yarn/pnpm/g.

If you have particular problems open a new issue including details of the error message that you're encountering.

[nicu-chiciuc]

@nathanhammond

Remove the lockfile

this sounds like a bad advice since (I assume) it means that the packages that I expect to be installed on my machine are not guaranteed to be the packages installed in docker.

I've mentioned it in a different issue, but it seems bizarre that the official docs for turborepo recommend using pnpm

but provide examples of using yarn with docker (and no example with pnpm).

If you have particular problems open a new issue including details of the error message that you're encountering.

As someone who just started writing Dockerfiles, it's hard to pinpoint the exact issues I have. Since I basically took the yarn Dockerfile and started going line by line and (with the help of ChatGPT and dive) updating it to pnpm.
But now I'm in the position where I have to deeply understand how pnpm and docker works just to make it work for me.

I think the correct solution is either:

• Don't recommend pnpm in the official docs (I would've used npm I guess)
• Provide an example of using pnpm

[nicu-chiciuc]

I think my previous message sounded too demanding, so I made a PR in the hopes that we solve the issue using the second option (Provide an example of using pnpm)
#5536

[thenbe]

In case someone's use case is not covered by @nicu-chiciuc's example, or you just want to look at more examples, you can get pretty specific with sourcegraph's search.