pnpm in with-docker example

[bibblebabl]

With #1819 prune is now supported for repos using pnpm

It looks like pnpm is the most common package manager in your repository and examples. Turbo prune has recently got pnpm support. Do you have plans to update to update this example to pnpm? I was thinking of doing a PR, but I wanted to clarify, maybe you have your own thoughts on this.

[Manubi]

Would love to see that. Would also like to see an example that is a little more complex like with eg. nginx, vitejs, express, prisma and postgres. Kind of a template to quickly get going.

[abetoots]

Any updates on this? @bibblebabl

[trm217]

Could anyone of you get this working?
I am trying to build a docker image using pnpm + turbo and during the prune step I get runtime errors left and right from the turbo command, wasted around 5h at this point.

[trm217]

So when trying to build apps/web in the reproduction linked above, this is the output:

docker build -t foo  -f ./apps/web/Dockerfile . --no-cache
[+] Building 39.5s (22/29)
 => [internal] load build definition from Dockerfile                                                                                                                       0.0s
 => => transferring dockerfile: 37B                                                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                                                          0.0s
 => => transferring context: 34B                                                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/node:alpine                                                                                                             1.8s
 => [auth] library/node:pull token for registry-1.docker.io                                                                                                                0.0s
 => [internal] load build context                                                                                                                                          0.1s
 => => transferring context: 128.92kB                                                                                                                                      0.1s
 => CACHED [runner 1/9] FROM docker.io/library/node:alpine@sha256:d3a3d691797cef0b70e361788a2aeb9dd7925112996719628d4bcd4bd27009b0                                         0.0s
 => CACHED [runner 2/9] WORKDIR /app                                                                                                                                       0.0s
 => [runner 3/9] RUN addgroup --system --gid 1001 nodejs                                                                                                                   0.3s
 => [builder 2/8] RUN apk add --no-cache libc6-compat                                                                                                                      1.0s
 => [runner 4/9] RUN adduser --system --uid 1001 nextjs                                                                                                                    0.3s
 => [builder 3/8] RUN apk update                                                                                                                                           0.9s
 => [builder 4/8] WORKDIR /app                                                                                                                                             0.0s
 => [builder 5/8] RUN yarn global add pnpm                                                                                                                                 2.7s
 => [installer  6/13] COPY .gitignore .gitignore                                                                                                                           0.0s
 => [builder 6/8] COPY . .                                                                                                                                                 0.1s
 => [builder 7/8] RUN pnpm i                                                                                                                                              17.8s
 => [builder 8/8] RUN pnpm dlx turbo prune --scope=web --docker                                                                                                            2.3s
 => [installer  7/13] COPY --from=builder /app/out/json/ .                                                                                                                 0.0s
 => [installer  8/13] COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml                                                                                         0.0s
 => [installer  9/13] COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml                                                                               0.0s
 => [installer 10/13] RUN pnpm install                                                                                                                                    12.8s
 => ERROR [installer 11/13] COPY --from=builder /app/out/full/ .                                                                                                           0.0s
------
 > [installer 11/13] COPY --from=builder /app/out/full/ .:
------
cannot copy to non-directory: /var/lib/docker/overlay2/lgzbjapsep7ex3be9nhi4w9gf/merged/app/apps/web/node_modules/@types/node

[trm217]

All I did was migrate the with-docker example from yarn to pnpm as you mentioned in the other comment you made (here).

[trm217]

Finally managed to resolve it, the issue was the same as described here (solution with .docker-ignore worked)

[bibblebabl]

Finally managed to resolve it, the issue was the same as described here (solution with .docker-ignore worked)

Do you mean that you solved it by adding to .dockerignore the following lines?

node_modules
**/node_modules

[trm217]

The last line you mentioned above did the job for me 👍🏻

[ARMATAV]

Still a massive pain, need a Docker + pnpm example

[enyelsequeira]

any luck with this?

[ARMATAV]

None at all - switching to yarn 3 with nodeLinker: node-modules for now; it seems that they have some work to do integrating it properly with how pnpm does symlinking into Docker containers.

[enyelsequeira]

has anyone been able to deploy in docker using the example that was given? I am having trouble with it and getting this error

[bibblebabl]

This thread is specifically about turbo+docker+pnpm.

First of all, have you seen the possible solution with adding node_modules to dockerignore above? Did it help? And if you are using the example yarn working repo could be downloaded from here https://github.com/vercel/turbo/tree/main/examples/with-docker

[ARMATAV]

Ah I get it now. You must have the root node_modules or the symlinks inside of your docker container (in your app directory's node_modules) will point to nothing.

If they point to nothing, you get this cannot copy to non-directory error.

So either;

• COPY the root node_modules into your container
• Mount it as a volume

volumes:
      - ./apps/jobs:/app/apps/jobs
      - ./apps/jobs/node_modules:/app/apps/jobs/node_modules
      # lol
      - ./node_modules:/app/node_modules

• Or maybe someone could clarify the correct prune-like command that will put actual files in place of the symbolic links when you run the prune --docker command. Maybe that command is deploy or something? No idea.

Either way, those symlinks must point to something, or you're stuck. No issue with yarn in this capacity, since they aren't symlinks.

Another person found this to be true: #1997 (comment)

Also big note/caveat, my use case is purely development with docker-compose, deployment is not done via containers, so just mounting it works for me.

[JasperAlexander]

How does your Dockerfile look like?

[ARMATAV]

Like this, it's purely for running a container to spin up a jobs system so I can do dev stuff, so it's effectively empty

dev Dockerfile

FROM node:alpine AS builder
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
RUN apk update

# Set working directory
WORKDIR /app
RUN yarn global add pnpm
COPY . .
RUN pnpm dlx turbo prune --scope=@myapp/jobs --docker

FROM node:alpine AS runner
RUN yarn global add pnpm

WORKDIR /app

COPY --from=builder /app/out/full/ .
COPY turbo.json turbo.json

CMD pnpm --dir ./apps/jobs run develop

[ARMATAV]

However, this will break with Prisma, since it will generate the types in some weird symlinked location.

It does look like the best way forward is to drop pnpm with turbo in favor of yarn/npm with turbo.

[JasperAlexander]

Does anyone have a working version of a Dockerfile for an app? I cannot find this on GitHub and tried a lot of things, but I keep running into issues. I'm using PNPM.

[JasperAlexander]

This Dockerfile which uses pnpm works for me:

# Builder
FROM node:alpine AS builder
RUN apk add --no-cache libc6-compat
RUN apk update

WORKDIR /app
RUN yarn global add turbo
COPY . .
RUN turbo prune --scope=appname --docker

# Installer
FROM node:alpine AS installer
RUN apk add --no-cache libc6-compat
RUN apk update

WORKDIR /app
RUN yarn global add pnpm
RUN yarn global add turbo

COPY --from=builder /app/out/json/ .
COPY --from=builder /app/out/pnpm-lock.yaml ./pnpm-lock.yaml
COPY --from=builder /app/out/pnpm-workspace.yaml ./pnpm-workspace.yaml
RUN pnpm install

COPY --from=builder /app/out/full/ .
COPY turbo.json turbo.json

# Uncomment and use build args to enable remote caching
# ARG TURBO_TEAM
# ENV TURBO_TEAM=$TURBO_TEAM

# ARG TURBO_TOKEN
# ENV TURBO_TOKEN=$TURBO_TOKEN

RUN turbo build --filter=appname...

# Runner
FROM node:alpine AS runner
WORKDIR /app

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
USER nextjs

COPY --from=installer /app/apps/appname/next.config.js .
COPY --from=installer /app/apps/appname/package.json .

# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=installer --chown=nextjs:nodejs /app/apps/appname/.next/standalone ./
COPY --from=installer --chown=nextjs:nodejs /app/apps/appname/.next/static ./apps/appname/.next/static
# COPY --from=installer --chown=nextjs:nodejs /app/apps/appname/public ./apps/appname/public

CMD node apps/appname/server.js

[Jarrodsz]

Could you please share how you deploy your apps with this dockerfile?

like what files are involved and in what directory you run which commands to get deployment working? Have been struggling for days with this totally pissedoff on pnpm and turbo repro atm. Spend lots of time moving everything to monorepro and now its such an headache to get apps deployed

[JasperAlexander]

Put the Dockerfile in the folder of your app and the following docker-compose.yml file in the root of your monorepo, than run docker-compose up in the root of your monorepo. Make sure Docker is running. What are you running into?

version: '3.8'

services:
  app:
    container_name: [name]
    build: 
      context: .
      dockerfile: ./apps/[name of your app]/Dockerfile
    ports:
      - '3000:3000'

[crice88]

We had to discontinue usage of pnpm and turborepo. Though we really enjoyed the performance gains of using pnpm the way that it was symlinking types in our dependencies was a constant headache.

[Jarrodsz]

Also wasted 3 days on this already.
Several discord questions got no working response.

anyone has a working pnpm repro for docker deployments and pnpm.

none of the examples provide a example.

i just want to deploy an app in
/apps/myapp

with a dockerfile to a docker container

[JasperAlexander]

This Turborepo contains a working Docker setup with PNPM.

[Jarrodsz]

This Turborepo contains a working Docker setup with PNPM.

Thank you
But this uses a docker-compose and is basically the turbo example with docker?

I need to build a single app from a dockerfile and have no need for a docker-compose.

I need to build a single app in apps/myapp for example towards a docker container. So its a selfcontained docker container with only that app with installed local packages.

The documentation does not mention how to build the apps or do that. Did i misa something?

Thank you for your reply🙏

[marianoquevedo]

@Jarrodsz we also couldn't make it work with pnpm and we switched to using yarn as the package manager.
We used the with-docker example, picked yarn as the package manager and it worked

[JasperAlexander]

The instructions of how to run a docker container are on the docker website. Just build and run the container. What problems are you running into?

[zdenkolini]

I got a setup working including a docker-compose for 3 frontend applications, the setup is definitely unoptimized, but I don't have the experience, nor time to fiddle with it.

Let me know if this helped.

Dockerfile

# Base image
FROM node:18-alpine AS base
RUN npm install -g turbo pnpm

# This Dockerfile is copy-pasted into our main docs at /docs/handbook/deploying-with-docker.
# Make sure you update both files!

# Builder stage
FROM base AS builder
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk update && apk add --no-cache libc6-compat
# Set working directory
WORKDIR /app
COPY . .
RUN turbo prune --scope=admin-panel --docker

# Installer stage
FROM base AS installer
RUN apk update && apk add --no-cache libc6-compat
WORKDIR /app

# First install the dependencies (as they change less often)
COPY --from=builder /app/out/json/ .
RUN pnpm install

# Build the project
COPY --from=builder /app/out/full/ .
RUN pnpm turbo build --filter=admin-panel...
RUN ls -la apps/admin-panel/dist

# Runner stage
FROM nginx:alpine AS runner
WORKDIR /usr/share/nginx/html

RUN ls -la
# Copy built files from installer stage
COPY --from=installer /app/apps/admin-panel/dist .

# Copy nginx configuration
COPY nginx.conf /etc/nginx/nginx.conf

# Expose port 80
EXPOSE 80

# Start nginx
CMD ["nginx", "-g", "daemon off;"]

docker-compose.yml

version: "3"

services:
  customer-portal:
    container_name: customer-portal
    build:
      context: .
      dockerfile: ./apps/customer-portal/Dockerfile
    restart: always
    ports:
      - 3000:80
    # networks:
    #   - app_network
  employee-portal:
    container_name: employee-portal
    build:
      context: .
      dockerfile: ./apps/employee-portal/Dockerfile
    restart: always
    ports:
      - 3001:80
    # networks:
    #   - app_network
  admin-panel:
    container_name: admin-panel
    build:
      context: .
      dockerfile: ./apps/admin-panel/Dockerfile
    restart: always
    ports:
      - 3002:80
    # networks:
    #   - app_network
# Define a network, which allows containers to communicate
# with each other, by using their container name as a hostname
# networks:
#   app_network:
#     external: true

nginx.conf

events {
}

http {
  include mime.types;

  server {
    listen       80;
    server_name  localhost;

    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
        try_files $uri $uri/ /index.html;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   400 401 403 404 405 408 500 502 503 504 =200 /index.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
        try_files $uri $uri/ /index.html;
    }
  }
}