`2.4.1` backend docker run error `cannot create /root/.netrc: Permission denied` #3552

sinlov · 2024-03-25T06:41:44Z

Component

server, agent

Describe the bug

labels: # https://woodpecker-ci.org/docs/usage/workflow-syntax#labels
  backend: docker
  platform: linux/amd64

steps:
  python-poetry-gcc:
    image: fnndsc/python-poetry:1.7.1
    pull: false
    commands:
      - whoami
      - id

will be error as

/bin/sh: 3: cannot create /root/.netrc: Permission denied

but if use woodpecker 2.4.0 can run and print

+ whoami
mambauser
+ id
uid=1000(mambauser) gid=1000(mambauser) groups=1000(mambauser)

System Info

## woodpecker v2.4.1
labels: # https://woodpecker-ci.org/docs/usage/workflow-syntax#labels
  backend: docker
  platform: linux/amd64

steps:
  python-poetry-gcc:
    image: fnndsc/python-poetry:1.7.1
    pull: false
    commands:
      - whoami
      - id

Additional context

beacuse fnndsc/python-poetry:1.7.1 image not use root

For security, the python build file and the permission to run the cache are isolated

Validations

Read the docs.
Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
Checked that the bug isn't fixed in the next version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]

The text was updated successfully, but these errors were encountered:

qwerty287 · 2024-04-09T14:24:51Z

Are you sure that's a woodpecker issue? We didn't change anything related to docker in 2.4.1. For non-root images see #1077

sinlov · 2024-04-09T17:55:19Z

Are you sure that's a woodpecker issue? We didn't change anything related to docker in 2.4.1. For non-root images see #1077

Confirm that 2.4.1 has appeared, 2.4.0 In the same host environment, the docker-compose file only changes the version number and can be used

docker information

Client: Docker Engine - Community
 Version:           26.0.0
 API version:       1.45
 Go version:        go1.21.8
 Git commit:        2ae903e
 Built:             Wed Mar 20 15:17:48 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.0.0
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.8
  Git commit:       8b79278
  Built:            Wed Mar 20 15:17:48 2024
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

some config of .docker-python-poetry-gcc.yml

labels:
  backend: docker
  platform: linux/amd64

skip_clone: true

steps:
  init:
    image: sinlov/woodpecker-plugin-env:1.13.0 # https://hub.docker.com/r/sinlov/woodpecker-plugin-env/tags
    pull: false
    settings:
      # debug: true
      env_printer_padding_left_max: 36 # padding left max
  python-poetry-gcc:
    image: fnndsc/python-poetry:1.7.1
    pull: false
    commands:
      - export HOME="$(getent passwd $(id -u) | cut '-d:' -f6)"
      - whoami
      - id
      - pwd
      - ls -alFh

2.4.1 log and print

...
CI_SYSTEM_PLATFORM                   linux/amd[6](https://woodpecker.xxxxxx.com/repos/1/pipeline/26/2#L6)4
CI_SYSTEM_VERSION                    2.4.1

CI_WORKFLOW_NAME                     docker-python-poetry-gcc
...
....
/bin/sh: 3: cannot create /root/.netrc: Permission denied

2.4.0 log and print

CI_SYSTEM_PLATFORM                   linux/amd[6](https://woodpecker.xxxxxxxx.cn/repos/1/pipeline/11/7#L6)4
CI_SYSTEM_VERSION                    2.4.0

CI_WORKFLOW_NAME                     docker-python-poetry-gcc
...

+ export HOME="$(getent passwd $(id -u) | cut '-d:' -f6)"
+ whoami
mambauser
+ id
uid=1000(mambauser) gid=1000(mambauser) groups=1000(mambauser)
+ pwd
/woodpecker/src/gitea.xxxxxxxx.cn/woodpecker-zoo/guidance-woodpecker-docker
+ ls -alFh
total 2.0K
drwxr-xr-x 2 root root 2 Apr  9 17:44 ./
drwxr-xr-x 3 root root 3 Apr  9 17:44 ../

Inheriting the mirror image separately, maintaining a mirror to change user to root will fx. but rolling back the version will lead to unnecessary trouble.

FROM fnndsc/python-poetry:1.7.1

USER root:root

RUN apt-get update \
  && apt-get install -y make gcc \
  && apt-get autoclean \
  && apt-get clean \
  && apt-get autoremove \
  && rm -rf /var/lib/apt/lists/*

# USER mambauser:mambauser

drone runer drone/drone-runner-docker:1.8.2 can change user as

...
  - name: python-poetry-build
    image: fnndsc/python-poetry:1.7.1 # https://hub.docker.com/r/fnndsc/python-poetry/tags
    pull: if-not-exists
    user: root # for image fnndsc/python-poetry:1.7.1 default use `mambauser` can not management file
    volumes:
      - name: tmp-python-poetry-gcc-cache
        path: /root/.cache
      - name: tmp-python-poetry-gcc-local
        path: /root/.local
    commands:
      - whoami
      - id
      - pwd
      - ls -alFh .
...

RaymiiOrg · 2024-05-09T15:09:50Z

Experiencing this issue as well after upgrading from 1.0.0 to 2.4.1. Reverting back to 2.4.0 did not fix it.

My docker environment is alpine 3.19

RaymiiOrg · 2024-05-13T05:37:29Z

The docker image I used has a

USER:

section. I have rebuilt that image without the USER: part and now the problem does not occur anymore.

Hentioe · 2024-07-03T22:41:39Z

This is probably because Woodpecker incorrectly modified some environment variables when starting the container, causing the user executed in the container to not match the HOME variable. This also caused other related failures on my side, and I had to manually set the HOME variable.

SourceCodeBot · 2024-08-06T08:53:50Z

since the milestones changed, is that issue planed for 2.x.x or 3.x.x?

qwerty287 · 2024-08-06T14:40:08Z

3.x.x. because we'll release 3.0.0 next.

xoxys · 2024-08-18T18:42:37Z

See https://github.com/woodpecker-ci/woodpecker/blob/main/pipeline/backend/common/script.go#L31

Why do we hard-code HOME=/root while we don't enforce that the container user is root? Either we don't hard-code HOME or we set --user=root if we start step containers. Or am I missing something?

As @Hentioe said already setting HOME=/home/<container-user> manually works, however keep in mind you have to set it in the commands section via export HOME=/home/<container-user>, using the step environment option doesn't work.

@woodpecker-ci/maintainers do you have a preferred way? I tend to drop the hard-coded HOME env var, but not sure about side effects.

6543 · 2024-08-19T12:31:30Z

yes we can drop the hardcoded home var

xoxys · 2024-08-19T13:41:24Z

Ok, but what happens if $HOME is not set by the container, e.g. on distroless/scratch containers or due to any other reason? We rely on it at some parts e.g. to write $HOME/.netrc.

zc-devs · 2024-08-19T15:56:39Z

on distroless/scratch

There is no shell.

$HOME is not set by the container

What if I run container as user and group 12345 (suppose, there is no such user/group)?
I guess, container works under arbitrary user or not. The same with HOME.

If HOME is not set, we probably could set it to workspace root (/woodpecker).

mortbauer · 2024-09-20T09:36:32Z

Any fix or workaround for this, just trying to make the switch from drone and falling over this?

qwerty287 · 2024-09-20T11:32:26Z

What about #3552 (comment)?

mortbauer · 2024-09-21T08:13:07Z

@qwerty287 thanks for the suggestion but doesn't seem to work for me, will adapt the image i use for now.

zc-devs · 2024-09-21T13:40:51Z

manually set the HOME variable

doesn't seem to work for me

@mortbauer, what exactly did you try?

#3164, #1077

pat-s · 2024-10-01T19:08:49Z

I've debugged this a bit (using 2.7.0) and I think there are two different issues here:

In "trusted" mode when using a non-root user: $HOME is hard-coded to root and can't be overwritten and the .netrc injection is happening before the first command starts.
In "untrusted" mode when using a non-root user (related to Add support for nonroot OCI images #1077): While the .netrc injection does not happen here, a manual clone as the first command still fails because the container user is starting in $CI_WORKSPACE (which belongs to root) and doesn't have permissions there.

For (1), I currently only see the option to use an image with root as the container user until the hard-coded $HOME is removed and we ensured that the .netrc injection is happening dynamically in whatever $HOME is in the container (currently is still hard-coded to /root in the codebase).

For (2), the following should be a generic snippet that works for most pipelines (untrusted) and containers:

      - export HOME=/home/$(whoami)
      - cd ~ && git clone --depth 1 https://<username>:$$<secret token>@<forge URL>/${CI_REPO}.git && cd ${CI_REPO_NAME}

In addition, we should ensure that $CI_WORKSPACE is owned by the container user so the cd ~ is not needed.

Manually setting HOME in environment: doesn't have an effect as it does not overwrite the hard-coded default.

sinlov added the bug Something isn't working label Mar 25, 2024

qwerty287 added the backend/docker label Mar 25, 2024

qwerty287 added this to the 2.5.0 milestone Mar 25, 2024

qwerty287 modified the milestones: 2.5.0, 2.6.0 Apr 15, 2024

qwerty287 modified the milestones: 2.6.0, 2.7.0 Jun 13, 2024

6543 modified the milestones: 2.7.0, 2.8.0 Jul 17, 2024

qwerty287 added this to the 3.0.0 milestone Sep 21, 2024

pat-s mentioned this issue Oct 2, 2024

Kubernetes | Docker: Add support for rootless images #4151

Merged

1 task

6543 closed this as completed in #4151 Nov 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`2.4.1` backend docker run error `cannot create /root/.netrc: Permission denied` #3552

`2.4.1` backend docker run error `cannot create /root/.netrc: Permission denied` #3552

sinlov commented Mar 25, 2024 •

edited

Loading

qwerty287 commented Apr 9, 2024

sinlov commented Apr 9, 2024 •

edited

Loading

RaymiiOrg commented May 9, 2024 •

edited

Loading

RaymiiOrg commented May 13, 2024

Hentioe commented Jul 3, 2024 •

edited

Loading

SourceCodeBot commented Aug 6, 2024

qwerty287 commented Aug 6, 2024

xoxys commented Aug 18, 2024 •

edited

Loading

6543 commented Aug 19, 2024

xoxys commented Aug 19, 2024

zc-devs commented Aug 19, 2024 •

edited

Loading

mortbauer commented Sep 20, 2024

qwerty287 commented Sep 20, 2024

mortbauer commented Sep 21, 2024

zc-devs commented Sep 21, 2024

pat-s commented Oct 1, 2024

2.4.1 backend docker run error cannot create /root/.netrc: Permission denied #3552

2.4.1 backend docker run error cannot create /root/.netrc: Permission denied #3552

Comments

sinlov commented Mar 25, 2024 • edited Loading

Component

Describe the bug

System Info

Additional context

Validations

qwerty287 commented Apr 9, 2024

sinlov commented Apr 9, 2024 • edited Loading

RaymiiOrg commented May 9, 2024 • edited Loading

RaymiiOrg commented May 13, 2024

Hentioe commented Jul 3, 2024 • edited Loading

SourceCodeBot commented Aug 6, 2024

qwerty287 commented Aug 6, 2024

xoxys commented Aug 18, 2024 • edited Loading

6543 commented Aug 19, 2024

xoxys commented Aug 19, 2024

zc-devs commented Aug 19, 2024 • edited Loading

mortbauer commented Sep 20, 2024

qwerty287 commented Sep 20, 2024

mortbauer commented Sep 21, 2024

zc-devs commented Sep 21, 2024

pat-s commented Oct 1, 2024

`2.4.1` backend docker run error `cannot create /root/.netrc: Permission denied` #3552

`2.4.1` backend docker run error `cannot create /root/.netrc: Permission denied` #3552

sinlov commented Mar 25, 2024 •

edited

Loading

sinlov commented Apr 9, 2024 •

edited

Loading

RaymiiOrg commented May 9, 2024 •

edited

Loading

Hentioe commented Jul 3, 2024 •

edited

Loading

xoxys commented Aug 18, 2024 •

edited

Loading

zc-devs commented Aug 19, 2024 •

edited

Loading