Pipenv no longer working under the root directory is not listed as a breaking change

[gps035]

Issue description

Our docker builds failed this morning due to a change in behaviour when running pipenv install in the root directory.
When running with the latest version (2020.5.28) an error is thrown after dependencies are built and installed ERROR: Pipenv is not intended to work under the root directory, please choose another path.
It looks like this is a change introduced in #3386, relating to an issue raised in #3434.
This issue is really just to point out that this behaviour change should be listed as breaking in the changelog, as it doesn't currently appear to be mentioned.
For the time being we have worked around this by pinning pipenv to the last version (2018.11.26).

Expected result

N/A

Actual result

N/A

Steps to replicate

N/A

[mohamedMok]

hi @gps035,

I'm having the same issue with pipenv.
Could you please show how you've pinned the pipenv version ?

Thanks

[jean-malo]

I can confirm that this behaviour does not happen on version 2018.11.26.

@mohamedMok You can use pip install 'pipenv==2018.11.26' which is the last version that does not have this breaking change.

[frostming]

@gps035 Any chance sending a PR to mention it in the CHANGELOG?

I have filed a PR to address this issue, thanks for everyone.

[kbenchaaboun]

Not a funny breaking changes, all our dockers are using pipenv install during build :/

[killuazhu]

Run into the same issue, using fix in #4273 (comment) worked for me.

[techalchemy]

Our docker builds failed this morning due to a change in behaviour when running pipenv install in the root directory.

can you explain the workflow here -- are you using --system?

[techalchemy]

As I just mentioned in #4275:

The main reason for the change in the first place is due to locating virtual environments & related python paths -- as far as I was aware, this was a substantial cause of bugs and breakages and basically didn't work. The fact that it is breaking workflows is the first I am hearing of it working at all.

This is not intended to be a breaking change, it's intended to prevent a previously broken interaction -- for anyone for whom this was working, please include the full set of command line arguments you were passing to pipenv (e.g. pipenv install --<whatever> and information about your workflow:

• were you using docker? some other container infrastructure?
• what user was invoking the command? what UID (basically, was it a root account)
• were you passing --system to pipenv, creating your own virtualenv, or allowing pipenv to create one for you?
• what OS was pipenv running under?
• What python version?

Thats probably enough for now

[engstrom]

@techalchemy This is the relevant portions of our Dockerfile which no longer works.

FROM python:3.8

RUN pip install --no-cache-dir pipenv
RUN pipenv install --system --deploy

[jayaddison]

@techalchemy Thanks for canvassing use cases. Here's another example:

• Containerization scenario: yes, although not strictly Docker (OCI images)
• Pipenv runs within the container build environment as uid=0
• The --system flag is not being passed
• Linux
• Python 3.8

In case it's useful for investigation or reproducing the issue, the build steps are visible in this Make target.

[DylanBohlender]

For anyone who wants to use the latest pipenv with --system, adapting your Dockerfile by setting a WORKDIR and copying your Pipfile/lockfile into it as follows may be helpful:

WORKDIR /code
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
COPY . /code/

[killuazhu]

• building dockerfile
• root
• see code below
• python:3-slim docker image, comes with Debian GNU/Linux 10
• 3.8

FROM python:3-slim AS base

ENV PYROOT /pyroot
ENV PYTHONUSERBASE $PYROOT
ENV PATH $PATH:$PYROOT/bin

FROM base AS builder
RUN pip install pipenv
COPY Pipfile* ./
RUN PIP_USER=1 PIP_IGNORE_INSTALLED=1 pipenv install --system --deploy --ignore-pipfile

[ebordon]

For anyone who wants to use the latest pipenv with --system, adapting your Dockerfile by setting a WORKDIR and copying your Pipfile/lockfile into it as follows may be helpful:

WORKDIR /code
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
COPY . /code/

This worked for me. Just remember to create the /code dir before.

[micahjsmith]

This worked for me. Just remember to create the /code dir before.

the WORKDIR command already creates the directory if it does not exist

[killuazhu]

Using WORKDIR did not work for me. I'm getting error

Step 9/9 : RUN PIP_USER=1 PIP_IGNORE_INSTALLED=1 pipenv install --system --deploy --ignore-pipfile
 ---> Running in da6fa387210f
Installing dependencies from Pipfile.lock (387af5)…
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/build-5NmaZ4l5/bin/python: not found

Output: 
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/build-5NmaZ4l5/bin/python: not found

Output: 
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/build-5NmaZ4l5/bin/python: not found

Output: 
^Cmake: *** [build-image-base] Interrupt: 2

when using dockerfile below

FROM python:3-slim AS base

ENV PYROOT /pyroot
ENV PYTHONUSERBASE $PYROOT
ENV PATH $PATH:$PYROOT/bin

FROM base AS builder
WORKDIR /build
RUN pip install pipenv
COPY Pipfile* /build/
RUN PIP_USER=1 PIP_IGNORE_INSTALLED=1 pipenv install --system --deploy --ignore-pipfile

[micahjsmith]

this does not seem to actually be build error, see #4220

[techalchemy]

As a sidenote, $PIP_USER gets unset by pipenv and I am relatively uncertain about how $PYTHONUSERBASE interacts with it

Also, the --deploy flag would be somewhat pointless with the --ignore-pipfile flag -- --deploy is used to guarantee that your Pipfile and your Pipfile.lock are in alignment, i.e. that your Pipfile.lock was generated from the corresponding Pipfile. If you indicate that you want to ignore your pipfile, this check never happens.

In any case @killuazhu the error in the logs you included are possibly related to the manipulations of your python path, but will require further investigation if you can file a separate issue

[frostming]

For reference, the original issue of #3434 occurs when one tries to pipenv install under / without a Pipfile. And the setup in this ticket is to pipenv install under / with a Pipfile, which used to be working on 2018.11.26. However, #3386 chose a wrong resolution approach, that it prevents the usage from the root directory entirely.

[nateg-gladly]

Thank you for fixing, do you have an ETA on when the fix will be included in a new release of the pypi package?

[frostming]

we need to ensure all regression issues are fixed and the new release will be out in the next week

[nateg-gladly]

Awesome thank you, appreciate it!

[roykim98]

I can confirm that this behaviour does not happen on version 2018.11.26.

@mohamedMok You can use pip install 'pipenv==2018.11.26' which is the last version that does not have this breaking change.

I was getting a slightly different error when running python3 -m pipenv install --three --system

Output:
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

Pinning to the old version also worked for me. Thanks!

[iamWing]

I can confirm that this behaviour does not happen on version 2018.11.26.
@mohamedMok You can use pip install 'pipenv==2018.11.26' which is the last version that does not have this breaking change.

I was getting a slightly different error when running python3 -m pipenv install --three --system

Output:
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

Pinning to the old version also worked for me. Thanks!

I’m having the same issue. Now pinning to the old version as a workaround

[mlej8]

we need to ensure all regression issues are fixed and the new release will be out in the next week

This issue is still present in the 2020.6.2 release:

Output:
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-lp47FrbD/bin/python: not found

Could you please confirm if this issue was expected to be fixed in 2020.6.2 release?

[gpakosz]

I can confirm I'm hitting this issue with the following Dockerfile

FROM python:3.7-slim

ENV LC_ALL C.UTF-8
ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && \
    apt-get upgrade && \
    apt-get install -y --no-install-recommends libldap2-dev libsasl2-dev libssl-dev && \
    apt-get clean autoclean && rm -rf /var/lib/apt/* /var/cache/apt/* && \
    apt-get autoremove --purge && \
    pip install pipenv --no-cache-dir

WORKDIR /app

COPY Pipfile Pipfile.lock ./
RUN pipenv install --deploy --system --verbose

ENTRYPOINT ["uvicorn", "web.main:app", "--host", "0.0.0.0"]

EXPOSE 8000/tcp

[gpakosz]

@frostming Can you please reopen the issue?

[alexwaweru]

I can also confirm I'm hitting this issue with the following Dockerfile:

FROM python:3.7.6-slim-stretch
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /app
COPY .  /app
RUN pip install --upgrade pip
RUN pip install pipenv
RUN pipenv install --system --deploy --ignore-pipfile
CMD ["/bin/bash", "scripts/entrypoint.sh"]

Here is the error:

Step 10/11 : RUN pipenv install --system --deploy --ignore-pipfile
 ---> Running in 00386bcedd89
Installing dependencies from Pipfile.lock (d14b54)…
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

Output: 
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

Output: 
Failed to load paths: /bin/sh: 1: /root/.local/share/virtualenvs/app-4PlAip0Q/bin/python: not found

[franciscojavierarceo]

For folks still having this issue the easiest fix is to configure your Dockerfile as:

FROM python:3.7-slim

# Set environment varibles
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Set work directory
WORKDIR /code


# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv==2018.11.26 && pipenv install --system             # <- this is the fix
...

[gnu-lorien]

This bug manifested slightly differently for me, but #4273 (comment) solved it nonetheless.

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/pipenv/cli/command.py", line 233, in install
    retcode = do_install(
  File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 2052, in do_install
    do_init(
  File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 1304, in do_init
    do_install_dependencies(
  File "/usr/local/lib/python3.9/site-packages/pipenv/core.py", line 842, in do_install_dependencies
    deps_list = list(lockfile.get_requirements(dev=dev, only=dev_only))
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/lockfile.py", line 273, in get_requirements
    yield Requirement.from_pipfile(k, v)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2747, in from_pipfile
    r = FileRequirement.from_pipfile(name, pipfile)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1841, in from_pipfile
    arg_dict["parsed_line"] = Line(line)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 171, in __init__
    self.parse()
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1304, in parse
    self.parse_name()
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1030, in parse_name
    name = self._parse_name_from_path()
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 993, in _parse_name_from_path
    metadata = get_metadata(self.path)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 567, in get_metadata
    dist = get_distinfo_dist(path, pkg_name=pkg_name)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 535, in get_distinfo_dist
    dist_dir = next(iter(find_distinfo(path, pkg_name=pkg_name)), None)
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 527, in find_distinfo
    for dist_dir in dist_dirs:
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 517, in <genexpr>
    dist_dirs = (
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 490, in iter_metadata
    with contextlib.closing(ScandirCloser(p)) as path_iterator:
  File "/usr/local/lib/python3.9/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 450, in __init__
    self.iterator = scandir(path)
FileNotFoundError: [Errno 2] No such file or directory: '/dev/fd/3'