Locking may fail while unparsing setup.py if version is read from environment #4274

pesstibo · 2020-05-28T11:01:00Z

Issue description

When the source directory is added to the Pipfile (using pipenv install -e .) and the version number defined in setyp.py is read from the environment, then unparsing setup.py may throw an unexpected exception when running pipenv lock.

Running pipenv lock --clear, as suggested on the "diagnose documentation" page, throws the same exception.

Expected result

I would expect locking operation to continue (and complete), while not necessarily detecting the version number of the source package.

Actual result

The operation aborts due to an unexpected exception:

❯ pipenv lock --verbose
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1151, in ast_unparse
    unparsed[unparse(k)] = unparse(v)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1040, in ast_unparse
    items = unparse(analyzer.assignments[assignment])
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1140, in ast_unparse
    val = unparse(item.value, recurse=False)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1004, in ast_unparse
    unparsed = unparsed[unparse(item.slice.value)]
TypeError: 'Attribute' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 370, in lock
    write=not state.quiet,
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1117, in do_lock
    keep_outdated=keep_outdated
  File "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 1318, in venv_resolve_deps
    deps, project, r=False, include_index=True
  File "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 1434, in convert_deps_to_pip
    new_dep = Requirement.from_pipfile(dep_name, dep)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2743, in from_pipfile
    r = FileRequirement.from_pipfile(name, pipfile)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1843, in from_pipfile
    arg_dict["setup_info"] = arg_dict["parsed_line"].setup_info
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 813, in setup_info
    self.setup_info = self.get_setup_info()
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 802, in get_setup_info
    setup_info = SetupInfo.from_ireq(self.ireq, subdir=self.subdirectory)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1882, in from_ireq
    ireq.source_dir, subdirectory=subdir, ireq=ireq, kwargs=kwargs, stack=stack
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1918, in create
    created.get_initial_info()
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1747, in get_initial_info
    self.update_from_dict(self.parse_setup_py())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1504, in parse_setup_py
    parsed = ast_parse_setup_py(self.setup_py.as_posix())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1202, in ast_parse_setup_py
    function_names = ast_analyzer.parse_functions()
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 962, in parse_functions
    retries = self.parse_function_names(function_map=self.function_map)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 958, in parse_function_names
    self.resolved_function_names[fn_name] = ast_unparse(v, analyzer=self)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1153, in ast_unparse
    unparsed[k] = unparse(v)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1040, in ast_unparse
    items = unparse(analyzer.assignments[assignment])
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1140, in ast_unparse
    val = unparse(item.value, recurse=False)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1004, in ast_unparse
    unparsed = unparsed[unparse(item.slice.value)]
TypeError: 'Attribute' object is not subscriptable

Steps to replicate

I think this is the simplest way I can reproduce it. With a setup.py file of:

from setuptools import setup, find_packages
import os

if 'PACKAGE_VERSION' in os.environ:
    my_version = os.environ['PACKAGE_VERSION']
else:
    my_version = '0.0.1'

setup(
    name='pipenv-unparse',
    version=my_version,
    install_requires=[
        'flask==1.1.1'
    ],
    python_requires='~=3.6',
    packages=find_packages('.')
)

And a Pipfile of:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pipenv-unparse = {editable = true, path = "."}

[requires]
python_version = "3.7"

Running the command pipfile lock fails with the described error.

Notes:

using the value directly in the keyword argument (version=os.environ['PACKAGE_VERSION']) works, but would obviously fail if run without the environment variable defined
using os.environ.get('PACKAGE_VERSION', '0.0.1') works as well
python is installed using homebrew; pipenv is installed using python3 -m pip install pipenv

$ pipenv --support

Pipenv version: '2020.5.28'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/opt/python/bin/python3.7'

Python installations found:

3.7.7: /usr/local/bin/python3
3.7.7: /usr/local/bin/python3.7m
3.7.7: /usr/local/bin/python3.7
3.7.3: /usr/bin/python3
2.7.16: /usr/bin/python2
2.7.16: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.7',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '19.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 19.5.0: Thu Apr 30 18:25:59 PDT '
                     '2020; root:xnu-6153.121.1~7/RELEASE_X86_64',
 'python_full_version': '3.7.7',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

TERM_SESSION_ID
SSH_AUTH_SOCK
LC_TERMINAL_VERSION
COLORFGBG
ITERM_PROFILE
XPC_FLAGS
PWD
SHELL
LC_CTYPE
TERM_PROGRAM_VERSION
TERM_PROGRAM
PATH
DISPLAY
LC_TERMINAL
COLORTERM
TERM
HOME
TMPDIR
USER
XPC_SERVICE_NAME
LOGNAME
ITERM_SESSION_ID
__CF_USER_TEXT_ENCODING
SHLVL
OLDPWD
P9K_TTY
ZSH
LESS
PAGER
LSCOLORS
P9K_SSH
LC_ALL
LANG
_
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /Users/pes/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin
SHELL: /bin/zsh
LANG: en_US.UTF-8
PWD: /Users/pes/work/Projects/CUE/work/pipenv-unparse

Contents of Pipfile ('/Users/pes/work/Projects/CUE/work/pipenv-unparse/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pipenv-unparse = {editable = true, path = "."}

[requires]
python_version = "3.7"

Contents of Pipfile.lock ('/Users/pes/work/Projects/CUE/work/pipenv-unparse/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "aea1c7e6f5b86dd435a104a000cfad6d7c5664098cb0812ddd89ce3e20070f46"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "click": {
            "hashes": [
                "sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a",
                "sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==7.1.2"
        },
        "flask": {
            "hashes": [
                "sha256:13f9f196f330c7c2c5d7a5cf91af894110ca0215ac051b5844701f2bfd934d52",
                "sha256:45eb5a6fd193d6cf7e0cf5d8a5b31f83d5faae0293695626f539a823e93b13f6"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==1.1.1"
        },
        "itsdangerous": {
            "hashes": [
                "sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
                "sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.1.0"
        },
        "jinja2": {
            "hashes": [
                "sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0",
                "sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==2.11.2"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
                "sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
                "sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
                "sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
                "sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
                "sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
                "sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
                "sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
                "sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
                "sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
                "sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
                "sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b",
                "sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
                "sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
                "sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
                "sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
                "sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
                "sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
                "sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
                "sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
                "sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
                "sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
                "sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
                "sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
                "sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
                "sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
                "sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
                "sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
                "sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
                "sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
                "sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
                "sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
                "sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
            "version": "==1.1.1"
        },
        "pipenv-unparse": {
            "editable": true,
            "path": "."
        },
        "werkzeug": {
            "hashes": [
                "sha256:2de2a5db0baeae7b2d2664949077c2ac63fbd16d98da0ff71837f7d1dea3fd43",
                "sha256:6c80b1e5ad3665290ea39320b91e1be1e0d5f60652b964a3070216de83d2e47c"
            ],
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
            "version": "==1.0.1"
        }
    },
    "develop": {}
}

The text was updated successfully, but these errors were encountered:

techalchemy · 2020-05-28T16:54:47Z

sorry for the issue! I can see why this would cause problems. Thanks for reporting!

triage-new-issues bot added the triage label May 28, 2020

techalchemy added Type: Bug 🐛 This issue is a bug. Type: Regression This issue is a regression of a previous behavior. Type: Vendored Dependencies This issue affects vendored dependencies within pipenv. labels May 28, 2020

triage-new-issues bot removed triage labels May 28, 2020

frostming added this to the 2020.6.x bugfix release milestone Jun 1, 2020

frostming mentioned this issue Jun 1, 2020

Update vendor libraries #4302

Merged

2 tasks

techalchemy closed this as completed in #4302 Jun 1, 2020

dependabot bot mentioned this issue Mar 14, 2021

Bump pipenv from 2018.11.26 to 2020.11.15 in /python/helpers codisart/dependabot-core#1

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Locking may fail while unparsing setup.py if version is read from environment #4274

Locking may fail while unparsing setup.py if version is read from environment #4274

pesstibo commented May 28, 2020

techalchemy commented May 28, 2020

Locking may fail while unparsing setup.py if version is read from environment #4274

Locking may fail while unparsing setup.py if version is read from environment #4274

Comments

pesstibo commented May 28, 2020

Issue description

Expected result

Actual result

Steps to replicate

techalchemy commented May 28, 2020