pip-sync fails if virtualenv is not activated

[chaoflow]

./bin/pip-sync req.txt should work but fails with:

Traceback (most recent call last):
  File "/opt/bagbunker-venv/bin/pip-sync", line 11, in <module>
    sys.exit(cli())
  File "/opt/bagbunker-venv/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/opt/bagbunker-venv/local/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/opt/bagbunker-venv/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/bagbunker-venv/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/opt/bagbunker-venv/local/lib/python2.7/site-packages/piptools/scripts/sync.py", line 75, in cli
    pip_flags=pip_flags))
  File "/opt/bagbunker-venv/local/lib/python2.7/site-packages/piptools/sync.py", line 137, in sync
    check_call(['pip', 'uninstall', '-y'] + pip_flags + sorted(to_uninstall))
  File "/usr/lib/python2.7/subprocess.py", line 535, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 522, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 13] Permission denied

source ./bin/activate && pip-sync req.txt works as expected.

[nvie]

pip-sync invokes pip, which will resolve to whatever which pip is on your system. In this case, since the env is not activated, this will likely be /usr/local/bin/pip (global installation), not your env. Before 1.4.1, the behaviour was to use the local pip, but this led to more confusion as people who would install pip-tools system-wide would see their system wide packages installed/uninstalled, which is more harmful than this. I guess we can't have the cake and eat it too :)

[nvie]

Closing this now. If this was not the case, feel free to reopen it.

[chaoflow]

This feels confusing to me: pip-tools and pip belong closely together and should resemble each other's command line experience. pip seems not to support being installed globally while working locally, the --user option being an exception. Please correct me if wrong, I only briefly scanned pip install --help. If pip does not support this mode of operation, I think pip-tools should neither.

Instead pip-tools should always use the pip right next to it, instead of consulting $PATH. If it is installed globally already, it is cheap to install it also locally and by doing so you would have two pip-sync, one always operating globally and one locally.

pip (and easy_install, and python and all other python tools I experienced so far) behave identical, independent of whether the virtualenv is activated or not.

I do not have the power to reopen here, but feel it should be reopened ;)

[nvie]

All I can say here is that I'm very torn on the subject. Your argument is perfectly valid, and I can see why it confuses you. On the other hand, the argument against inadvertently removing system-wide installed packages is also valid: if users did not install pip-tools in their local envs, either by accident or because they assumed they could use a globally installed pip-sync. So I don't know a good solution here that works for everyone. That's why I'm erring on the safe side with this.

[chaoflow]

I think I found a solution:

activated virtualenv: pip-sync has to alter the activated virtualenv, independent of where it is installed. $PATH is consulted to find pip (#203).

deactivated virtualenv: pip-sync has to alter the virtualenv it is installed to (Current issue).

Updated: Currently, if no virtualenv is activated it consults $PATH. That I consider a bug that could be fixed by adjusting behaviour depending on whether we are in an activated virtualenv or not. ($VIRTUAL_ENV could be used to detect an activated virtualenv)

[nvie]

I think this is a good approach indeed! Do you feel like implementing a PR for this, @chaoflow?

[chaoflow]

@nvie I'll look into it.

An even simpler solution could be, if pip-sync is called as pip-sync it will consult PATH, as it was also found via PATH, if it is called with an absolute or relative path ./foo/bar/bin/pip-sync it will look for pip right next to it.

What do you think?

[chaoflow]

That would also mean, that within an activated virtualenv one would alter globally if calling /usr/bin/pip-sync. Is that desirable behaviour or not?