Six issue when installing package

[securitybites]

When trying to install awscli I get the following error.

admins-MacBook-Pro-5:~ user$ pip install awscli
Collecting awscli
  Using cached awscli-1.8.9-py2.py3-none-any.whl
Collecting botocore==1.2.7 (from awscli)
  Using cached botocore-1.2.7-py2.py3-none-any.whl
Collecting rsa<=3.3.0,>=3.1.2 (from awscli)
  Using cached rsa-3.2-py2.py3-none-any.whl
Collecting colorama<=0.3.3,>=0.2.5 (from awscli)
Collecting docutils>=0.10 (from awscli)
Requirement already satisfied (use --upgrade to upgrade): jmespath<1.0.0,>=0.7.1 in /Library/Python/2.7/site-packages (from botocore==1.2.7->awscli)
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.2.7->awscli)
  Using cached python_dateutil-2.4.2-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa<=3.3.0,>=3.1.2->awscli)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.2.7->awscli)
  Using cached six-1.9.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, docutils, botocore, pyasn1, rsa, colorama, awscli
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/6t/h404bjcd5tb_4q86tpv_251rv_0h0j/T/pip-sYsqDS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

Other user have mentioned the issue and point to pip:

aws/aws-cli#1522

https://bitbucket.org/gutworth/six/issues/134/uninstall-error-operation-not-permitted

[dstufft]

You're on El-Capitan right?

[moeriki]

I have this issue when installing AWS-SDK, which depends on six.

On El-Capitan indeed.

[dstufft]

This is because OS X El Capitan ships with six 1.4.1 installed already and when it attempts to uninstall it (because awscli depends on botocore, botocore depends on python-dateutil, and python-dateutil depends on six >= 1.5) it doesn't have permission to do so because System Integrity Protection doesn't allow even root to modify those directories.

Ideally, pip should just skip uninstalling those items since they aren't installed to site-packages they are installed to a special Apple directory. However, even if pip skips uninstalling those items and installs six into site-packages we'll hit another bug where Apple puts their pre-installed stuff earlier in the sys.path than site-packages. I've talked to Apple about this and I'm not sure if they're going to do anything about it or not.

[arubdesu]

This affects many other packages which rely on Six, pardon my obliviousness, but is there any way to tell pip to either 1. install an updated version to /Library's site-packages and be pointed to prefer that one (instead of saying 'requirement satisfied! and exiting) or 2. ignore the version found in /System on 10.11+?

[dstufft]

pip install --ignore-installed six

[arubdesu]

$ sudo -H pip install --ignore-installed six
Collecting six
  Using cached six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
Successfully installed six-1.4.1

Sure, but pip won't then prefer that upgraded version of Six when performing sudo installs:

sudo -H pip install mock
Requirement already satisfied (use --upgrade to upgrade): mock in /Library/Python/2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): funcsigs in /Library/Python/2.7/site-packages (from mock)
Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /Library/Python/2.7/site-packages (from mock)
Collecting six>=1.7 (from mock)
  Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-sy_6uC-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

[arubdesu]

...
Sorry about that, for some reason it's working after... running it a second time? Needed to --ignore-installed mock as well.

[arubdesu]

Ah, so this is less a pip issue to address (although it seems like --ignore-installed is going to get very popular) than the user changing their path appropriately. (You'll just get all the support requests. Thanks, Apple!)
As per http://peak.telecommunity.com/DevCenter/setuptools#development-mode you can set a .pth file system-wide to make sure that directory is consulted first. I'll compose a radar to Apple so they can remove their heads from their butts about this.

[arubdesu]

...Shouldn't there be some other, more permanent fix? easy_install has a .pth file, so there is some precedent here. I've documented what I think is the best near-term solution here: http://apple.stackexchange.com/a/210021/216

[reganjohnson]

THANKS! The following allowed it to install correctly.

pip install awsebcli --upgrade --ignore-installed six

Running on OSX El Capitan

$ eb --version
EB CLI 3.5.4 (Python 2.7.1)

[dstufft]

@arubdesu I'm hoping the more permanent fix will be to have Apple just fix this, which I am talking to them about.

[propupul]

Hello all!
I came across the same problem(thank goodness for this post!) while trying to install virtualenvwrapper. As all of you, and lastly @reganjohnson suggested --upgrade --ignore-installed six will work.
I did
sudo pip install virtualenvwrapper --upgrade --ignore-installed six

Running El Capitan as well

P.S: Please forgive me if I'm not posting this correctly I'm a complete noob to github :(

[nalindak]

sudo pip install virtualenvwrapper --upgrade --ignore-installed six

Did work for me! thanks guys

[mattdeboard]

You can disable System Integrity Protection completely via running csrutil disable in recovery mode. Do at your own risk etc. etc.

[arubdesu]

Anyone can respond to this issue however they want, (playing chicken with Apple over this is like waiting for an updated version of rsync, in my opinion...) but if you're coming to this thread now, just know that you can work around this issue by:

1. installing with pip to your user folder *
2. installing with pip and --ignoring-installed *
3. using a virtualenv
4. installing your own python, e.g. with homebrew
   And after all of those options? Sure, you can TEMPORARILY disable SIP during the time you pip upgrade something in the System folder, but then you're also messing with the version Apple expects to be present. You can even "at your own risk etc. etc." disable, reboot, then enable --without fs to permanently stop just the filesystem protections. But I don't think you should get used to employing SIP-related fixes for python package installs, if anything the SIP-disabling options will go away, not get more lenient, so dealing with this new situation sooner rather than later seems wise.
   * also then possibly requiring you to employ a .pth file to ensure python picks up on your desired lookup order, as previously mentioned about sys.path

[mattdeboard]

Just a suggestion friend, no need to get alarmed.

[ftiff]

I agree with @arubdesu
Messing with system has never been a good idea.
Just use any of the other possibilities. And it's easier.

[arubdesu]

And I agree with @dstufft that Apple should hard-code any of their python package dependencies to the System folder and allow the default path to check /Library before /System. (Not that I'm putting words in his mouth, just saying that's what I filed a radar for. http://www.openradar.me/radar?id=6192110889861120)

[monte-hayward]

both of these result in six version 1.41:
pip install --ignore-installed six
pip install awsebcli --upgrade --ignore-installed six
El Capitan must go down with the ship...

[dstufft]

@monte-hayward Try pip install --user awsebcli (might need a --upgrade flag too).

[ftiff]

Hi folks, FYI:
Engineering has determined that your bug report (23234649) is a duplicate of another issue (20471644) and will be closed.

If you fill a bug report please mention Radar # 20471644

[DogDogGoose]

I found the easiest solution was to use homebrew to install a local version of python (thus pip). This avoids the complexities of El Capitan's protected install of python and instead installs python into /usr/local/bin. The various solutions for the "--ignore-installed six" did not work on my system.

Just do:

brew install python

The pip install command should work fine after that.

More information is available here:
http://apple.stackexchange.com/questions/209572/how-to-use-pip-after-the-el-capitan-max-os-x-upgrade

[owendall]

Thanks much for this:
$ sudo pip install awscli --upgrade --ignore-installed six

[othyn]

Yep, still an issue in High Sierra, 10.13.1.

Solved with: sudo pip install awscli --upgrade --ignore-installed six

[ileend]

to install pytest on mac with el capitan, this worked:

sudo pip install pytest --upgrade --ignore-installed six

"Installing collected packages: py, attrs, pluggy, funcsigs, six, setuptools, pytest
Running setup.py install for pluggy ... done
Successfully installed attrs-17.3.0 funcsigs-1.0.2 pluggy-0.6.0 py-1.5.2 pytest-3.3.1 setuptools-38.2.4 six-1.11.0"
then to verify installation:

pytest --version

"This is pytest version 3.3.1, imported from /Library/Python/2.7/site-packages/pytest.pyc"

[ShatishPandey]

Fixed installing IPython on MacOS High Sierra:

For me, using 'pip3' instead of 'pip' worked. Probable reason was pip was trying to deal with the default python installation provided by Mac. However I have installed Python3 using brew. So the following worked for me:

$ sudo pip3 install ipython[All]

[Ivoz]

@ShatishPandey to use pip with a specific python, you can always incant:

$ {mypython} -m pip (install/list/etc...)

Where {mypython} is python3 or /usr/bin/local/python2.5 or whatever else.

[davidwjx]

@dstufft Thaaanks.

pip install --user jupyter

works fine in High Sierra, 10.13.3.

[codedogfish]

Just do not use the system python, let pyenv save your world.

[rupav]

@codedogfish can you explain, I am facing similar issue on travis build, but activating virtualenv not doing anything different on the build. Also its not possible to use --ignore-installed command for all packages. I was getting similar error for other package too, so I tihnk --ignore-installed is needed too.
Similar issue raised in:
https://stackoverflow.com/questions/49932759/pip-10-and-apt-how-to-avoid-cannot-uninstall-x-errors-for-distutils-packages

[cod3fran]

pip install --ignore-installed six

thanks !!

[arjangch]

I have the same problem in Mac Mojave right now.
pip install --ignore-installed six didn't help either