fix installation using --target

[BrownTruck]

We encountered a problem on CentOS 7 machines when using the --target option
to install packages that are equipped with c-extensions.

Observe:

me@box ~ $ uname -a
Linux box.d.rz.is 3.10.0-327.4.4.el7.x86_64 #1 SMP Tue Jan 5 16:07:00 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

me@box ~ $ python --version
Python 2.7.5

me@box ~ $ virtualenv --version
1.10.1

me@box ~ $ virtualenv venv
New python executable in venv/bin/python
Installing Setuptools.......................................................................................................................................................done.
Installing Pip............................................................................................................................................................................................................................................................................done.

me@box ~ $ source venv/bin/activate

(venv)me@box ~ $ mkdir target

(venv)me@box ~ $ ls
target venv

(venv)me@box ~ $ pip install --index-url https://my_proxy:5000/dev/dev -U pip
Collecting pip
Downloading https://my_proxy:5000/root/pypi/+f/205/6f553d5b593d3/pip-8.0.2-py2.py3-none-any.whl (1.2MB)
100% |████████████████████████████████| 1.2MB 14.3MB/s
Installing collected packages: pip
Found existing installation: pip 1.4.1
Uninstalling pip-1.4.1:
Successfully uninstalled pip-1.4.1
Successfully installed pip-8.0.2

(venv)me@box ~ $ pip install --index-url https://my_proxy:5000/dev/dev --target target pyyaml
Collecting pyyaml
Downloading https://my_proxy:5000/root/pypi/+f/89c/bc92cda979042/PyYAML-3.11.zip (371kB)
100% |████████████████████████████████| 372kB 18.6MB/s
Installing collected packages: pyyaml
Running setup.py install for pyyaml ... done
Successfully installed pyyaml
Exception:
Traceback (most recent call last):
File "/data/home/me/venv/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/data/home/me/venv/lib/python2.7/site-packages/pip/commands/install.py", line 357, in run
for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmpz2LoDs/lib/python'

(venv)me@box ~ $ pip install --index-url https://my_proxy:5000/dev/dev --target target simplejson
Collecting simplejson
Downloading https://my_proxy:5000/root/pypi/+f/b84/41f1053edd9dc/simplejson-3.8.1.tar.gz (76kB)
100% |████████████████████████████████| 77kB 19.0MB/s
Installing collected packages: simplejson
Running setup.py install for simplejson ... done
Successfully installed simplejson
Exception:
Traceback (most recent call last):
File "/data/home/me/venv/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/data/home/me/venv/lib/python2.7/site-packages/pip/commands/install.py", line 357, in run
for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmpEFNOZq/lib/python'

(venv)me@box ~ $ pip install --index-url https://my_proxy:5000/dev/dev --target target markupsafe
Collecting markupsafe
Downloading https://my_proxy:5000/root/pypi/+f/f5a/b3deee4c37cd6/MarkupSafe-0.23.tar.gz
Installing collected packages: markupsafe
Running setup.py install for markupsafe ... done
Successfully installed markupsafe
Exception:
Traceback (most recent call last):
File "/data/home/me/venv/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
status = self.run(options, args)
File "/data/home/me/venv/lib/python2.7/site-packages/pip/commands/install.py", line 357, in run
for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmp4UuvDJ/lib/python'

(venv)me@box ~ $ ls /tmp/tmp4UuvDJ/
lib64

As you can see using --target first installs into a temporary directory and
then installes into the desired target. Well, at least it tries to: for any
packages that have c-extensions (in this case pyyaml, simplejson and
markupsafe) it incorrectly looks for a lib directory whereas it should be
looking for a lib64 directory. This patch fixes that for us by looking in a
fallback location. We can confirm that this fixes the issue for us.

Please advise.

---

This was automatically migrated from #3450 to reparent it to the master branch. Please see original pull request for any previous discussion.

Original Submitter: @esc

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will eligible for code review and hopefully merging!

[pradyunsg]

It seems to me that this can be closed... #3694 does fix this; correct?

[esc]

Yes, I suppose so.