Known issue with virtualenv and ansible on SELinux-enabled hosts

[postmodern]

OS / Environment

Fedora Linux 25

Ansible version

2.2.1.0

Version of components from requirements.txt

Name: msrestazure
Version: 0.4.7
Summary: AutoRest swagger generator Python client runtime. Azure-specific module.
Home-page: https://github.com/Azure/msrestazure-for-python
Author: Microsoft Corporation
Author-email: UNKNOWN
License: MIT License
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: keyring, adal, msrest
---
Name: setuptools
Version: 34.3.3
Summary: Easily download, build, install, upgrade, and uninstall Python packages
Home-page: https://github.com/pypa/setuptools
Author: Python Packaging Authority
Author-email: distutils-sig@python.org
License: UNKNOWN
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: appdirs, packaging, six
---
Name: ansible
Version: 2.2.0.0
Summary: Radically simple IT automation
Home-page: http://ansible.com/
Author: Ansible, Inc.
Author-email: info@ansible.com
License: GPLv3
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: PyYAML, jinja2, paramiko, pycrypto, setuptools
---
Name: dopy
Version: 0.3.5
Summary: Python client for the Digital Ocean API
Home-page: https://github.com/devo-ps/dopy
Author: Vincent Viallet
Author-email: vincent@devo.ps
License: The MIT License (MIT)
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: requests
---
Name: boto
Version: 2.46.1
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: 
---
Name: boto3
Version: 1.4.4
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: UNKNOWN
License: Apache License 2.0
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: jmespath, botocore, s3transfer
---
Name: azure
Version: 2.0.0rc5
Summary: Microsoft Azure Client Libraries for Python
Home-page: https://github.com/Azure/azure-sdk-for-python
Author: Microsoft Corporation
Author-email: ptvshelp@microsoft.com
License: MIT License
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: azure-batch, azure-servicemanagement-legacy, azure-storage, azure-graphrbac, azure-mgmt, azure-servicebus
---
Name: msrest
Version: 0.4.1
Summary: AutoRest swagger generator Python client runtime.
Home-page: https://github.com/xingwu1/autorest/tree/python/ClientRuntimes/Python/msrest
Author: Microsoft Corporation
Author-email: UNKNOWN
License: MIT License
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: requests, certifi, chardet, requests-oauthlib, keyring, enum34, isodate
---
Name: apache-libcloud
Version: 1.5.0
Summary: A standard Python library that abstracts away differences among multiple cloud provider APIs. For more information and documentation, please see http://libcloud.apache.org
Home-page: http://libcloud.apache.org/
Author: Apache Software Foundation
Author-email: dev@libcloud.apache.org
License: Apache License (2.0)
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: 
---
Name: six
Version: 1.10.0
Summary: Python 2 and 3 compatibility utilities
Home-page: http://pypi.python.org/pypi/six/
Author: Benjamin Peterson
Author-email: benjamin@python.org
License: MIT
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: 
---
Name: pyOpenSSL
Version: 16.2.0
Summary: Python wrapper module around the OpenSSL library
Home-page: https://pyopenssl.readthedocs.io/
Author: Hynek Schlawack
Author-email: hs@ox.cx
License: Apache License, Version 2.0
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: cryptography, six
---
Name: Jinja2
Version: 2.8
Summary: A small but fast and easy to use stand-alone template engine written in pure python.
Home-page: http://jinja.pocoo.org/
Author: Armin Ronacher
Author-email: armin.ronacher@active-4.com
License: BSD
Location: /home/postmodern/algo/env/lib/python2.7/site-packages
Requires: MarkupSafe

Summary of the problem

There are some issues with virtualenv, ansible and python on SELinux-based systems. There are two workarounds:

A. set localhost ansible_python_interpreter=/usr/bin/python.
B. Install python dependencies with --user (ex: pip install --user -r requirements.txt) and rely on system python.

Steps to reproduce the behavior

1. python -m virtualenv env && source env/bin/activate && python -m pip install -r requirements.txt
2. ./algo

The way of deployment (cloud or local)

cloud

Expected behavior

Ansible finds the libselinux-python module that is installed into /usr/lib64/python2.7/site-packages/selinux/.

Actual behavior

Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!

Full log

  What provider would you like to use?
    1. DigitalOcean
    2. Amazon EC2
    3. Google Compute Engine
    4. Microsoft Azure
    5. Install to existing Ubuntu server

Enter the number of your desired provider
: 1

Enter your API token (https://cloud.digitalocean.com/settings/api/tokens):
[pasted values will not be displayed]
: 
Name the vpn server:
[algo.local]: test.vpn

  What region should the server be located in?
    1.  Amsterdam        (Datacenter 2)
    2.  Amsterdam        (Datacenter 3)
    3.  Frankfurt
    4.  London
    5.  New York         (Datacenter 1)
    6.  New York         (Datacenter 2)
    7.  New York         (Datacenter 3)
    8.  San Francisco    (Datacenter 1)
    9.  San Francisco    (Datacenter 2)
    10. Singapore
    11. Toronto
    12. Bangalore
Enter the number of your desired region:
[7]: 8

Do you want macOS/iOS clients to enable "VPN On Demand" when connected to cellular networks?
[y/N]: N

Do you want macOS/iOS clients to enable "VPN On Demand" when connected to Wi-Fi?
[y/N]: N

Do you want to install a DNS resolver on this VPN server, to block ads while surfing?
[y/N]: N

Do you want each user to have their own account for SSH tunneling?
[y/N]: N

Do you want to apply operating system security enhancements on the server? (warning: replaces your sshd_config)
[y/N]: y

Do you want the VPN to support Windows 10 clients? (requires RSA certificates and key exchange, less secure)
[y/N]: N

Do you want to retain the CA key? (required to add users in the future, but less secure)
[y/N]: N

PLAY [Configure the server] ****************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [Generate the SSH private key] ********************************************
ok: [localhost -> localhost]

TASK [Generate the SSH public key] *********************************************
ok: [localhost -> localhost]

TASK [Change mode for the SSH private key] *************************************
ok: [localhost -> localhost]

TASK [Ensure the dynamic inventory exists] *************************************
ok: [localhost]

TASK [cloud-digitalocean : Set the DigitalOcean Access Token fact] *************
ok: [localhost]

TASK [cloud-digitalocean : Delete the existing Algo SSH keys] ******************
FAILED - RETRYING: TASK: cloud-digitalocean : Delete the existing Algo SSH keys (10 retries left).
ok: [localhost]

TASK [cloud-digitalocean : Upload the SSH key] *********************************
changed: [localhost]

TASK [cloud-digitalocean : Creating a droplet...] ******************************
changed: [localhost]

TASK [cloud-digitalocean : Add the droplet to an inventory group] **************
changed: [localhost]

TASK [cloud-digitalocean : set_fact] *******************************************
ok: [localhost]

TASK [cloud-digitalocean : Tag the droplet] ************************************
changed: [localhost]

TASK [cloud-digitalocean : Get droplets] ***************************************
ok: [localhost]

TASK [cloud-digitalocean : Ensure the group digitalocean exists in the dynamic inventory file] ***
ok: [localhost]

TASK [cloud-digitalocean : Populate the dynamic inventory] *********************
ok: [localhost] => (item={u'status': u'active', u'kernel': None, u'volume_ids': [], u'locked': False, u'name': u'sfo.west.vpn', u'backup_ids': [], u'created_at': u'2017-04-05T05:20:10Z', u'snapshot_ids': [], u'size_slug': u'512mb', u'networks': {u'v4': [{u'type': u'public', u'netmask': u'255.255.240.0', u'ip_address': u'XXX.XXX.XXX.XXX', u'gateway': u'XXX.XXX.XXX.1'}], u'v6': [{u'type': u'public', u'netmask': 64, u'ip_address': u'XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX', u'gateway': u'XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:0001'}]}, u'next_backup_window': None, u'vcpus': 1, u'size': {u'price_monthly': 5.0, u'available': True, u'transfer': 1.0, u'price_hourly': 0.00744, u'regions': [u'ams1', u'ams2', u'ams3', u'blr1', u'fra1', u'lon1', u'nyc1', u'nyc2', u'nyc3', u'sfo1', u'sfo2', u'sgp1', u'tor1'], u'vcpus': 1, u'memory': 512, u'disk': 20, u'slug': u'512mb'}, u'image': {u'min_disk_size': 20, u'name': u'16.04.2 x64', u'created_at': u'2017-03-27T12:46:50Z', u'slug': u'ubuntu-16-04-x64', u'regions': [u'nyc1', u'sfo1', u'nyc2', u'ams2', u'sgp1', u'lon1', u'nyc3', u'ams3', u'fra1', u'tor1', u'sfo2', u'blr1'], u'id': 23754420, u'distribution': u'Ubuntu', u'type': u'snapshot', u'public': True, u'size_gigabytes': 0.33}, u'memory': 512, u'region': {u'available': True, u'sizes': [u'512mb', u'1gb', u'2gb', u'4gb', u'8gb', u'16gb'], u'slug': u'sfo2', u'name': u'San Francisco 2', u'features': [u'private_networking', u'backups', u'ipv6', u'metadata', u'install_agent', u'storage']}, u'disk': 20, u'id': 44817521, u'tags': [u'environment:algo'], u'features': [u'ipv6']})
failed: [localhost] (item={u'status': u'active', u'kernel': None, u'volume_ids': [], u'locked': False, u'name': u'test.vpn', u'backup_ids': [], u'created_at': u'2017-04-05T20:51:59Z', u'snapshot_ids': [], u'size_slug': u'512mb', u'networks': {u'v4': [{u'type': u'public', u'netmask': u'255.255.192.0', u'ip_address': u'XXX.XXX.XXX.XXX', u'gateway': u'XXX.XXX.XXX.1'}], u'v6': [{u'type': u'public', u'netmask': 64, u'ip_address': u'XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:A001', u'gateway': u'XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:0001'}]}, u'next_backup_window': None, u'vcpus': 1, u'size': {u'price_monthly': 5.0, u'available': True, u'transfer': 1.0, u'price_hourly': 0.00744, u'regions': [u'ams1', u'ams2', u'ams3', u'blr1', u'fra1', u'lon1', u'nyc1', u'nyc2', u'nyc3', u'sfo1', u'sfo2', u'sgp1', u'tor1'], u'vcpus': 1, u'memory': 512, u'disk': 20, u'slug': u'512mb'}, u'image': {u'min_disk_size': 20, u'name': u'16.04.2 x64', u'created_at': u'2017-03-27T12:46:50Z', u'slug': u'ubuntu-16-04-x64', u'regions': [u'nyc1', u'sfo1', u'nyc2', u'ams2', u'sgp1', u'lon1', u'nyc3', u'ams3', u'fra1', u'tor1', u'sfo2', u'blr1'], u'id': 23754420, u'distribution': u'Ubuntu', u'type': u'snapshot', u'public': True, u'size_gigabytes': 0.33}, u'memory': 512, u'region': {u'available': True, u'sizes': [u'512mb', u'1gb', u'2gb', u'4gb', u'8gb', u'16gb'], u'slug': u'sfo1', u'name': u'San Francisco 1', u'features': [u'private_networking', u'backups', u'ipv6', u'metadata', u'install_agent']}, u'disk': 20, u'id': 44873340, u'tags': [u'environment:algo'], u'features': [u'ipv6']}) => {"failed": true, "item": {"backup_ids": [], "created_at": "2017-04-05T20:51:59Z", "disk": 20, "features": ["ipv6"], "id": 44873340, "image": {"created_at": "2017-03-27T12:46:50Z", "distribution": "Ubuntu", "id": 23754420, "min_disk_size": 20, "name": "16.04.2 x64", "public": true, "regions": ["nyc1", "sfo1", "nyc2", "ams2", "sgp1", "lon1", "nyc3", "ams3", "fra1", "tor1", "sfo2", "blr1"], "size_gigabytes": 0.33, "slug": "ubuntu-16-04-x64", "type": "snapshot"}, "kernel": null, "locked": false, "memory": 512, "name": "test.vpn", "networks": {"v4": [{"gateway": "XXX.XXX.XXX.1", "ip_address": "XXX.XXX.XXX.XXX", "netmask": "255.255.192.0", "type": "public"}], "v6": [{"gateway": "XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:0001", "ip_address": "XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:A001", "netmask": 64, "type": "public"}]}, "next_backup_window": null, "region": {"available": true, "features": ["private_networking", "backups", "ipv6", "metadata", "install_agent"], "name": "San Francisco 1", "sizes": ["512mb", "1gb", "2gb", "4gb", "8gb", "16gb"], "slug": "sfo1"}, "size": {"available": true, "disk": 20, "memory": 512, "price_hourly": 0.00744, "price_monthly": 5.0, "regions": ["ams1", "ams2", "ams3", "blr1", "fra1", "lon1", "nyc1", "nyc2", "nyc3", "sfo1", "sfo2", "sgp1", "tor1"], "slug": "512mb", "transfer": 1.0, "vcpus": 1}, "size_slug": "512mb", "snapshot_ids": [], "status": "active", "tags": ["environment:algo"], "vcpus": 1, "volume_ids": []}, "msg": "Aborting, target uses selinux but python bindings (libselinux-python) aren't installed!"}

PLAY RECAP *********************************************************************
localhost                  : ok=14   changed=4    unreachable=0    failed=1   

[oleyka]

Another alternative is to create the virtualenv with system packages enabled:

virtualenv --system-site-packages env

[roboto84]

Thanks for the post people, I can confirm that the following workaround works for Fedora 25, Ansible 2.2.0.0, Python 2.7.13 ...

virtualenv --system-site-packages env

Thanks for the work. Thanks @oleyka

[sbocinec]

Another workaround is to set the ansible_python_interpreter variable to abolute path of the system installed python interpreter as mentioned in redhat-openstack/tripleo-quickstart@07d0e71

hosts: localhost
gather_facts: no
vars:
  ansible_python_interpreter: "/usr/bin/python"

[TheAtomicOption]

Another workaround if you don't want to change ansible_python_interpreter to the system python version (for example on CentOS 7 where the system version is 2.7.5 which can't handle SNI) is to make your version of python look at the site-packages for the system version.

This code appends a path to an environment variable that python uses when looking for modules, and only does so for the scope of the task which needs it.

tasks:
- name: get a url
  get_url:
        src: http://a_real_url.com
        dest: /some/place/nice
   environment: 
        PYTHONPATH: "/usr/lib64/python2.7/site-packages"

If you have an existing PYTHONPATH you'll want to add something like {{ lookup( 'env', 'PYTHONPATH')}}: to the setting to preserve the existing paths.

Warning: probably don't try importing site-packages from across major python versions. Some might work, but python3 and python2 packages aren't naturally compatible.

[PatWirth]

(Fedora release 26, ansible 2.5.2, python 2.7.14)

Solution A : Worked.

[local]
#localhost ansible_connection=local ansible_python_interpreter=python
localhost ansible_connection=local ansible_python_interpreter=/usr/bin/python

Solution B: Didn't work right away, this is what worked for me:

unzip master.zip
cd algo-master
vi config.cfg

python -m virtualenv algo-env && \
source algo-env/bin/activate && \
python -m pip install -r requirements.txt

virtualenv --system-site-packages algo-env
./algo

The "--system-site-packages" parameter worked for me too.

But the order seemed to be important. If I tried to put the system-site-packages in the initial setup or not at all then './algo' would fail with the ansible error about libselinux-python.