Install from PyPI: numpy not found

[DanielLenz]

Hi! This issue is related to the JOSS review (openjournals/joss-reviews#2663).

I tried to install the Python package from pip in a fresh venv and noticed that the requirements aren't installed as they should be.

~ pipenv install ReplicatedFocusingBeliefPropagation

Installing ReplicatedFocusingBeliefPropagation…
Error:  An error occurred while installing ReplicatedFocusingBeliefPropagation!
Error text: Collecting replicatedfocusingbeliefpropagation
  Downloading ReplicatedFocusingBeliefPropagation-1.0.1.tar.gz (417 kB)

    ERROR: Command errored out with exit status 1:
     command: /Users/lenzda/.local/share/virtualenvs/py-Ob9eXrK4/bin/python3.7m -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/4x/6yrptfcn4kxdynxfy3qv1g20lmvv2v/T/pip-install-uu4qb3_7/replicatedfocusingbeliefpropagation/setup.py'"'"'; __file__='"'"'/private/var/folders/4x/6yrptfcn4kxdynxfy3qv1g20lmvv2v/T/pip-install-uu4qb3_7/replicatedfocusingbeliefpropagation/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/4x/6yrptfcn4kxdynxfy3qv1g20lmvv2v/T/pip-pip-egg-info-vplpgjcz
         cwd: /private/var/folders/4x/6yrptfcn4kxdynxfy3qv1g20lmvv2v/T/pip-install-uu4qb3_7/replicatedfocusingbeliefpropagation/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/4x/6yrptfcn4kxdynxfy3qv1g20lmvv2v/T/pip-install-uu4qb3_7/replicatedfocusingbeliefpropagation/setup.py", line 8, in <module>
        import numpy as np
    ModuleNotFoundError: No module named 'numpy'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

This is likely caused by a bug in ReplicatedFocusingBeliefPropagation. Report this to its maintainers.
✘ Installation Failed

I checked your travis config and noticed that you had to implement some workarounds because of a setuptools issue (pypa/setuptools#2352). Could you use this to patch the pip install accordingly?

[Nico-Curti]

Hi @DanielLenz. Thanks for you comment.
The numpy and cython packages are required for the C++ compilation and thus they need to be installed beforehand.
We couldn't find any way to make pip install them before running the setup.py
Therefore we put the warning for the user in the documentation asking to install numpy and cython before running the pip command.
The same issue is not verified in the travis CI since we directly call the setup from the cloned repository.

If you have any suggestion about how overcome this issue please let us know.

[DanielLenz]

There's a couple of ways to address this I think, some are described here.

I'll go ahead and test this on my machine as soon as possible - I can highly relate to the frustration of trying to debug mac OS issues via travis, it just takes forever and you run into the most obscure errors...

[Nico-Curti]

Hi @DanielLenz,
We have found a possible solution for our installation issue here. Using a pyproject.toml file the pip installation should pre-install all the requirements (cython and numpy in our specific case). In the latest release we have added this file (and some comments in the documentation to warn the users). We hope we have fixed the issue in this way.

[DanielLenz]

Thanks for working on that and congratulations on the paper!

The install via pip in a fresh venv worked fine, but I ran into an issue when trying to test the package:

In [2]: from ReplicatedFocusingBeliefPropagation import ReplicatedFocusingBeliefPropagation as rFBP
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-2-1fd08364e9f5> in <module>
----> 1 from ReplicatedFocusingBeliefPropagation import ReplicatedFocusingBeliefPropagation as rFBP

~/.local/share/virtualenvs/rFBP-ecxdbiWG/lib/python3.7/site-packages/ReplicatedFocusingBeliefPropagation/__init__.py in <module>
      7 import sys
      8 import warnings
----> 9 from .__version__ import __version__
     10
     11 from .rfbp.ReplicatedFocusingBeliefPropagation import ReplicatedFocusingBeliefPropagation

~/.local/share/virtualenvs/rFBP-ecxdbiWG/lib/python3.7/site-packages/ReplicatedFocusingBeliefPropagation/__version__.py in <module>
     44   LOCAL = ''
     45
---> 46 VERSION = read_version(os.path.join(LOCAL, './CMakeLists.txt'))
     47
     48 __version__ = '.'.join(map(str, VERSION))

~/.local/share/virtualenvs/rFBP-ecxdbiWG/lib/python3.7/site-packages/ReplicatedFocusingBeliefPropagation/__version__.py in read_version(CMakeLists)
     25   revision = re.compile(r'set\s+\(RFBP_REVISION\s+(\d+)\)')
     26
---> 27   with open(CMakeLists, 'r') as fp:
     28     cmake = fp.read()
     29

FileNotFoundError: [Errno 2] No such file or directory: '/Users/lenzda/.local/share/virtualenvs/rFBP-ecxdbiWG/lib/python3.7/site-packages/ReplicatedFocusingBeliefPropagation/.././CMakeLists.txt'

[Nico-Curti]

Hi @DanielLenz
Thanks for your comment. We had many problems with the file location in the Pypi uploading. Since the problem seems to be related to the CMake file, in the latest commit (and also release) we have changed the version-detection.
It should work now.

[DanielLenz]

Thanks @Nico-Curti !

I now run into the following issue:

In [2]: import ReplicatedFocusingBeliefPropagation
/Users/lenzda/.local/share/virtualenvs/rFBP-ecxdbiWG/lib/python3.7/site-packages/ReplicatedFocusingBeliefPropagation/__init__.py:31: UserWarning: Atanherf file not found. It will be downloaded before continue
  warnings.warn('Atanherf file not found. It will be downloaded before continue')
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-1b130b189fd3> in <module>
----> 1 import ReplicatedFocusingBeliefPropagation

~/.local/share/virtualenvs/rFBP-ecxdbiWG/lib/python3.7/site-packages/ReplicatedFocusingBeliefPropagation/__init__.py in <module>
     33   sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'scripts'))
     34
---> 35   from download_atanherf import get_atanherf
     36
     37   get_atanherf('179OcORFBvcUGzGHBEVU8lUGLS70c1ZI_')

ModuleNotFoundError: No module named 'download_atanherf'

Have you considered also adding a pip build to the test matrix? That might help to discover and to address these issues - I'm sure most users will try to install via pip before installing from source. Cheers :)