-
Notifications
You must be signed in to change notification settings - Fork 135
installationAdvanced
This page is for when the recommended methods of installation do not work. Also see Troubleshooting. Note that these are not regularly tested, so they probably will require some modification for the current version of RAVEN. Miniconda or PIP can be used to install the raven libraries and hints for using them are below. Easybuild has also been used, but is only recommended if you are already using Easybuild for other software.
This is the standard changes needed for using a pip only install. (Note that swig needs to be installed (either from a package manager or from https://www.swig.org/ )
Changed the establish_conda_env.sh install command
./scripts/establish_conda_env.sh --install --installation-manager PIP
Build raven as usual:
./build_raven
Add --skip-conda when running tests and with raven_framework
/run_tests --skip-conda
On supported Python versions, RAVEN can experimentally be installed with pip:
pip install raven-framework
Also the plugins HERON and TEAL (as heron-ravenframework and teal-ravenframework) are available.
Which version are supported can be seen by checking https://pypi.org/project/raven-framework/#files to see which we have provided files for.
RAVEN includes a script (that is used by establish_conda_env.sh) to find the commands needed for installation.
For example, to find the commands for conda, the following can be used:
python3 ./scripts/library_handler.py conda --action install
python3 ./scripts/library_handler.py conda --action install --subset pip
example output:
conda install --name raven_libraries -y -c conda-forge h5py=2.10 numpy=1.18 scipy=1.2 scikit-learn=0.24 pandas=1.1 xarray=0.16 netcdf4=1.5 matplotlib=3.2 statsmodels cloudpickle=1.6 tensorflow=2.0 python=3 hdf5 swig pylint coverage lxml psutil pip importlib_metadata pyside2 nomkl numexpr imageio=2.9 setuptools numpy-financial dill pyomo pyutilib glpk ipopt coincbc
pip install ray[default]==1.9.* xmlschema
Then, the conda and the pip install commands can be run manually.
For pip, the command can be found with:
python3 ./scripts/library_handler.py pip --action install
with example output:
pip3 install h5py==2.10.* numpy==1.18.* scipy==1.2.* scikit-learn==0.24.* pandas==1.1.* xarray==0.16.* netcdf4==1.5.* matplotlib==3.2.* statsmodels cloudpickle==1.6.* tensorflow==2.0.* pylint coverage lxml psutil importlib_metadata pyside2 ray[default]==1.9.* imageio==2.9.* xmlschema setuptools numpy-financial dill pyomo pyutilib
Note that creating pip wheels is experimental. A setup.cfg file can be created by running:
python3 ./scripts/library_handler.py pip --action=setup.cfg > setup.cfg
After that a pip wheel can be created:
pip3 --verbose wheel .
The dependencies are stored in the file dependencies.xml This can be modified to change the dependencies used (however, this may result in failures in RAVEN.
For example, to remove the requirement for a specific scipy version, the line:
<scipy>1.2</scipy>
could be changed to:
<scipy/>
The pip commands can be gotten from scripts/library_handler.py, and changed to just download on an internet connected computer, and then on the non-internet connected computer, the downloaded pip packages can be installed.
The following install instructions can be used to install with PIP in a virtual environment
BASE_DIR="$HOME/raven_libs"
INSTALL_DIR="$BASE_DIR/install"
VE_DIR="$BASE_DIR/ve"
mkdir -p $BASE_DIR
cd $BASE_DIR
wget https://downloads.sourceforge.net/project/swig/swig/swig-3.0.12/swig-3.0.12.tar.gz
tar -xvzf swig-3.0.12.tar.gz
cd swig-3.0.12/
./configure --prefix="$INSTALL_DIR"
make
make install
export PATH="$INSTALL_DIR/bin:$PATH"
cd
pip install --upgrade --target="$INSTALL_DIR" virtualenv
python "$INSTALL_DIR"/virtualenv.py "$VE_DIR"
source "$VE_DIR/bin/activate"
pip install numpy==1.11.0 h5py==2.6.0 scipy==0.17.1 scikit-learn==0.17.1 matplotlib==1.5.1
BASE_DIR="$HOME/raven_libs"
INSTALL_DIR="$BASE_DIR/install"
VE_DIR="$BASE_DIR/ve"
export PATH="$INSTALL_DIR/bin:$PATH"
source "$VE_DIR/bin/activate"
If the machine raven is being installed on can't get pip to install from the internet, pip can download the files on one machine and then install them on another. The below example is to install lazy-import.
mkdir pip_download
cd pip_download
pip download lazy-import
Then copy the files that pip downloads in to the pip_download directory:
cp -a orig_pip_download new_pip_download #or scp or whatever it takes to get them to the correct place
and then go to the raven_libraries environment and run:
conda activate raven_libraries
cd new_pip_download
pip install *
If you need a complete pip install, than run:
python scripts/library_handler.py pip --action install
to find out which libraries are needed, and then run something like (substitute in the libraries from library_handler.py)
mkdir pip_download
cd pip_download
pip download h5py==2.9.0 numpy==1.18.1 scipy==1.2.1 scikit-learn==0.21.2 pandas==0.24.2 xarray==0.12.1 netcdf4==1.4.2 matplotlib==3.1.1 statsmodels==0.9.0 cloudpickle==1.1.1 tensorflow==1.13.1 pylint coverage lxml psutil importlib_metadata pyside2 lazy-impor
Then do copy the files in the pip_download directory to the other computer, and use pip install on them.
This will probably need to be customized for different cluster, but these were used on one CentOS 7.2 cluster in 2017:
BASE_DIR="/opt/raven_libs"
INSTALL_DIR="$BASE_DIR/install"
VE_DIR="$BASE_DIR/ve"
mkdir -p $BASE_DIR
cd $BASE_DIR
wget https://downloads.sourceforge.net/project/swig/swig/swig-3.0.12/swig-3.0.12.tar.gz
tar -xvzf swig-3.0.12.tar.gz
cd swig-3.0.12/
./configure --prefix="$INSTALL_DIR"
make
make install
export PATH="$INSTALL_DIR/bin:$PATH"
cd $BASE_DIR
wget http://prdownloads.sourceforge.net/tcl/tcl8.6.6-src.tar.gz
tar -xvzf tcl8.6.6-src.tar.gz
cd tcl8.6.6/unix
./configure --prefix="$INSTALL_DIR"
make
make test
make install
cd $BASE_DIR
wget http://prdownloads.sourceforge.net/tcl/tk8.6.6-src.tar.gz
tar -xvzf tk8.6.6-src.tar.gz
cd tk8.6.6/unix/
./configure --prefix="$INSTALL_DIR"
make
make install
export LD_LIBRARY_PATH="$INSTALL_DIR"/lib:"$LD_LIBRARY_PATH"
cd $BASE_DIR
wget https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tgz
tar -xvzf Python-2.7.13.tgz
cd Python-2.7.13/
./configure --prefix="$INSTALL_DIR"
make -j16
make install
cd
pip install --upgrade --target="$INSTALL_DIR" virtualenv
python "$INSTALL_DIR"/virtualenv.py "$VE_DIR"
source "$VE_DIR/bin/activate"
pip install numpy==1.11.0 h5py==2.6.0 scipy==0.17.1 scikit-learn==0.17.1
pip install matplotlib
#Commands to run each time.
BASE_DIR="/opt/raven_libs"
INSTALL_DIR="$BASE_DIR/install"
VE_DIR="$BASE_DIR/ve"
export PATH="$INSTALL_DIR/bin:$PATH"
export LD_LIBRARY_PATH="$INSTALL_DIR"/lib:"$LD_LIBRARY_PATH"
source "$VE_DIR/bin/activate"