Skip to content

Commit

Permalink
fix
Browse files Browse the repository at this point in the history
Signed-off-by: Igoshev, Iaroslav <iaroslav.igoshev@intel.com>
  • Loading branch information
YarShev committed Nov 8, 2023
1 parent 6157b96 commit 5e1c617
Show file tree
Hide file tree
Showing 10 changed files with 25 additions and 23 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ If you want to install Modin with a specific engine, we recommend:
```bash
pip install "modin[ray]" # Install Modin dependencies and Ray.
pip install "modin[dask]" # Install Modin dependencies and Dask.
pip install "modin[unidist]" # Install Modin dependencies and Unidist.
pip install "modin[mpi]" # Install Modin dependencies and MPI through unidist.
```

Modin automatically detects which engine(s) you have installed and uses that for scheduling computation.
Expand All @@ -85,7 +85,7 @@ Each engine can also be installed individually (and also as a combination of sev
```bash
conda install -c conda-forge modin-ray # Install Modin dependencies and Ray.
conda install -c conda-forge modin-dask # Install Modin dependencies and Dask.
conda install -c conda-forge modin-unidist # Install Modin dependencies and Unidist.
conda install -c conda-forge modin-mpi # Install Modin dependencies and MPI through unidist.
conda install -c conda-forge modin-hdk # Install Modin dependencies and HDK.
```

Expand Down
9 changes: 5 additions & 4 deletions docs/development/architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -216,8 +216,8 @@ documentation page on :doc:`contributing </development/contributing>`.
- Uses the `Dask Futures`_ execution framework.
- The storage format is `pandas` and the in-memory partition type is a pandas DataFrame.
- For more information on the execution path, see the :doc:`pandas on Dask </flow/modin/core/execution/dask/implementations/pandas_on_dask/index>` page.
- :doc:`pandas on Unidist </development/using_pandas_on_unidist>`
- Uses the Unidist_ execution framework.
- :doc:`pandas on MPI </development/using_pandas_on_mpi>`
- Uses MPI_ through the Unidist_ execution framework.
- The storage format is `pandas` and the in-memory partition type is a pandas DataFrame.
- For more information on the execution path, see the :doc:`pandas on Unidist </flow/modin/core/execution/unidist/implementations/pandas_on_unidist/index>` page.
- :doc:`pandas on Python </development/using_pandas_on_python>`
Expand All @@ -228,8 +228,8 @@ documentation page on :doc:`contributing </development/contributing>`.
- Uses the Ray_ execution framework.
- The storage format is `pandas` and the in-memory partition type is a pandas DataFrame.
- For more information on the execution path, see the :doc:`experimental pandas on Ray </flow/modin/experimental/core/execution/ray/implementations/pandas_on_ray/index>` page.
- pandas on Unidist (experimental)
- Uses the Unidist_ execution framework.
- pandas on MPI (experimental)
- Uses MPI_ through the Unidist_ execution framework.
- The storage format is `pandas` and the in-memory partition type is a pandas DataFrame.
- For more information on the execution path, see the :doc:`experimental pandas on Unidist </flow/modin/experimental/core/execution/unidist/implementations/pandas_on_unidist/index>` page.
- pandas on Dask (experimental)
Expand Down Expand Up @@ -375,6 +375,7 @@ details. The documentation covers most modules, with more docs being added every
.. _Arrow tables: https://arrow.apache.org/docs/python/generated/pyarrow.Table.html
.. _Ray: https://github.com/ray-project/ray
.. _Unidist: https://github.com/modin-project/unidist
.. _MPI: https://www.mpi-forum.org/
.. _code: https://github.com/modin-project/modin/blob/master/modin/core/dataframe
.. _Dask: https://github.com/dask/dask
.. _Dask Futures: https://docs.dask.org/en/latest/futures.html
Expand Down
2 changes: 1 addition & 1 deletion docs/development/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Development
using_pandas_on_ray
using_pandas_on_dask
using_pandas_on_python
using_pandas_on_unidist
using_pandas_on_mpi
using_hdk
using_pyarrow_on_ray

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
pandas on Unidist
=================
pandas on MPI through unidist
=============================

This section describes usage related documents for the pandas on Unidist component of Modin.
This section describes usage related documents for the pandas on MPI through unidist component of Modin.

Modin uses pandas as a primary memory format of the underlying partitions and optimizes queries
ingested from the API layer in a specific way to this format. Thus, there is no need to care of choosing it
but you can explicitly specify it anyway as shown below.

One of the execution engines that Modin uses is Unidist. Currently, Modin only supports Unidist on MPI backend.
To enable the pandas on Unidist execution using MPI backend you should set the following environment variables:
One of the execution engines that Modin uses is MPI through unidist.
To enable the pandas on MPI through unidist execution you should set the following environment variables:

.. code-block:: bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ PandasOnUnidist Execution
Queries that perform data transformation, data ingress or data egress using the `pandas on Unidist` execution
pass through the Modin components detailed below.

To enable `pandas on Unidist` execution, please refer to the usage section in :doc:`pandas on Unidist </development/using_pandas_on_unidist>`.
To enable `pandas on MPI through unidist` execution, please refer to the usage section in :doc:`pandas on Unidist </development/using_pandas_on_mpi>`.

Data Transformation
'''''''''''''''''''
Expand Down
2 changes: 1 addition & 1 deletion docs/getting_started/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ and Modin will do computation with that engine:
pip install "modin[dask]" # Install Modin dependencies and Dask to run on Dask
export MODIN_ENGINE=dask # Modin will use Dask
pip install "modin[unidist]" # Install Modin dependencies and Unidist to run on Unidist.
pip install "modin[mpi]" # Install Modin dependencies and MPI to run on MPI through unidist.
export MODIN_ENGINE=unidist # Modin will use Unidist
export UNIDIST_BACKEND=mpi # Unidist will use MPI backend.
Expand Down
13 changes: 7 additions & 6 deletions docs/getting_started/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Modin can be used with :doc:`Ray</development/using_pandas_on_ray>`, :doc:`Dask<
pip install "modin[ray]" # Install Modin dependencies and Ray to run on Ray
pip install "modin[dask]" # Install Modin dependencies and Dask to run on Dask
pip install "modin[unidist]" # Install Modin dependencies and Unidist to run on Unidist
pip install "modin[mpi]" # Install Modin dependencies and Unidist to run on Unidist
pip install "modin[all]" # Install all of the above
Modin will automatically detect which engine you have installed and use that for
Expand Down Expand Up @@ -65,7 +65,7 @@ storage formats or for different functionalities of Modin. Here is a list of dep
.. code-block:: bash
pip install "modin[unidist]" # If you want to use the Unidist execution engine
pip install "modin[mpi]" # If you want to use the Unidist execution engine
Installing on Google Colab
"""""""""""""""""""""""""""
Expand Down Expand Up @@ -106,7 +106,7 @@ it is possible to install modin with chosen engine(s) alongside. Current options
+---------------------------------+---------------------------+-----------------------------+
| modin-ray | Ray_ | Linux, Windows |
+---------------------------------+---------------------------+-----------------------------+
| modin-unidist | Unidist_ | Linux, Windows, MacOS |
| modin-mpi | MPI_ through Unidist_ | Linux, Windows, MacOS |
+---------------------------------+---------------------------+-----------------------------+
| modin-hdk | HDK_ | Linux |
+---------------------------------+---------------------------+-----------------------------+
Expand All @@ -117,7 +117,7 @@ For installing Dask, Ray and Unidist engines into conda environment following co

.. code-block:: bash
conda install -c conda-forge modin-ray modin-dask modin-unidist
conda install -c conda-forge modin-ray modin-dask modin-mpi
All set of engines could be available in conda environment by specifying:

Expand All @@ -129,7 +129,7 @@ or explicitly:

.. code-block:: bash
conda install -c conda-forge modin-ray modin-dask modin-unidist modin-hdk
conda install -c conda-forge modin-ray modin-dask modin-mpi modin-hdk
``conda`` may be slow installing ``modin-all`` or combitations of execution engines so we currently recommend using libmamba solver for the installation process.
To do this install it in a base environment:
Expand Down Expand Up @@ -171,7 +171,7 @@ also use ``pip``.
This will install directly from the repo without you having to manually clone it! Please be aware
that these changes have not made it into a release and may not be completely stable.

If you would like to install Modin with a specific engine, you can use ``modin[ray]`` or ``modin[dask]`` or ``modin[unidist]`` instead of ``modin[all]`` in the command above.
If you would like to install Modin with a specific engine, you can use ``modin[ray]`` or ``modin[dask]`` or ``modin[mpi]`` instead of ``modin[all]`` in the command above.

Windows
-------
Expand Down Expand Up @@ -214,6 +214,7 @@ Once cloned, ``cd`` into the ``modin`` directory and use ``pip`` to install:
.. _WSL: https://docs.microsoft.com/en-us/windows/wsl/install-win10
.. _Ray: http://ray.readthedocs.io
.. _Dask: https://github.com/dask/dask
.. _MPI: https://www.mpi-forum.org/
.. _Unidist: https://github.com/modin-project/unidist
.. _HDK: https://github.com/intel-ai/hdk
.. _`Intel Distribution of Modin`: https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/distribution-of-modin.html#gs.86stqv
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ of the targets:
pip install "modin[ray]" # Install Modin dependencies and Ray to run on Ray
pip install "modin[dask]" # Install Modin dependencies and Dask to run on Dask
pip install "modin[unidist]" # Install Modin dependencies and Unidist to run on Unidist
pip install "modin[mpi]" # Install Modin dependencies and MPI to run on MPI through Unidist
pip install "modin[all]" # Install all of the above
Modin will automatically detect which engine you have installed and use that for
Expand Down
2 changes: 1 addition & 1 deletion examples/tutorial/jupyter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Currently we provide tutorial notebooks for the following execution backends:

- [PandasOnRay](https://modin.readthedocs.io/en/latest/development/using_pandas_on_ray.html)
- [PandasOnDask](https://modin.readthedocs.io/en/latest/development/using_pandas_on_dask.html)
- [PandasOnUnidist](https://modin.readthedocs.io/en/latest/development/using_pandas_on_unidist.html)
- [PandasOnMPI through unidist](https://modin.readthedocs.io/en/latest/development/using_pandas_on_mpi.html)
- [HdkOnNative](https://modin.readthedocs.io/en/latest/development/using_hdk.html)

## Creating a development environment
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
fsspec>=2022.05.0
jupyterlab
ipywidgets
modin[unidist]
modin[mpi]
modin[spreadsheet]

0 comments on commit 5e1c617

Please sign in to comment.