Skip to content

Commit

Permalink
Remove mentions to .egg and stop producing them from kedro package (
Browse files Browse the repository at this point in the history
#2568)

* Stop producing `.egg` files from `kedro package`

Fix gh-2273.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Upgrade import-linter

Fix gh-2570.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Partially remove reliance on setup.py for micropackaging

Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert "Partially remove reliance on setup.py for micropackaging"

This reverts commit fd3efa8.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
  • Loading branch information
astrojuanlu authored May 11, 2023
1 parent 37b1d17 commit 891771d
Show file tree
Hide file tree
Showing 7 changed files with 16 additions and 46 deletions.
1 change: 1 addition & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
## Major features and improvements
## Bug fixes and other changes
## Breaking changes to the API
* `kedro package` does not produce `.egg` files anymore, and now relies exclusively on `.whl` files.
## Upcoming deprecations for Kedro 0.19.0


Expand Down
1 change: 1 addition & 0 deletions dependency/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
anyconfig~=0.10.0
attrs>=21.3
build
cachetools~=5.3
click<9.0
cookiecutter>=2.1.1, <3.0
Expand Down
12 changes: 3 additions & 9 deletions docs/source/deployment/single_machine.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,24 +47,18 @@ Run the following in your project’s root directory:
kedro package
```

Kedro builds the package into the `dist/` folder of your project, and creates one `.egg` file and one `.whl` file, which are [Python packaging formats for binary distribution](https://packaging.python.org/overview/).
Kedro builds the package into the `dist/` folder of your project, and creates a `.whl` file, which is [a Python packaging format for binary distribution](https://packaging.python.org/overview/).

The resulting `.egg` and `.whl` packages only contain the Python source code of your Kedro pipeline, not any of the `conf/` and `data/` subfolders nor the `pyproject.toml` file.
The resulting `.whl` package only contains the Python source code of your Kedro pipeline, not any of the `conf/` and `data/` subfolders nor the `pyproject.toml` file.
The project configuration is packaged separately in a `tar.gz` file. This compressed version of the config files excludes any files inside your `local` directory.
This means that you can distribute the project to run elsewhere, such as on a separate computer with different configuration, data and logging. When distributed, the packaged project must be run from within a directory that contains the `pyproject.toml` file and `conf/` subfolder (and `data/` if your pipeline loads/saves local data). This means that you will have to create these directories on the remote servers manually.

Recipients of the `.egg` and `.whl` files need to have Python and `pip` set up on their machines, but do not need to have Kedro installed. The project is installed to the root of a folder with the relevant `conf/` and `data/` subfolders, by navigating to the root and calling:
Recipients of the `.whl` file need to have Python and `pip` set up on their machines, but do not need to have Kedro installed. The project is installed to the root of a folder with the relevant `conf/` and `data/` subfolders, by navigating to the root and calling:

```console
pip install <path-to-wheel-file>
```

Or when using the .egg file:

```console
easy_install <path-to-egg-file>
```

After having installed your project on the remote server, run the Kedro project as follows from the root of the project:

```console
Expand Down
2 changes: 1 addition & 1 deletion docs/source/development/commands_reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ A parameterised run is best used for dynamic parameters, i.e. running the same p

### Deploy the project

The following packages your application as one `.egg` file and one `.whl` file within the `dist/` folder of your project. It packages the project configuration separately in a `tar.gz` file:
The following packages your application as one `.whl` file within the `dist/` folder of your project. It packages the project configuration separately in a `tar.gz` file:

```bash
kedro package
Expand Down
23 changes: 5 additions & 18 deletions kedro/framework/cli/project.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,28 +148,15 @@ def ipython(
@project_group.command()
@click.pass_obj # this will pass the metadata as first argument
def package(metadata: ProjectMetadata):
"""Package the project as a Python egg and wheel."""
"""Package the project as a Python wheel."""
source_path = metadata.source_dir
call(
[
sys.executable,
"setup.py",
"clean",
"--all",
"bdist_egg",
"--dist-dir",
"../dist",
],
cwd=str(source_path),
)
call(
[
sys.executable,
"setup.py",
"clean",
"--all",
"bdist_wheel",
"--dist-dir",
"-m",
"build",
"--wheel",
"--outdir",
"../dist",
],
cwd=str(source_path),
Expand Down
2 changes: 1 addition & 1 deletion test_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ gcsfs>=2023.1, <2023.3; python_version >= '3.8'
geopandas>=0.6.0, <1.0
hdfs>=2.5.8, <3.0
holoviews~=1.13.0
import-linter[toml]==1.2.6
import-linter[toml]==1.8.0
ipython>=7.31.1, <8.0; python_version < '3.8'
ipython~=8.10; python_version >= '3.8'
isort~=5.0
Expand Down
21 changes: 4 additions & 17 deletions tests/framework/cli/test_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -296,23 +296,10 @@ def test_happy_path(
mocker.call(
[
sys.executable,
"setup.py",
"clean",
"--all",
"bdist_egg",
"--dist-dir",
"../dist",
],
cwd=str(fake_repo_path / "src"),
),
mocker.call(
[
sys.executable,
"setup.py",
"clean",
"--all",
"bdist_wheel",
"--dist-dir",
"-m",
"build",
"--wheel",
"--outdir",
"../dist",
],
cwd=str(fake_repo_path / "src"),
Expand Down

0 comments on commit 891771d

Please sign in to comment.