Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use pyproject.toml instead of setup.py for kedro micropkg #2414

Closed
merelcht opened this issue Mar 13, 2023 · 1 comment · Fixed by #2761
Closed

Use pyproject.toml instead of setup.py for kedro micropkg #2414

merelcht opened this issue Mar 13, 2023 · 1 comment · Fixed by #2761
Assignees
@astrojuanlu
Labels
Issue: Feature Request New feature or improvement to existing feature
Milestone
Move to `pyprojec...

Comments

@merelcht
Copy link
Member

merelcht commented Mar 13, 2023 •
edited
Loading

Description

Inline with modernising the packaging/setup in Kedro to use pyproject.toml.

Context

Move along with the rest of the python community.

Possible Implementation

Ideally implement in a backwards compatible way to look for setup.py first and if that's not available use pyproject.toml in the 0.18.X series. Then in 0.19.0 remove support for the setup.py way. This is not as important as for kedro package.
@merelcht merelcht added the Issue: Feature Request New feature or improvement to existing feature label Mar 13, 2023
@astrojuanlu astrojuanlu mentioned this issue May 8, 2023
Closed
astrojuanlu added a commit that referenced this issue May 8, 2023
@astrojuanlu
Partially remove reliance on setup.py for micropackaging
ba1fa87 
Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 8, 2023
@astrojuanlu
Partially remove reliance on setup.py for micropackaging
7da5795 
Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 8, 2023
@astrojuanlu
Partially remove reliance on setup.py for micropackaging
346d8a9 
Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu mentioned this issue May 8, 2023
Merged
5 tasks
astrojuanlu added a commit that referenced this issue May 9, 2023
@astrojuanlu
Partially remove reliance on setup.py for micropackaging
fd3efa8 
Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 11, 2023
@astrojuanlu
Partially remove reliance on setup.py for micropackaging
0a5ac42 
Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 11, 2023
@astrojuanlu
Partially remove reliance on setup.py for micropackaging
04c5f35 
Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 11, 2023
@astrojuanlu
Remove mentions to .egg and stop producing them from kedro package (
891771d 
#2568)

* Stop producing `.egg` files from `kedro package`

Fix gh-2273.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Upgrade import-linter

Fix gh-2570.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Partially remove reliance on setup.py for micropackaging

Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert "Partially remove reliance on setup.py for micropackaging"

This reverts commit fd3efa8.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu mentioned this issue May 29, 2023
Closed
@astrojuanlu

This comment was marked as resolved.

Sign in to view
astrojuanlu added a commit that referenced this issue May 29, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
870ef01 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 29, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
c668556 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 29, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
4bb5216 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu mentioned this issue May 29, 2023
Merged
5 tasks
@astrojuanlu astrojuanlu self-assigned this May 29, 2023
@astrojuanlu astrojuanlu moved this to In Progress in Kedro Framework May 29, 2023
astrojuanlu added a commit that referenced this issue May 29, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
3b132c9 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 29, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
6e1784d 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue May 30, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
922503e 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu moved this from In Progress to In Review in Kedro Framework May 30, 2023
astrojuanlu added a commit that referenced this issue Jun 5, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
adbce1d 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue Jun 5, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
cdbfbc5 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue Jun 5, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
e88274f 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@yetudada yetudada moved this from In Review to In Progress in Kedro Framework Jun 6, 2023
noklam added a commit that referenced this issue Jun 7, 2023
@idanov @stichbury
@merelcht @jmholzer @astrojuanlu @noklam @antonymilne @ankatiyar @rashidakanchwala @AhdraMeraliQB @tomasvanpottelbergh @debugger24 @tynandebold @Huongg @ravi-kumar-pilla
[AUTO-MERGE] Merge main into develop via merge-main-to-develop (#2654)
ab1f7c8 
* Remove unnecessary files and subheaders in documentation (#2563)

Tidying up.

* Leverage PEP-585 (#2540)

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>

* [READY] Bring deployment docs up-to-date and add new pages for additional targets (#2557)

Ready at last. Still some conversation to be had about how we reflect "confidence" in the plugins we describe (what version they were last tested against) but the text is ready and better than what's there currently, so let's go!

* Remove mentions to `.egg` and stop producing them from `kedro package` (#2568)

* Stop producing `.egg` files from `kedro package`

Fix gh-2273.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Upgrade import-linter

Fix gh-2570.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Partially remove reliance on setup.py for micropackaging

Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert "Partially remove reliance on setup.py for micropackaging"

This reverts commit fd3efa8.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* OmegaConfigLoader returns `Dict` instead of `DictConfig`, resolves runtime_params properly (#2467)

* Fix typehint

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* test push

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* POC of fix to solve the runtime param resolution problem

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix OmegaConfigLoadaer - resolve runtime_params early

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Delegate the intialization of runtime_params to AbstractConfigLoader

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add test for interpolated value and globals

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* add more test and linting

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* refactor

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* update release note

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Apply comments and refactor the test

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update RELEASE.md

Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>

* Revert special logic handling DictConfig (#2582)

* Partially revert #2378 as OCL no longer returning DictConfig

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Revert #2176 with special logging handling

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* remove unused import

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Re-title and modify introduction to databricks testing guide in contribution section (#2579)

* modify databricks testing guide for contributors

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/contribution/development_for_databricks.md

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>

* Update development_for_databricks.md

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>

* Quick fix .gitpod.yml (#2585)

* `OmegaConfigLoader` should not show `MissingConfigException` when the file is empty (#2584)

* Allow empty config files

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Remove condition for empty config

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Revert "Remove condition for empty config"

This reverts commit 7961155.

* Update release notes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add unit test

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Proposal of updated settings.py (#2587)

* Proposal of updated settings.py

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Update settings after comment

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Collaborative Experiment Tracking Docs  (#2589)

* Add metadata attribute to `kedro.io` datasets (#2537)

* Add metadata attribute to kedro.io datasets

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update specs in pipeline hooks docstrings (#2598)

* Update pipeline hooks docstrings

Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com>

* Acknowledge community contribution

---------

Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Miscellaneous hooks improvements (#2596)

* Add debug level logging for hook execution order

Fix gh-1942.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Clarify role of setuptools in hooks

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Remove mentions to outdated and unused `pkg_resources`

Fix gh-2391.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Adapt example hook to spec signature

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Grammar and style fixes in docs

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert "Adapt example hook to spec signature"

This reverts commit ac1b495.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Minor modification to titles of deployment docs to replace "deprecated" with "legacy" (#2606)

* FIX: Typo in markdown for kedro notebooks docs (#2610)

* FIX: Typo in markdown for kedro notebooks docs

Signed-off-by: debugger24 <rahulcomp24@gmail.com>

* Acknowledge community contribution

---------

Signed-off-by: debugger24 <rahulcomp24@gmail.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add the Kedro RichHandler (#2592)

* Add the Kedro RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Clean up imports and comments

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Map arguments to add flexibility to customize arguments available in RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Clean up rich_logger.py and unused imports

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add docs - first iteration

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix imports and small refactoring

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Improve the doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update the project's template

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* format docstring

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Apply Jo's comment on the doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix the link with RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Apply suggestions from code review

Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>

* Fix broken doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* update template defaults

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* refactoring RichHandler and improve doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix broken test due to new default RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Move kedro.extras.logging.RichHandler -> kedro.logging.RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* add kedro/logging.py and remove the unnecessary import that cause namespace collision

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Refactor test

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add more tests

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix lots of import, docstring

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Reorder docstring

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix tests

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Link options to doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Clean up doc and unused logger

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix spelling `customise`

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Refactor test with fixture

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Remove unnecessary copy() because it is now a fixture

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Split up test_rich_traceback_configuration() to smaller tests

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add RELEASE.md

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix template logging

* Mention awesome-kedro in docs (#2620)

* Mention awesome-kedro in docs

See kedro-org/kedro-devrel#40

* Update CONTRIBUTING.md

* Spelling and style fixes

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update SnowparkTableDataSet docs (#2485)

* Update SnowparkTableDataSet docs

* Add missing snowpark dependencies for docs building

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Release 0.18.9 (#2619)

* Update __init__.py

* Update release note

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Update Version Numbers across docs

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Update RELEASE.md

* Update Release Note

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add documentation for deploying packaged Kedro projects on Databricks (#2595)

* Add deployment workflow page

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add table of contents, entry point guide, data and conf upload guide

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add detailed instructions for creating a job on Databricks

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add images and automated deployment resources

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove use of 'allows', add summary

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove link to missing image

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add deployment workflow to toctree

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Lint and fix missing link

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Minor style, syntax and grammar improvements

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fixes for correctness during validation

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add instructions for creating log output location

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Lint

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Lint databricks_run

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Minor wording change in reference to logs

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Modify reference to Pyspark-Iris

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Fix linter errors to enable docs build for inspection

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update build-docs.sh

* Fix broken link

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Remove spurious word

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add advantages subheading

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add alternative ways to upload data to DBFS

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Move note on unpackaged config and data

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix broken links

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Move databricks back into deployment section

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Remove references to PySpark Iris (pyspark-iris) starter

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Graphics links fixes, revise titles

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Fix broken internal link

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Fix links broken by new folder

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Remove logs directory

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update image of final job configuration

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add full stops in list.

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix conda environment name.

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Modify wording and image for creating a new job cluster

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Modify wording in guide to create new job cluster

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove --upgrade option

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add both ways of creating a new job

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

---------

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Update docs font family to Inter and swap the logo (#2611)

* Update docs font family to Inter

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Match weights to previous Google font

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update logo images; revert css formatting for easier PR review

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update text styles

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Add trailing whitespace (?)

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update Kedro image path

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update font size

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* update font to font-family: 'Inter', sans-serif;

Signed-off-by: huongg <huongg1409@gmail.com>

* testing with important for font-family

Signed-off-by: huongg <huongg1409@gmail.com>

* Make project name lowercase in docs

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* revert back to kedro/develop

Signed-off-by: huongg <huongg1409@gmail.com>

---------

Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Signed-off-by: huongg <huongg1409@gmail.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>
Co-authored-by: huongg <huongg1409@gmail.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Enable variable interpolation for the catalog with `OmegaConfigLoader` (#2621)

* Enable variable interpolation in catalog by escaping _

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>

* Add archive link on docs home page (#2636)

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* add ravi-kumar-pilla to tsc (#2629)

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add file for search console verification (#2631)

* add verification file and change to conf

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Fix conf.py so google file is moved to root of docs build

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Chore/update assets for docs (#2625)

* Update docs font family to Inter

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Match weights to previous Google font

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update logo images; revert css formatting for easier PR review

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update text styles

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Add trailing whitespace (?)

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update Kedro image path

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update font size

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* update font to font-family: 'Inter', sans-serif;

Signed-off-by: huongg <huongg1409@gmail.com>

* testing with important for font-family

Signed-off-by: huongg <huongg1409@gmail.com>

* update assets for experiment tracking, docs, pipeline, focus mode, matplotlib and plotly

* update modular pipelines section

Signed-off-by: huongg <huongg1409@gmail.com>

* autoreload

Signed-off-by: huongg <huongg1409@gmail.com>

* reupload autoreload gif

Signed-off-by: huongg <huongg1409@gmail.com>

* revert back to develop branch

Signed-off-by: huongg <huongg1409@gmail.com>

* gif for ET comparison

Signed-off-by: huongg <huongg1409@gmail.com>

* remove jupyter notebook screenshot since its not used anywhere

Signed-off-by: huongg <huongg1409@gmail.com>

* Update databricks viz demo

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* pipeline_show_metrics gif

Signed-off-by: huongg <huongg1409@gmail.com>

* update different gif format

Signed-off-by: huongg <huongg1409@gmail.com>

* point to main instead of develop for kedro-banner

Signed-off-by: huongg <huongg1409@gmail.com>

---------

Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Signed-off-by: huongg <huongg1409@gmail.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Co-authored-by: Tynan DeBold <thdebold@gmail.com>
Co-authored-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix the level heading for collab exp tracking so it shows in sidebar (#2647)

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update Kedro icons in docs with for rebrand (#2652)

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Modify logos (#2651)

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix lint

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>

* remove the subsection of kedro.extras.extension

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Fix lint

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* remove the subsection of kedro.extras.extension

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com>
Signed-off-by: debugger24 <rahulcomp24@gmail.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Signed-off-by: huongg <huongg1409@gmail.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>
Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com>
Co-authored-by: Tomas Van Pottelbergh <85937396+tomasvanpottelbergh@users.noreply.github.com>
Co-authored-by: Rahul Kumar <rahulcomp24@gmail.com>
Co-authored-by: Tynan DeBold <thdebold@gmail.com>
Co-authored-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>
Co-authored-by: huongg <huongg1409@gmail.com>
Co-authored-by: Ravi Kumar Pilla <ravi_kumar_pilla@mckinsey.com>
Co-authored-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Co-authored-by: Merel Theisen <merel.theisen@quantumblack.com>
astrojuanlu added a commit that referenced this issue Jun 13, 2023
@astrojuanlu
Do not rely on setup.py to generate sdist
1456b8e 
See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu moved this from In Progress to To Do in Kedro Framework Jun 13, 2023
astrojuanlu added a commit that referenced this issue Jun 13, 2023
@astrojuanlu
micropkg packaging improvements (#2614)
37277c3 
* Make `kedro micropkg package` accept `--verbose`

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Improve error when `micropkg pull` does not find sdist
Fix gh-2542.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Stop using pkg_resources

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Do not rely on setup.py to generate sdist

See gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Stop relying on .egg-info directories

Fix gh-2567.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Note change from pkg_requirements

See pypa/packaging#644 (comment)

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Improve code comments

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix equality checks of equivalent requirements

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add micropackaging improvements to release notes

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert sdist check to make it more testable

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix micropkg pull error handling

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add tests for new micropkg pull branches

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Remove untested path of private code

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix micropkg tests

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add more detailed explanation of Requirement custom subclass

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue Jul 3, 2023
@astrojuanlu
Make micropkg package use pyproject.toml
eb08378 
Fix gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu mentioned this issue Jul 3, 2023
Merged
5 tasks
@astrojuanlu astrojuanlu moved this from To Do to In Review in Kedro Framework Jul 3, 2023
@astrojuanlu astrojuanlu mentioned this issue Jul 3, 2023
Closed
5 tasks
astrojuanlu added a commit that referenced this issue Jul 5, 2023
@astrojuanlu
Make micropkg package use pyproject.toml
7723112 
Fix gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue Jul 7, 2023
@astrojuanlu
Make micropkg package use pyproject.toml
37526f2 
Fix gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue Jul 10, 2023
@astrojuanlu
Make micropkg package use pyproject.toml
80055fd 
Fix gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu added a commit that referenced this issue Jul 10, 2023
@astrojuanlu
Make micropkg package use pyproject.toml (#2761)
58167a8 
Fix gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@github-project-automation github-project-automation bot moved this from In Review to Done in Kedro Framework Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@astrojuanlu astrojuanlu

Labels
Issue: Feature Request New feature or improvement to existing feature
Projects
Kedro Framework
Archived in project
Milestone
Move to `pyproject.toml` package defi...
Development

Successfully merging a pull request may close this issue.

Make kedro micropkg package use pyproject.toml kedro-org/kedro
2 participants
@astrojuanlu @merelcht