Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(datasets): add SparkStreamingDataSet #198

Merged
merged 100 commits into from
May 31, 2023
Merged

feat(datasets): add SparkStreamingDataSet #198

merged 100 commits into from
May 31, 2023

Commits on May 1, 2023

  1. Fix links on GitHub issue templates (kedro-org#150) 

    Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @astrojuanlu @tingtingQB
    astrojuanlu authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    46bb394 View commit details
    Browse the repository at this point in the history

  2. add spark_stream_dataset.py 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    c9421ae View commit details
    Browse the repository at this point in the history

  3. Migrate most of kedro-datasets metadata to pyproject.toml (kedro-… 

    …org#161)

* Include missing requirements files in sdist

Fix kedro-orggh-86.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Migrate most project metadata to `pyproject.toml`

See kedro-org/kedro#2334.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Move requirements to `pyproject.toml`

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @astrojuanlu @tingtingQB
    astrojuanlu authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    63f578a View commit details
    Browse the repository at this point in the history

  4. restructure the strean dataset to align with the other spark dataset 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    4b387ff View commit details
    Browse the repository at this point in the history

  5. adding README.md for specification 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    39ad9fd View commit details
    Browse the repository at this point in the history

  6. Update kedro-datasets/kedro_datasets/spark/spark_stream_dataset.py 

    Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB @noklam
    tingtingQB and noklam committed May 1, 2023
    Configuration menu
    Copy the full SHA
    69eb8be View commit details
    Browse the repository at this point in the history

  7. rename the dataset 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    3106068 View commit details
    Browse the repository at this point in the history

  8. resolve comments 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    b8141a7 View commit details
    Browse the repository at this point in the history

  9. fix format and pylint 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    738625e View commit details
    Browse the repository at this point in the history

  10. Update kedro-datasets/kedro_datasets/spark/README.md 

    Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB @deepyaman
    tingtingQB and deepyaman committed May 1, 2023
    Configuration menu
    Copy the full SHA
    a54cc67 View commit details
    Browse the repository at this point in the history

  11. add unit tests and SparkStreamingDataset in init.py 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    b924ad6 View commit details
    Browse the repository at this point in the history

  12. add unit tests 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    743b823 View commit details
    Browse the repository at this point in the history

  13. update test_save 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    3bb3717 View commit details
    Browse the repository at this point in the history

  14. Upgrade Polars (kedro-org#171) 

    * Upgrade Polars

Signed-off-by: Juan Luis Cano Rodríguez <hello@juanlu.space>

* Update Polars to 0.17.x

---------

Signed-off-by: Juan Luis Cano Rodríguez <hello@juanlu.space>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @astrojuanlu @tingtingQB
    astrojuanlu authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    ae3bc87 View commit details
    Browse the repository at this point in the history

  15. if release is failed, it return exit code and fail the CI (kedro-org#158 

    )

Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @noklam @tingtingQB
    noklam authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    eb634a1 View commit details
    Browse the repository at this point in the history

  16. Migrate kedro-airflow to static metadata (kedro-org#172) 

    * Migrate kedro-airflow to static metadata

See kedro-org/kedro#2334.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add explicit PEP 518 build requirements for kedro-datasets

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Typos

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Remove dangling reference to requirements.txt

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add release notes

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @astrojuanlu @tingtingQB
    astrojuanlu authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    115940b View commit details
    Browse the repository at this point in the history

  17. Migrate kedro-telemetry to static metadata (kedro-org#174) 

    * Migrate kedro-telemetry to static metadata

See kedro-org/kedro#2334.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add release notes

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @astrojuanlu @tingtingQB
    astrojuanlu authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    35231af View commit details
    Browse the repository at this point in the history

  18. ci: port lint, unit test, and e2e tests to Actions (kedro-org#155) 

    * Add unit test + lint test on GA

* trigger GA - will revert

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Fix lint

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add end to end tests

* Add cache key

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add cache action

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Rename workflow files

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Lint + add comment + default bash

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add windows test

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update workflow name + revert changes to READMEs

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add kedro-telemetry/RELEASE.md to trufflehog ignore

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add pytables to test_requirements remove from workflow

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Revert "Add pytables to test_requirements remove from workflow"

This reverts commit 8203daa.

* Separate pip freeze step

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @ankatiyar @tingtingQB
    ankatiyar authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    8c2ea1b View commit details
    Browse the repository at this point in the history

  19. Migrate kedro-docker to static metadata (kedro-org#173) 

    * Migrate kedro-docker to static metadata

See kedro-org/kedro#2334.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Address packaging warning

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix tests

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Actually install current plugin with dependencies

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add release notes

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @astrojuanlu @tingtingQB
    astrojuanlu authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    a73b216 View commit details
    Browse the repository at this point in the history

  20. Introdcuing .gitpod.yml to kedro-plugins (kedro-org#185) 

    Currently opening gitpod will installed a Python 3.11 which breaks everything because we don't support it set. This PR introduce a simple .gitpod.yml to get it started.

Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @noklam @tingtingQB
    noklam authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    7f4527d View commit details
    Browse the repository at this point in the history

  21. sync APIDataSet from kedro's develop (kedro-org#184) 

    * Update APIDataSet

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Sync ParquetDataSet

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Sync Test

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Linting

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Revert Unnecessary ParquetDataSet Changes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Sync release notes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @noklam @tingtingQB
    noklam authored and tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    57a11d6 View commit details
    Browse the repository at this point in the history

  22. formatting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    11c3888 View commit details
    Browse the repository at this point in the history

  23. formatting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    634d884 View commit details
    Browse the repository at this point in the history

  24. formatting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    9e8f55c View commit details
    Browse the repository at this point in the history

  25. formatting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    dbdf19c View commit details
    Browse the repository at this point in the history

  26. Merge remote-tracking branch 'origin/add-stream-datasets' into add-st… 

    …ream-datasets

# Conflicts:
#	.github/workflows/check-plugin.yml
#	kedro-datasets/tests/api/test_api_dataset.py
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    4e49fd9 View commit details
    Browse the repository at this point in the history

  27. add spark_stream_dataset.py 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    1a7a477 View commit details
    Browse the repository at this point in the history

  28. restructure the strean dataset to align with the other spark dataset 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    e877944 View commit details
    Browse the repository at this point in the history

  29. adding README.md for specification 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    09e9cf2 View commit details
    Browse the repository at this point in the history

  30. Update kedro-datasets/kedro_datasets/spark/spark_stream_dataset.py 

    Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB @noklam
    tingtingQB and noklam committed May 1, 2023
    Configuration menu
    Copy the full SHA
    2e30ec0 View commit details
    Browse the repository at this point in the history

  31. rename the dataset 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    6147636 View commit details
    Browse the repository at this point in the history

  32. resolve comments 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    29376e9 View commit details
    Browse the repository at this point in the history

  33. fix format and pylint 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    42ed37a View commit details
    Browse the repository at this point in the history

  34. Update kedro-datasets/kedro_datasets/spark/README.md 

    Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB @deepyaman
    tingtingQB and deepyaman committed May 1, 2023
    Configuration menu
    Copy the full SHA
    d93d9b9 View commit details
    Browse the repository at this point in the history

  35. add unit tests and SparkStreamingDataset in init.py 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    5b83444 View commit details
    Browse the repository at this point in the history

  36. add unit tests 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    5b0630e View commit details
    Browse the repository at this point in the history

  37. update test_save 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    1433808 View commit details
    Browse the repository at this point in the history

  38. formatting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    c7778b5 View commit details
    Browse the repository at this point in the history

  39. formatting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    7341429 View commit details
    Browse the repository at this point in the history

  40. formatting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    d8d3bc2 View commit details
    Browse the repository at this point in the history

  41. formatting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    be4a3e5 View commit details
    Browse the repository at this point in the history

  42. Merge remote-tracking branch 'origin/add-stream-datasets' into add-st… 

    …ream-datasets
    @tingtingQB
    tingtingQB committed May 1, 2023
    Configuration menu
    Copy the full SHA
    d3bc0d2 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2023

  1. lint 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 2, 2023
    Configuration menu
    Copy the full SHA
    e39c639 View commit details
    Browse the repository at this point in the history

  2. lint 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 2, 2023
    Configuration menu
    Copy the full SHA
    66440f4 View commit details
    Browse the repository at this point in the history

  3. lint 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 2, 2023
    Configuration menu
    Copy the full SHA
    0ed5b90 View commit details
    Browse the repository at this point in the history

  4. update test cases 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 2, 2023
    Configuration menu
    Copy the full SHA
    04c623b View commit details
    Browse the repository at this point in the history

  5. add negative test 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 2, 2023
    Configuration menu
    Copy the full SHA
    a76f944 View commit details
    Browse the repository at this point in the history

  6. remove code snippets fpr testing 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 2, 2023
    Configuration menu
    Copy the full SHA
    30b002d View commit details
    Browse the repository at this point in the history

  7. lint 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 2, 2023
    Configuration menu
    Copy the full SHA
    9bef3a2 View commit details
    Browse the repository at this point in the history

  8. update tests 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 2, 2023
    Configuration menu
    Copy the full SHA
    0bb5fe1 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2023

  1. update test and remove redundacy 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 4, 2023
    Configuration menu
    Copy the full SHA
    e0ebe27 View commit details
    Browse the repository at this point in the history

  2. linting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 4, 2023
    Configuration menu
    Copy the full SHA
    5bb5766 View commit details
    Browse the repository at this point in the history

  3. refactor file format 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    2075781 View commit details
    Browse the repository at this point in the history

  4. fix read me file 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    e8ea0d3 View commit details
    Browse the repository at this point in the history

  5. docs: Add community contributions (kedro-org#199) 

    * Add community contributions

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Use newer link to docs

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
    @astrojuanlu @kuriantom369
    astrojuanlu authored and kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    f08dd09 View commit details
    Browse the repository at this point in the history

  6. adding test for raise error 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB @kuriantom369
    tingtingQB authored and kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    24bb527 View commit details
    Browse the repository at this point in the history

  7. update test and remove redundacy 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @tingtingQB @kuriantom369
    tingtingQB authored and kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    437e77e View commit details
    Browse the repository at this point in the history

  8. linting 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @tingtingQB @kuriantom369
    tingtingQB authored and kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    a3fdbf6 View commit details
    Browse the repository at this point in the history

  9. refactor file format 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    9d60f25 View commit details
    Browse the repository at this point in the history

  10. fix read me file 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    ced007d View commit details
    Browse the repository at this point in the history

  11. adding test for raise error 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @tingtingQB @kuriantom369
    tingtingQB authored and kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    0b88324 View commit details
    Browse the repository at this point in the history

  12. fix readme file 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    ed26aad View commit details
    Browse the repository at this point in the history

  13. fix readme 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    170b092 View commit details
    Browse the repository at this point in the history

  14. fix conflicts 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    e63a53a View commit details
    Browse the repository at this point in the history

  15. fix ci erors 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    d986c75 View commit details
    Browse the repository at this point in the history

  16. Merge branch 'kedro-org:main' into add-stream-datasets

    @tingtingQB
    tingtingQB authored May 4, 2023
    Configuration menu
    Copy the full SHA
    88e6ee4 View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. fix lint issue 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @tingtingQB @kuriantom369
    tingtingQB authored and kuriantom369 committed May 5, 2023
    Configuration menu
    Copy the full SHA
    64232fa View commit details
    Browse the repository at this point in the history

  2. update class documentation 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 5, 2023
    Configuration menu
    Copy the full SHA
    8a61b41 View commit details
    Browse the repository at this point in the history

Commits on May 16, 2023

  1. add additional test cases 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    37e66e8 View commit details
    Browse the repository at this point in the history

  2. add s3 read test cases 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    07032a8 View commit details
    Browse the repository at this point in the history

  3. add s3 read test cases 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    2470de1 View commit details
    Browse the repository at this point in the history

  4. add s3 read test case 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    c4e0f4e View commit details
    Browse the repository at this point in the history

  5. test s3 read 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    7e3555e View commit details
    Browse the repository at this point in the history

Commits on May 17, 2023

  1. remove redundant test cases 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 17, 2023
    Configuration menu
    Copy the full SHA
    6a0029d View commit details
    Browse the repository at this point in the history

Commits on May 23, 2023

  1. fix streaming dataset configurations 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 23, 2023
    Configuration menu
    Copy the full SHA
    e8f6696 View commit details
    Browse the repository at this point in the history

Commits on May 25, 2023

  1. update streaming datasets doc 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 25, 2023
    Configuration menu
    Copy the full SHA
    9a5ebad View commit details
    Browse the repository at this point in the history

  2. resolve comments re documentation 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 25, 2023
    Configuration menu
    Copy the full SHA
    eacdd46 View commit details
    Browse the repository at this point in the history

  3. bugfix lint 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 25, 2023
    Configuration menu
    Copy the full SHA
    68b6e1b View commit details
    Browse the repository at this point in the history

  4. update link 

    Signed-off-by: Tingting_Wan <tingting_wan@mckinsey.com>
    @tingtingQB
    tingtingQB committed May 25, 2023
    Configuration menu
    Copy the full SHA
    5b2a479 View commit details
    Browse the repository at this point in the history

Commits on May 26, 2023

  1. revert the changes on CI 

    Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
    @noklam
    noklam committed May 26, 2023
    Configuration menu
    Copy the full SHA
    b94f211 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2023

  1. test(docker): remove outdated logging-related step (kedro-org#207) 

    * fixkedro- docker e2e test

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* fix: add timeout to request to satisfy bandit lint

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @noklam @deepyaman @kuriantom369
    2 people authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    9381816 View commit details
    Browse the repository at this point in the history

  2. ci: ensure plugin requirements get installed in CI (kedro-org#208) 

    * ci: install the plugin alongside test requirements

* ci: install the plugin alongside test requirements

* Update kedro-airflow.yml

* Update kedro-datasets.yml

* Update kedro-docker.yml

* Update kedro-telemetry.yml

* Update kedro-airflow.yml

* Update kedro-datasets.yml

* Update kedro-airflow.yml

* Update kedro-docker.yml

* Update kedro-telemetry.yml

* ci(telemetry): update isort config to correct sort

* Don't use profile ¯\_(ツ)_/¯

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* chore(datasets): remove empty `tool.black` section

* chore(docker): remove empty `tool.black` section

---------

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @deepyaman @kuriantom369
    deepyaman authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    373e166 View commit details
    Browse the repository at this point in the history

  3. ci: Migrate the release workflow from CircleCI to GitHub Actions (ked… 

    …ro-org#203)

* Create check-release.yml

* change from test pypi to pypi

* split into jobs and move version logic into script

* update github actions output

* lint

* changes based on review

* changes based on review

* fix script to not append continuously

* change pypi api token logic

Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @SajidAlamQB @kuriantom369
    SajidAlamQB authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    f033b95 View commit details
    Browse the repository at this point in the history

  4. build: Relax Kedro bound for kedro-datasets (kedro-org#140) 

    * Less strict pin on Kedro for datasets

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @merelcht @kuriantom369
    merelcht authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    3fdb71c View commit details
    Browse the repository at this point in the history

  5. ci: don't run checks on both push/pull_request (kedro-org#192) 

    * ci: don't run checks on both `push`/`pull_request`

* ci: don't run checks on both `push`/`pull_request`

* ci: don't run checks on both `push`/`pull_request`

* ci: don't run checks on both `push`/`pull_request`

Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @deepyaman @kuriantom369
    deepyaman authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    b08aa6f View commit details
    Browse the repository at this point in the history

  6. chore: delete extra space ending check-release.yml (kedro-org#210) 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @deepyaman @kuriantom369
    deepyaman authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    148b464 View commit details
    Browse the repository at this point in the history

  7. ci: Create merge-gatekeeper.yml to make sure PR only merged when all … 

    …tests checked. (kedro-org#215)

* Create merge-gatekeeper.yml

* Update .github/workflows/merge-gatekeeper.yml

---------

Co-authored-by: Sajid Alam <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @noklam @SajidAlamQB @kuriantom369
    2 people authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    be2431c View commit details
    Browse the repository at this point in the history

  8. ci: Remove the CircleCI setup (kedro-org#209) 

    * remove circleci setup files and utils

* remove circleci configs in kedro-telemetry

* remove redundant .github in kedro-telemetry

* Delete continue_config.yml

* Update check-release.yml

* lint

* increase timeout to 40 mins for docker e2e tests

Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @SajidAlamQB @kuriantom369
    SajidAlamQB authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    74a211f View commit details
    Browse the repository at this point in the history

  9. feat: Dataset API add save method (kedro-org#180) 

    * [FEAT] add save method to APIDataset

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [ENH] create save_args parameter for api_dataset

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [ENH] add tests for socket + http errors

Signed-off-by: <jmcdonnell@fieldbox.ai>
Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [ENH] check save data is json

Signed-off-by: <jmcdonnell@fieldbox.ai>
Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [FIX] clean code

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [ENH] handle different data types

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [FIX] test coverage for exceptions

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [ENH] add examples in APIDataSet docstring

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* sync APIDataSet  from kedro's `develop` (kedro-org#184)

* Update APIDataSet

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Sync ParquetDataSet

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Sync Test

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Linting

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Revert Unnecessary ParquetDataSet Changes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Sync release notes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [FIX] remove support for delete method

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [FIX] lint files

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [FIX] fix conflicts

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [FIX] remove fail save test

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [ENH] review suggestions

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [ENH] fix tests

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

* [FIX] reorder arguments

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>

---------

Signed-off-by: jmcdonnell <jmcdonnell@fieldbox.ai>
Signed-off-by: <jmcdonnell@fieldbox.ai>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: jmcdonnell <jmcdonnell@fieldbox.ai>
Co-authored-by: Nok Lam Chan <mediumnok@gmail.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @McDonnellJoseph @noklam @kuriantom369
    3 people authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    9d7820a View commit details
    Browse the repository at this point in the history

  10. ci: Automatically extract release notes for GitHub Releases (kedro-or… 

    …g#212)

* ci: Automatically extract release notes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* fix lint

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Raise exceptions

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Lint

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Lint

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @ankatiyar @kuriantom369
    ankatiyar authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    36de4b9 View commit details
    Browse the repository at this point in the history

  11. feat: Add metadata attribute to datasets (kedro-org#189) 

    * Add metadata attribute to all datasets

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @AhdraMeraliQB @kuriantom369
    AhdraMeraliQB authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    870e623 View commit details
    Browse the repository at this point in the history

  12. feat: Add ManagedTableDataset for managed Delta Lake tables in Databr… 

    …icks (kedro-org#206)

* committing first version of UnityTableCatalog with unit tests. This datasets allows users to interface with Unity catalog tables in Databricks to both read and write.

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* renaming dataset

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* adding mlflow connectors

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* fixing mlflow imports

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* cleaned up mlflow for initial release

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* cleaned up mlflow references from setup.py for initial release

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* fixed deps in setup.py

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* adding comments before intiial PR

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* moved validation to dataclass

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* bug fix in type of partition column and cleanup

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* updated docstring for ManagedTableDataSet

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* added backticks to catalog

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* fixing regex to allow hyphens

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/test_requirements.txt

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* adding backticks to catalog

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Require pandas < 2.0 for compatibility with spark < 3.4

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Replace use of walrus operator

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add test coverage for validation methods

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove unused versioning functions

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix exception catching for invalid schema, add test for invalid schema

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add pylint ignore

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add tests/databricks to ignore for no-spark tests

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Nok Lam Chan <mediumnok@gmail.com>

* Update kedro-datasets/kedro_datasets/databricks/managed_table_dataset.py

Co-authored-by: Nok Lam Chan <mediumnok@gmail.com>

* Remove spurious mlflow test dependency

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add explicit check for database existence

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove character limit for table names

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Refactor validation steps in ManagedTable

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove spurious checks for table and schema name existence

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

---------

Signed-off-by: Danny Farah <danny_farah@mckinsey.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Co-authored-by: Danny Farah <danny.farah@quantumblack.com>
Co-authored-by: Danny Farah <danny_farah@mckinsey.com>
Co-authored-by: Nok Lam Chan <mediumnok@gmail.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @jmholzer @dannyrfar
    @noklam @kuriantom369
    4 people authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    9d66cc8 View commit details
    Browse the repository at this point in the history

  13. docs: Update APIDataset docs and refactor (kedro-org#217) 

    * Update APIDataset docs and refactor

* Acknowledge community contributor

* Fix more broken doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Lint

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix release notes of upcoming kedro-datasets

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @noklam @astrojuanlu
    @jmholzer @kuriantom369
    3 people authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    0aaa922 View commit details
    Browse the repository at this point in the history

  14. feat: Release kedro-datasets version 1.3.0 (kedro-org#219) 

    * Modify release version and RELEASE.md

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add proper name for ManagedTableDataSet

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update kedro-datasets/RELEASE.md

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert lost semicolon for release 1.2.0

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

---------

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @jmholzer @astrojuanlu @kuriantom369
    2 people authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    ccec03b View commit details
    Browse the repository at this point in the history

  15. docs: Fix APIDataSet docstring (kedro-org#220) 

    * Fix APIDataSet docstring

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add release notes

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Separate [docs] extras from [all] in kedro-datasets

Fix kedro-orggh-143.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @astrojuanlu @kuriantom369
    astrojuanlu authored and kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    c2a7128 View commit details
    Browse the repository at this point in the history

  16. Update kedro-datasets/tests/spark/test_spark_streaming_dataset.py 

    Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369 @deepyaman
    kuriantom369 and deepyaman committed May 30, 2023
    Configuration menu
    Copy the full SHA
    64446dc View commit details
    Browse the repository at this point in the history

  17. Update kedro-datasets/kedro_datasets/spark/spark_streaming_dataset.py 

    Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369 @deepyaman
    kuriantom369 and deepyaman committed May 30, 2023
    Configuration menu
    Copy the full SHA
    497001d View commit details
    Browse the repository at this point in the history

  18. Update kedro-datasets/setup.py 

    Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369 @deepyaman
    kuriantom369 and deepyaman committed May 30, 2023
    Configuration menu
    Copy the full SHA
    7f25f3c View commit details
    Browse the repository at this point in the history

  19. Merge branch 'main' into add-stream-datasets

    @kuriantom369
    kuriantom369 authored May 30, 2023
    Configuration menu
    Copy the full SHA
    bd88b99 View commit details
    Browse the repository at this point in the history

  20. fix linting issue 

    Signed-off-by: Tom Kurian <tom_kurian@mckinsey.com>
    @kuriantom369
    kuriantom369 committed May 30, 2023
    Configuration menu
    Copy the full SHA
    c094db1 View commit details
    Browse the repository at this point in the history