Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance test for OmegaConfigLoader #4225

Merged
merged 8 commits into from
Oct 16, 2024
Merged

Conversation

ankatiyar
Copy link
Contributor

@ankatiyar ankatiyar commented Oct 14, 2024

Description

Partial solution for #4125

Development notes

Added performance test for OmegaConfigLoader in benchmarks/benchmark_ocl.py for basic operations for loading catalog, parameters, globals
As well as another test class for operations with global and custom resolver.

Test locally with asv run

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

  • Read the contributing guidelines
  • Signed off each commit with a Developer Certificate of Origin (DCO)
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes
  • Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
benchmarks/benchmark_ocl.py Outdated Show resolved Hide resolved
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
@ankatiyar ankatiyar marked this pull request as ready for review October 14, 2024 14:55
Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! And the main difference between TimeOmegaConfigLoader and TimeOmegaConfigLoaderAdvanced is interpolation?

@ankatiyar
Copy link
Contributor Author

This looks great! And the main difference between TimeOmegaConfigLoader and TimeOmegaConfigLoaderAdvanced is interpolation?

@merelcht Yeah, I had to separate it out into a new class because the setup() function had to be different - asv runs the setup() and teardown() before each time_ test.

@ankatiyar ankatiyar mentioned this pull request Oct 15, 2024
7 tasks
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
@noklam noklam self-requested a review October 15, 2024 23:12
Copy link
Contributor

@ElenaKhaustova ElenaKhaustova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 👍

@ankatiyar ankatiyar enabled auto-merge (squash) October 16, 2024 13:51
@ankatiyar ankatiyar merged commit f594c8b into main Oct 16, 2024
28 checks passed
@ankatiyar ankatiyar deleted the performance-ocl-datacatalog branch October 16, 2024 14:10
ankatiyar added a commit that referenced this pull request Oct 17, 2024
* first pass doesn't work yet

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update ocl tests

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* revert some changes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update to use larger config

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update functions and docstrings

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* lint

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
ankatiyar added a commit that referenced this pull request Oct 18, 2024
* Update index.md (#4221)

Fixed an erroneous link to the Get started with Kedro - Create your first data pipeline with Kedro video.  It was accidentally linked to the previous video.

Signed-off-by: Greg Vaslowski <7269272+Vaslo@users.noreply.github.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Bump kedro-sphinx-theme from 2024.4.0 to 2024.10.0 (#4216)

* Bump kedro-sphinx-theme from 2024.4.0 to 2024.10.0

Bumps [kedro-sphinx-theme](https://github.com/kedro-org/kedro-sphinx-theme) from 2024.4.0 to 2024.10.0.
- [Release notes](https://github.com/kedro-org/kedro-sphinx-theme/releases)
- [Commits](kedro-org/kedro-sphinx-theme@v2024.4.0...v2024.10.0)

---
updated-dependencies:
- dependency-name: kedro-sphinx-theme
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* updated to 2024.10.2

* trigger_run

* trigger_run

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com>
Co-authored-by: rashidakanchwala <rashida_kanchwala@mckinsey.com>
Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Replace all instances of "data set" with "dataset" (#4211)

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Manually created sitemap.xml for improved control over indexed docs pages (#4145)

* Load manually created sitemap

Signed-off-by: Dmitry Sorokin <129520297+DmitrySorokinQB@users.noreply.github.com>

* Add projects remove lastmod for latest

Signed-off-by: Dmitry Sorokin <129520297+DmitrySorokinQB@users.noreply.github.com>

* Add latest for projects

Signed-off-by: Dmitry Sorokin <129520297+DmitrySorokinQB@users.noreply.github.com>

---------

Signed-off-by: Dmitry Sorokin <129520297+DmitrySorokinQB@users.noreply.github.com>
Co-authored-by: Dmitry Sorokin <129520297+DmitrySorokinQB@users.noreply.github.com>
Co-authored-by: ElenaKhaustova <157851531+ElenaKhaustova@users.noreply.github.com>
Co-authored-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Bump up version to 0.19.9 (#4219)

* Bump up version to 0.19.9

Signed-off-by: Laura Couto <laurarccouto@gmail.com>

* Add placeholders to release.md

Signed-off-by: Laura Couto <laurarccouto@gmail.com>

* Update citation.cff release date

Signed-off-by: Laura Couto <laurarccouto@gmail.com>

---------

Signed-off-by: Laura Couto <laurarccouto@gmail.com>
Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* first pass doesn't work yet

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update ocl tests

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* revert some changes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update to use larger config

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update functions and docstrings

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add performance tests for DataCatalog

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update mypy ignore messages (#4228)

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Revise Kedro project structure docs (#4208)

* Update project structure docs
---------

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Signed-off-by: Dmitry Sorokin <40151847+DimedS@users.noreply.github.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update CLI autocompletion docs with new Click syntax (#4213)

* Update CLI autocompletion docs with new Click syntax

Updated the autocompletion setup instructions for Bash, Zsh, and Fish shells to reflect the latest Click 8.1 syntax. Changed Fish shell completion script path to ~/.config/fish/completions/kedro.fish for correct placement.

Signed-off-by: hyew0nChoi <statistic9935@naver.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Bump import-linter from 2.0 to 2.1 (#4226)

Bumps [import-linter](https://github.com/seddonym/import-linter) from 2.0 to 2.1.
- [Changelog](https://github.com/seddonym/import-linter/blob/master/CHANGELOG.rst)
- [Commits](seddonym/import-linter@v2.0...v2.1)

---
updated-dependencies:
- dependency-name: import-linter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Performance test for `OmegaConfigLoader` (#4225)

* first pass doesn't work yet

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update ocl tests

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* revert some changes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update to use larger config

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update functions and docstrings

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* lint

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add a test for init and fix indent

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Revert "Add a test for init and fix indent"

This reverts commit 0dbe3c7.

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add a test for init and fix indent

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Greg Vaslowski <7269272+Vaslo@users.noreply.github.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Dmitry Sorokin <129520297+DmitrySorokinQB@users.noreply.github.com>
Signed-off-by: Laura Couto <laurarccouto@gmail.com>
Signed-off-by: L. R. Couto <57910428+lrcouto@users.noreply.github.com>
Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Signed-off-by: Dmitry Sorokin <40151847+DimedS@users.noreply.github.com>
Signed-off-by: hyew0nChoi <statistic9935@naver.com>
Signed-off-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants