Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add FunctionalTableTransformer #901

Merged
merged 19 commits into from
Jul 12, 2024

Conversation

Tarmandan
Copy link
Contributor

Closes #858

Summary of Changes

This implements a Transformer that wraps a Callable[[Table], Table] so operations on Tables can be inserted into the operation order of a SequentialTableTransformer.
Note that this transformer inherently cannot be invertible.
Since there is no type checking at runtime, callables with wrong argument or return types will only throw an exception upon calling transform but not during init.

@Tarmandan Tarmandan linked an issue Jul 12, 2024 that may be closed by this pull request
Copy link
Contributor

github-actions bot commented Jul 12, 2024

🦙 MegaLinter status: ✅ SUCCESS

Descriptor Linter Files Fixed Errors Elapsed time
✅ PYTHON black 4 0 0 0.98s
✅ PYTHON mypy 4 0 2.47s
✅ PYTHON ruff 4 0 0 0.36s
✅ REPOSITORY git_diff yes no 0.44s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff

MegaLinter is graciously provided by OX Security

Copy link

codecov bot commented Jul 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.72%. Comparing base (6f63a6c) to head (2e2154b).
Report is 39 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #901      +/-   ##
==========================================
+ Coverage   97.67%   97.72%   +0.04%     
==========================================
  Files         120      121       +1     
  Lines        6234     6499     +265     
==========================================
+ Hits         6089     6351     +262     
- Misses        145      148       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Tarmandan
Copy link
Contributor Author

The linter found 2 problems. I fixed the first by adding type hint to the funciton valid_callable in the tests.
However the other problem is that the linter correctly notices that invalid_callable has the wrong types. I am not sure if the test is superfluous. The test for internal error of the callable raises an exception in the same try-except, just later.

@Tarmandan
Copy link
Contributor Author

The test for invalid callable type has been removed.
The only problem remaining is test coverage. The hash test in test_table_transformer.py should be adjusted, but FunctionalTableTransformer should only give different hash when wrapping different callable, so it will need its own tests. It might be less work to implement it in the test_functional_table_transformer.py, but would break the pattern.

@lars-reimann
Copy link
Member

For now, it should suffice to add the transformer to https://github.com/Safe-DS/Stdlib/blob/69a780cab1a5dcab33c8c7a36f03bd2eb3367683/tests/safeds/data/tabular/transformation/test_table_transformer.py#L35. We can adjust the test later so hash is tested more in-depth for all transformers. Maybe add an issue for that.

@lars-reimann lars-reimann force-pushed the 858-functional-table-transformer branch from b5ae84e to 2e2154b Compare July 12, 2024 11:20
Copy link
Member

@lars-reimann lars-reimann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, nice work.

@lars-reimann lars-reimann merged commit 37905be into main Jul 12, 2024
10 checks passed
@lars-reimann lars-reimann deleted the 858-functional-table-transformer branch July 12, 2024 11:31
lars-reimann pushed a commit that referenced this pull request Jul 19, 2024
## [0.27.0](v0.26.0...v0.27.0) (2024-07-19)

### Features

*  join ([#870](#870)) ([5764441](5764441)), closes [#745](#745)
* activation function for forward layer ([#891](#891)) ([5b5bb3f](5b5bb3f)), closes [#889](#889)
* add `ImageDataset.split` ([#846](#846)) ([3878751](3878751)), closes [#831](#831)
* add FunctionalTableTransformer ([#901](#901)) ([37905be](37905be)), closes [#858](#858)
* add InvalidFitDataError ([#824](#824)) ([487854c](487854c)), closes [#655](#655)
* add KNearestNeighborsImputer ([#864](#864)) ([fcdfecf](fcdfecf)), closes [#743](#743)
* add moving average plot ([#836](#836)) ([abcf68a](abcf68a))
* add RobustScaler ([#874](#874)) ([62320a3](62320a3)), closes [#650](#650) [#873](#873)
* add SequentialTableTransformer ([#893](#893)) ([e93299f](e93299f)), closes [#802](#802)
* add temporal operations ([#832](#832)) ([06eab77](06eab77))
* added 'histogram_2d' in TablePlotter  ([#903](#903)) ([4e65ba9](4e65ba9)), closes [#869](#869) [#798](#798)
* added from_str_to_temporal and continues prediction ([#767](#767)) ([35f468a](35f468a)), closes [#806](#806) [#765](#765) [#740](#740) [#773](#773)
* added GRU layer ([#845](#845)) ([d33cb5d](d33cb5d))
* Adds Dropout Layer ([#868](#868)) ([a76f0a1](a76f0a1)), closes [#848](#848)
* dark mode for plots ([#911](#911)) ([5447551](5447551)), closes [#798](#798)
* easily create a baseline model ([#811](#811)) ([8e1b995](8e1b995)), closes [#710](#710)
* get first cell with value other than `None` ([#904](#904)) ([5a0cdb3](5a0cdb3)), closes [#799](#799)
* hyperparameter optimization for fnn models ([#897](#897)) ([c1f66e5](c1f66e5)), closes [#861](#861)
* implement violin plots ([#900](#900)) ([9f5992a](9f5992a)), closes [#867](#867)
* plot decision tree ([#876](#876)) ([d3f81dc](d3f81dc)), closes [#856](#856)
* prediction no longer takes a time series dataset only table ([#838](#838)) ([762e5c2](762e5c2)), closes [#837](#837)
* raise if `remove_colums` is called with unknown column by default ([#852](#852)) ([8f78163](8f78163)), closes [#807](#807)
* regularization strength for logistic classifier ([#866](#866)) ([9f74e92](9f74e92)), closes [#750](#750)
* reorders parameters of RangeScaler and makes them keyword-only ([#847](#847)) ([2b82db7](2b82db7)), closes [#809](#809)
* replace seaborn with matplotlib for box_plot ([#863](#863)) ([4ef078e](4ef078e)), closes [#805](#805) [#849](#849)
* replaced seaborn with matplotlib for correlation_heatmap ([#850](#850)) ([d4680d4](d4680d4)), closes [#800](#800) [#849](#849)

### Bug Fixes

* **deps:** bump urllib3 from 2.2.1 to 2.2.2 ([#842](#842)) ([b81bcd6](b81bcd6)), closes [#3122](https://github.com/Safe-DS/Library/issues/3122) [#3363](https://github.com/Safe-DS/Library/issues/3363) [#3122](https://github.com/Safe-DS/Library/issues/3122) [#3363](https://github.com/Safe-DS/Library/issues/3363) [#3406](https://github.com/Safe-DS/Library/issues/3406) [#3398](https://github.com/Safe-DS/Library/issues/3398) [#3399](https://github.com/Safe-DS/Library/issues/3399) [#3396](https://github.com/Safe-DS/Library/issues/3396) [#3394](https://github.com/Safe-DS/Library/issues/3394) [#3391](https://github.com/Safe-DS/Library/issues/3391) [#3316](https://github.com/Safe-DS/Library/issues/3316) [#3387](https://github.com/Safe-DS/Library/issues/3387) [#3386](https://github.com/Safe-DS/Library/issues/3386)
* labels of correlation heatmap ([#894](#894)) ([a88a609](a88a609)), closes [#871](#871)
* make multi-processing in baseline models more consistent ([#909](#909)) ([fa24560](fa24560)), closes [#907](#907)

### Performance Improvements

* improved performance in various methods in `Image` and `ImageList` ([#879](#879)) ([134e7d8](134e7d8))
@lars-reimann
Copy link
Member

🎉 This PR is included in version 0.27.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@lars-reimann lars-reimann added the released Included in a release label Jul 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released Included in a release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Functional table transformer
4 participants