feat: make add_lags dataframe-agnostic
#661
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
MarcoGorelli
commented
May 10, 2024
sklego/pandas_utils.py
Outdated
Comment on lines
346
to
348
| combos = (df[col].shift(-lag).rename(col + str(lag)) for col in cols for lag in lags) | ||
| answer = df.with_columns([nw.col(col).shift(-lag).alias(col + str(lag)) for col in cols for lag in lags]) | ||
|
|
||
| answer = pd.concat([df, *combos], axis=1) |
There was a problem hiding this comment.
this diff is surprisingly pleasing
MarcoGorelli
commented
May 10, 2024
Comment on lines
-65
to
+75
| def test_log_step(capsys, test_df): | ||
| def test_log_step(capsys, data): | ||
| """Base test of log_step without any arguments to the logger""" | ||
| test_df = pd.DataFrame(data) |
There was a problem hiding this comment.
I haven't worked on log_step in this PR, so for now I'm just keeping the test_log_step tests to pandas
MarcoGorelli
force-pushed
the
add-lags-df-agnostic
branch
from
a3dab2e to
539e657
Compare
MarcoGorelli
changed the title
add_lags dataframe-agnosticadd_lags dataframe-agnostic
🤔 not sure what's going on with tests in CI EDIT: same thing is happening for PRs against main: #662 |
FBruzzesi
reviewed
May 10, 2024
| raise KeyError("The column does not exist") | ||
|
|
||
| combos = (df[col].shift(-lag).rename(col + str(lag)) for col in cols for lag in lags) | ||
| answer = df.with_columns(nw.col(col).shift(-lag).alias(col + str(lag)) for col in cols for lag in lags) |
There was a problem hiding this comment.
Any plan for the .name namespace? Asking for a friend 😂
I can take a look myself during the weekend 😇
There was a problem hiding this comment.
definitely! nw.col(col).shift(-lag).name.suffix(str(lag)) would be even nicer
Closed
FBruzzesi
reviewed
May 10, 2024
.github/workflows/test.yml
Outdated
| @@ -28,6 +28,6 @@ jobs: | |||
| with: | |||
| python-version: ${{ matrix.python-version }} | |||
| - name: Install dependencies | |||
| run: python -m pip install -e ".[test-all]" | |||
| run: python -m pip install "cvxpy<1.5.0" -e ".[test-all]" | |||
There was a problem hiding this comment.
If you merge main there should be no need for this pinning now
MarcoGorelli
force-pushed
the
add-lags-df-agnostic
branch
from
5717d84 to
b8d5fca
Compare
FBruzzesi
approved these changes
May 10, 2024
koaning
pushed a commit
that referenced
this pull request
May 24, 2024
* placeholder to develop narwhals features * feat: make `ColumnDropper` dataframe-agnostic (#655) * feat: make ColumnDropped dataframe-agnostic * use narwhals[polars] in pyproject.toml, link to list of supported libraries * note that narwhals is used for cross-dataframe support * test refactor * docstrings --------- Co-authored-by: FBruzzesi <francesco.bruzzesi.93@gmail.com> * feat: make ColumnSelector dataframe-agnostic (#659) * columnselector with test rufformatted * adding whitespace * fixed the fit and transform * removed intendation in examples * font:false * feat: make `add_lags` dataframe-agnostic (#661) * make add_lags dataframe-agnostic * try getting tests to run? * patch: cvxpy 1.5.0 support (#663) --------- Co-authored-by: Francesco Bruzzesi <42817048+FBruzzesi@users.noreply.github.com> * Make `RegressionOutlier` dataframe-agnostic (#665) * make regression outlier df-agnostic * need to use eager-only for this one * pass native to check_array * remove cudf, link to check_X_y * feat: Make InformationFilter dataframe-agnostic * Make Timegapsplit dataframe-agnostic (#668) * make timegapsplit dataframe-agnostic * actually, include cuDF * feat: make FairClassifier data-agnostic (#669) * start all over * fixture working * wip * passing tests - again * pre-commit complaining * changed fixture on test_demographic_parity * feat: Make PandasTypeSelector selector dataframe-agnostic (#670) * make pandas dtype selector df-agnostic * bump version * 3.8 compat * Update sklego/preprocessing/pandastransformers.py Co-authored-by: Francesco Bruzzesi <42817048+FBruzzesi@users.noreply.github.com> * fixup pyproject.toml * unify (and test!) error message * deprecate * update readme * undo contribution.md change --------- Co-authored-by: Francesco Bruzzesi <42817048+FBruzzesi@users.noreply.github.com> * format typeselector and bump version * feat: Make grouped and hierarchical dataframe-agnostic (#667) * feat: make grouped and hierarchical dataframe-agnostic * add pyarrow * narwhals grouped_transformer * grouped transformer eureka * hierarchical narwhalified * so close but so far * return series instead of DataFrame for y * grouped WIP * merge branch and fix grouped * future annotations * format * handling negative indices * solve conflicts * hacking C * fairness: change C values in tests --------- Co-authored-by: Marco Edward Gorelli <marcogorelli@protonmail.com> Co-authored-by: Magdalena Anopsy <74981211+anopsy@users.noreply.github.com> Co-authored-by: Dea María Léon <deamarialeon@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before working on a large PR, please check with @FBruzzesi or @koaning to confirm that they agree with the direction of the PR. This discussion should take place in a Github issue before working on the PR, unless it's a minor change like spelling in the docs.
Description
add_lags is now dataframe-agnostic - this one was surprisingly easy
Fixes #(issue)
Type of change
Checklist:
If you feel your PR is ready for a review, ping @FBruzzesi or @koaning.