Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use "float64" dtypes in DaskExecutor.fit_phase #378

Merged
merged 6 commits into from
Jul 26, 2024

Conversation

rjzamora
Copy link
Contributor

@rjzamora rjzamora commented Jul 26, 2024

Addresses NVTabular test failures with rapids-24.04 (pandas-2).

This PR does not "fix" the existing dtype-preservation problem:

            # TODO: constructing meta like this loses dtype information on the ddf
            # and sets it all to 'float64'. We should propagate dtype information along
            # with column names in the columngroup graph. This currently only
            # happens during intermediate 'fit' transforms, so as long as statoperators
            # don't require dtype information on the DDF this doesn't matter all that much

However, it makes pandas-2 behavior consistent with pandas<2.

Copy link

Documentation preview

https://nvidia-merlin.github.io/core/review/pr-378

@rjzamora rjzamora added the bug Something isn't working label Jul 26, 2024
@rjzamora rjzamora changed the title Preserve dtypes in DaskExecutor.fit_phase Use "float64" dtypes in DaskExecutor.fit_phase Jul 26, 2024
Copy link
Member

@benfred benfred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@rjzamora rjzamora merged commit 6d396aa into NVIDIA-Merlin:main Jul 26, 2024
17 checks passed
@rjzamora rjzamora deleted the preserve-dtypes branch July 26, 2024 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants