Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FEAT-#7021: Implement to/from_dask functions #7022

Merged
merged 10 commits into from
Mar 18, 2024
Merged

FEAT-#7021: Implement to/from_dask functions #7022

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2c2a7e7
FEAT-#7021: Implement to/from_dask_dataframe functions
Retribution98 Mar 7, 2024
b2db4ff
fix after merge
Retribution98 Mar 7, 2024
77cea40
changes after review
Retribution98 Mar 13, 2024
744f50d
check fix dask import
Retribution98 Mar 14, 2024
5cdec20
update to from_dask
Retribution98 Mar 15, 2024
16b51bb
update test
Retribution98 Mar 15, 2024
edc59f7
fix inconsistent changes
Retribution98 Mar 18, 2024
d6e49f4
minor fixes
Retribution98 Mar 18, 2024
3ced01b
remove dask[dataframe] in dependecies
Retribution98 Mar 18, 2024
7183af8
remove import_optional_dependency
Retribution98 Mar 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions modin/core/execution/dask/implementations/pandas_on_dask/io/io.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
ExperimentalPandasXmlParser,
)
from modin.pandas.series import Series
from modin.utils import MODIN_UNNAMED_SERIES_LABEL, import_optional_dependency
from modin.utils import MODIN_UNNAMED_SERIES_LABEL


class PandasOnDaskIO(BaseIO):
Expand Down Expand Up @@ -171,9 +171,6 @@ def to_dask(cls, modin_obj):
dask.dataframe.DataFrame or dask.dataframe.Series
Converted object with type depending on input.
"""
import_optional_dependency(
"dask-expr", "dask-expr is required to create a Dask DataFrame."
)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default message from dask.dataframe is more user friendly, so I suggest not using this method.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also think that we should not check the version for dusk_expr. Yes, it has problems in 1.0.1 version, but this is a problem in Dusk, not our compatibility with it. In addition, this library releases new versions very quickly and, in my opinion, users are unlikely to encounter this problem in the future.

from dask.dataframe import from_delayed

partitions = unwrap_partitions(modin_obj, axis=0)
Expand Down
Loading