Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dataframe.dropna() doesn`t work correctly at MODIN_STORAGE_FORMAT=omnisci #3933

Closed
Rubtsowa opened this issue Jan 11, 2022 · 5 comments
Closed
Labels
bug 🦗 Something isn't working HDK Related to HDK (OmniSci successor) engine or backend P3 Very minor bugs, or features we can hopefully add some day. pandas concordance 🐼 Functionality that does not match pandas

Comments

@Rubtsowa
Copy link
Contributor

When I run:

import pandas
import modin.pandas as pd
import modin.config as cfg
cfg.StorageFormat.put('omnisci')
from modin.pandas.test.utils import (test_data_values, axis_values)

data = test_data_values[1]
axis = axis_values[2]
modin_df = pd.DataFrame(data)
pandas_df = pandas.DataFrame(data)
modin_result = modin_df.dropna(axis=axis)
pandas_result = pandas_df.dropna(axis=axis)
df_equals(modin_result, pandas_result)

I have error:

AssertionError: DataFrame are different

           DataFrame shape mismatch
           [left]:  (0, 0)
           [right]: (256, 0)
@Rubtsowa Rubtsowa added bug 🦗 Something isn't working HDK Related to HDK (OmniSci successor) engine or backend labels Jan 11, 2022
@Rubtsowa Rubtsowa changed the title Not correct work of dataframe.dropna() at MODIN_STORAGE_FORMAT=omnisci dataframe.dropna() is not working correctly at MODIN_STORAGE_FORMAT=omnisci Jan 14, 2022
@Rubtsowa Rubtsowa changed the title dataframe.dropna() is not working correctly at MODIN_STORAGE_FORMAT=omnisci dataframe.dropna() doesn`t work correctly at MODIN_STORAGE_FORMAT=omnisci Jan 17, 2022
@vnlitvinov vnlitvinov added pandas concordance 🐼 Functionality that does not match pandas P3 Very minor bugs, or features we can hopefully add some day. labels Aug 29, 2022
@vnlitvinov
Copy link
Collaborator

@modin-project/modin-omnisci could someone please check if this still happens?

@AndreyPavlenko
Copy link
Collaborator

Should be fixed by #4910

@YarShev
Copy link
Collaborator

YarShev commented Sep 27, 2022

@AndreyPavlenko, #4910 is merged. Should we close this issue?

@AndreyPavlenko
Copy link
Collaborator

The issue is not reproducible after the merge, just the following warning is printed:
UserWarning: HDK's dropna does not support 'thresh' and 'axis' parameters. is not currently supported by ExperimentalHdkOnNative, defaulting to pandas implementation.

@YarShev
Copy link
Collaborator

YarShev commented Sep 27, 2022

Okay, that looks like a separate feature to implement so I am closing this issue.

@YarShev YarShev closed this as completed Sep 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🦗 Something isn't working HDK Related to HDK (OmniSci successor) engine or backend P3 Very minor bugs, or features we can hopefully add some day. pandas concordance 🐼 Functionality that does not match pandas
Projects
None yet
Development

No branches or pull requests

4 participants