Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

read_csv with Ray engine fails with parse_dates parameter passed as dict #2073

Closed
amyskov opened this issue Sep 14, 2020 · 4 comments · Fixed by #2884 or #4572
Closed

read_csv with Ray engine fails with parse_dates parameter passed as dict #2073

amyskov opened this issue Sep 14, 2020 · 4 comments · Fixed by #2884 or #4572
Assignees
Labels
P1 Important tasks that we should complete soon Testing 📈 Issues related to testing

Comments

@amyskov
Copy link
Contributor

amyskov commented Sep 14, 2020

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
  • Modin version (modin.__version__): 0.7.3+350.g2ca3f34
  • Python version: 3.8.5
  • Code we can use to reproduce:
import os
os.environ["MODIN_ENGINE"] = "ray"

import pandas
import modin.pandas as pd

test_filename = "test.csv"
csv_single_col = """col1,col2
2000-01-01,2000-01-02
2000-01-03,2000-01-04
"""

try :
    with open(test_filename, "w") as f:
        f.write(csv_single_col)

    df_pandas = pandas.read_csv(test_filename, parse_dates={"foo": ["col1", "col2"]})
    print(df_pandas)
    df_pd = pd.read_csv(test_filename, parse_dates={"foo": ["col1", "col2"]})
    repr(df_pd)
    print(df_pd)
finally:
    os.remove(test_filename)

Describe the problem

Source code / logs

                         foo
0  2000-01-01 20:00:00-02:00
1  2000-01-03 20:00:00-04:00
Traceback (most recent call last):
  File "test_read_csv.py", line 20, in <module>
    repr(df_pd)
  File "/localdisk/amyskov/modin2/modin/pandas/dataframe.py", line 169, in __repr__
    result = repr(self._build_repr_df(num_rows, num_cols))
  File "/localdisk/amyskov/modin2/modin/pandas/base.py", line 108, in _build_repr_df
    return self.iloc[indexer]._query_compiler.to_pandas()
  File "/localdisk/amyskov/modin2/modin/pandas/indexing.py", line 377, in __getitem__
    result = super(_iLocIndexer, self).__getitem__(row_lookup, col_lookup, ndim)
  File "/localdisk/amyskov/modin2/modin/pandas/indexing.py", line 129, in __getitem__
    qc_view = self.qc.view(row_lookup, col_lookup)
  File "/localdisk/amyskov/modin2/modin/backends/pandas/query_compiler.py", line 2470, in view
    self._modin_frame.mask(row_numeric_idx=index, col_numeric_idx=columns)
  File "/localdisk/amyskov/modin2/modin/engines/base/frame/data.py", line 472, in mask
    return self.copy()
  File "/localdisk/amyskov/modin2/modin/engines/base/frame/data.py", line 643, in copy
    return self.__constructor__(
  File "/localdisk/amyskov/modin2/modin/engines/base/frame/data.py", line 73, in __init__
    ErrorMessage.catch_bugs_and_request_email(
  File "/localdisk/amyskov/modin2/modin/error_message.py", line 51, in catch_bugs_and_request_email
    raise Exception(
Exception: Internal Error. Please email bug_reports@modin.org with the traceback and command that caused this error.
Column widths: 2 != 1
@amyskov amyskov added the bug 🦗 Something isn't working label Sep 14, 2020
@devin-petersohn devin-petersohn added this to the bugs and regressions milestone Mar 16, 2021
devin-petersohn added a commit to devin-petersohn/modin that referenced this issue Mar 17, 2021
Resolves modin-project#2073

Co-authored-by: William Ma <12377941+williamma12@users.noreply.github.com>

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
williamma12 pushed a commit that referenced this issue Mar 18, 2021
Resolves #2073

Co-authored-by: William Ma <12377941+williamma12@users.noreply.github.com>

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
@YarShev
Copy link
Collaborator

YarShev commented Mar 19, 2021

Reopening the issue because of this.

@YarShev YarShev reopened this Mar 19, 2021
@mvashishtha
Copy link
Collaborator

This seems to work on Modin version d231df0. We need to add a test case for it.

@mvashishtha mvashishtha added Testing 📈 Issues related to testing and removed bug 🦗 Something isn't working labels Jun 13, 2022
@mvashishtha mvashishtha self-assigned this Jun 13, 2022
@pyrito pyrito added the P2 Minor bugs or low-priority feature requests label Aug 22, 2022
@pyrito
Copy link
Collaborator

pyrito commented Aug 22, 2022

@mvashishtha What's the status of this?

@mvashishtha
Copy link
Collaborator

I still have to address comments on #4572: #4572 (comment)

@mvashishtha mvashishtha added P1 Important tasks that we should complete soon and removed P2 Minor bugs or low-priority feature requests labels Aug 22, 2022
mvashishtha pushed a commit to mvashishtha/modin that referenced this issue Dec 3, 2022
Signed-off-by: mvashishtha <mahesh@ponder.io>
anmyachev pushed a commit that referenced this issue Dec 3, 2022
Signed-off-by: mvashishtha <mahesh@ponder.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 Important tasks that we should complete soon Testing 📈 Issues related to testing
Projects
None yet
5 participants