DEPR: pd.read_table

[dahlbaek]

pd.read_table is deprecated and replaced by pd.read_csv.

• closes DEPR: read_table #21948
• tests added / passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

[WillAyd]

Initial comments:

• Needs test(s)
• Since we are replacing read_table with read_csv and a sep of \tyou should reroute the call to read_table to reflect this
• Needs whatsnew note

[pep8speaks]

Hello @dahlbaek! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on August 02, 2018 at 10:48 Hours UTC

[dahlbaek]

Added a whatsnew note.

I'm not sure what to do about tests. pd.read_csv has been around for as long as I've been using pandas, so I would suspect that tests are already plentiful.

I changed the warning message so that it will reflect whether or not the user is supplying a value to sep. The result is now similar to what I proposed in #21948 but without the decorator, as suggested by @gfyoung. Result:

In [1]: import pandas as pd

In [2]: df = pd.DataFrame({"a": [1], "b": [2]})
   ...: df.to_csv("test.csv", sep="\t")

In [3]: pd.read_table("test.csv", index_col=0)
/home/dahlbaek/virtualenvs/pandas-dev/bin/ipython:1: FutureWarning: read_table is deprecated, use read_csv with sep='\t' instead.
  #!/home/dahlbaek/virtualenvs/pandas-dev/bin/python
Out[3]: 
   a  b
0  1  2

In [4]: pd.read_table("test.csv", "\t", index_col=0)
/home/dahlbaek/virtualenvs/pandas-dev/bin/ipython:1: FutureWarning: read_table is deprecated, use read_csv instead.
  #!/home/dahlbaek/virtualenvs/pandas-dev/bin/python
Out[4]: 
   a  b
0  1  2

In [5]: pd.read_table("test.csv", sep="\t", index_col=0)
/home/dahlbaek/virtualenvs/pandas-dev/bin/ipython:1: FutureWarning: read_table is deprecated, use read_csv instead.
  #!/home/dahlbaek/virtualenvs/pandas-dev/bin/python
Out[5]: 
   a  b
0  1  2

In [6]: pd.read_table("test.csv", delimiter="\t", index_col=0)
/home/dahlbaek/virtualenvs/pandas-dev/bin/ipython:1: FutureWarning: read_table is deprecated, use read_csv instead.
  #!/home/dahlbaek/virtualenvs/pandas-dev/bin/python
Out[6]: 
   a  b
0  1  2

In [7]: pd.read_csv("test.csv", "\t", index_col=0)
Out[7]: 
   a  b
0  1  2

In [8]: pd.read_csv("test.csv", sep="\t", index_col=0)
Out[8]: 
   a  b
0  1  2

In [9]: pd.read_csv("test.csv", delimiter="\t", index_col=0)
Out[9]: 
   a  b
0  1  2

[gfyoung]

We definitely have tests for read_table in the codebase. Modify them to assert that a warning is produced.

[dahlbaek]

@gfyoung I modified the tests, but am unsure how to proceed with the stacklevel parameter. It seems to me stacklevel=2 gives the correct behaviour. Since the warning message is hardcoded into the function body, this will give the user the file/line of the call to the function producing the warning message, so the user can make suitable adjustments. Example:

$ cat test_warn.py 
import pandas as pd

pd.read_table("test.csv", index_col=0)
$ python test_warn.py 
test_warn.py:3: FutureWarning: read_table is deprecated, use read_csv with sep='\t' instead.
  pd.read_table("test.csv", index_col=0)

This is exactly the behaviour I would want as a user. Nevertheless, pandas.util.testing.assert_produces_warning complains that the stacklevel is not set correctly, which causes failed tests.

Please Advise.

[gfyoung]

@dahlbaek : Pass in check_stacklevel=False in your tests.

[dahlbaek]

@dahlbaek : Pass in check_stacklevel=False in your tests.

Perfect, that's what I figured.

[gfyoung]

Why do we need this if condition here?

(yes, I see the other changes that you made for passing in default_sep. This question is more asking why the overall change was needed for this deprecation).

[dahlbaek]

If the user is setting sep or delimiter, I want to suggest using read_csv, but if neither is set, I also want to suggest passing sep='\t'. This was the simplest solution I could come up with which handles the case correctly where the user is passing sep=',' to read_table.

More generally, the if clause is needed because the definition of read_table is entangled with the definition of read_csv. The cleaner solution would be to just have two separate definitions, but that solution was deemed not worth the effort.

[gfyoung]

Hmm...I see. Okay, fair enough.

[dahlbaek]

some comments. can you run the test suite and make sure you are catching all deprecation warnings for read_table.

I'm not seeing any FutureWarnings from read_table, but I am getting the following failed test:

(pandas-dev) $ pwd
/home/dahlbaek/git/pandas/pandas/tests
(pandas-dev) $ pytest test_errors.py
=============================== test session starts ===============================
platform linux -- Python 3.6.6, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /home/dahlbaek/git/pandas, inifile: setup.cfg
plugins: xdist-1.22.2, forked-0.2, cov-2.5.1
collected 12 items                                                                

test_errors.py ..........F.                                                 [100%]

==================================== FAILURES =====================================
________________________________ test_error_rename ________________________________

    def test_error_rename():
        # see gh-12665
        from pandas.errors import ParserError
        from pandas.io.common import CParserError
    
        try:
            raise CParserError()
        except ParserError:
            pass
    
        try:
            raise ParserError()
        except CParserError:
            pass
    
        with catch_warnings(record=True):
            try:
>               raise ParserError()
E               pandas.errors.ParserError

test_errors.py:52: ParserError
======================= 1 failed, 11 passed in 0.07 seconds =======================
(pandas-dev) $ 

Not sure exactly what is happening here…

[gfyoung]

@dahlbaek : That's an odd test failure. Seems to be fine on Travis though. Also, the test logs don't seem to have any deprecation warnings related to read_table AFAICT.

[jreback]

@dahlbaek we don't usually run from the test dir itself, rather from the base dir e.t.
pytest path/to/test. running from the test dir might not find the fixtures.

[jreback]

@jorisvandenbossche @TomAugspurger any objections?

[dahlbaek]

@jreback After recreating my virtual environment from scratch, I still get that failed test, even on master, when running from the base dir. Not sure what the cause is, but as pointed out by @gfyoung it seems to be a local problem on my machine.

[jreback]

@dahlbaek see my comments above. The tests are not designed to be run from subdirs, nor are they on our CI. pytest can't find things.

[dahlbaek]

@jreback I think I am now running the test the way proposed in the docs, right? I still get the failure (even on master). In any case, this seems unrelated to the current PR. I suppose it is a local phenomenon that I will have to figure out myself…

(pandas-dev) $ pytest pandas/tests/test_errors.py -k test_error_rename
=============================== test session starts ===============================
platform linux -- Python 3.6.6, pytest-3.6.3, py-1.5.4, pluggy-0.6.0
rootdir: /home/dahlbaek/git/pandas, inifile: setup.cfg
plugins: xdist-1.22.3, forked-0.2, cov-2.5.1
collected 12 items / 11 deselected                                                

pandas/tests/test_errors.py F                                               [100%]

==================================== FAILURES =====================================
________________________________ test_error_rename ________________________________

    def test_error_rename():
        # see gh-12665
        from pandas.errors import ParserError
        from pandas.io.common import CParserError
    
        try:
            raise CParserError()
        except ParserError:
            pass
    
        try:
            raise ParserError()
        except CParserError:
            pass
    
        with catch_warnings(record=True):
            try:
>               raise ParserError()
E               pandas.errors.ParserError

pandas/tests/test_errors.py:52: ParserError
===================== 1 failed, 11 deselected in 0.06 seconds =====================
(pandas-dev) $ 

[jreback]

@dahlbaek hmm that looks like a real error

[dahlbaek]

@jreback I'll try to look into it when I have time and make a separate issue.

[dahlbaek]

@jreback I re-cloned the repo and installed a fresh python3.7 environment. I don't know what was causing the problem, but now it is gone.

[jreback]

@TomAugspurger if you have any issues here

[gfyoung]

@dahlbaek : This looks good to me, though if you could fix the merge conflicts, that would be good.

[dahlbaek]

@gfyoung Done.

[TomAugspurger]

Pushed a doc formatting fix.

Thanks @dahlbaek!