Skip to content

Commit

Permalink
BUG: to_csv line endings with compression (#25625)
Browse files Browse the repository at this point in the history
  • Loading branch information
chris-b1 authored and jorisvandenbossche committed Mar 11, 2019
1 parent c4fa3c9 commit 43b949d
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 1 deletion.
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.24.2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ Fixed Regressions
- Fixed regression in creating a period-dtype array from a read-only NumPy array of period objects. (:issue:`25403`)
- Fixed regression in :class:`Categorical`, where constructing it from a categorical ``Series`` and an explicit ``categories=`` that differed from that in the ``Series`` created an invalid object which could trigger segfaults. (:issue:`25318`)
- Fixed pip installing from source into an environment without NumPy (:issue:`25193`)
- Fixed regression in :meth:`DataFrame.to_csv` writing duplicate line endings with gzip compress (:issue:`25311`)

.. _whatsnew_0242.enhancements:

Expand Down
2 changes: 1 addition & 1 deletion pandas/io/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -434,7 +434,7 @@ def _get_handle(path_or_buf, mode, encoding=None, compression=None,
if (compat.PY3 and is_text and
(compression or isinstance(f, need_text_wrapping))):
from io import TextIOWrapper
f = TextIOWrapper(f, encoding=encoding)
f = TextIOWrapper(f, encoding=encoding, newline='')
handles.append(f)

if memory_map and hasattr(f, 'fileno'):
Expand Down
12 changes: 12 additions & 0 deletions pandas/tests/frame/test_to_csv.py
Original file line number Diff line number Diff line change
Expand Up @@ -1221,3 +1221,15 @@ def test_multi_index_header(self):
'1,5,6,7,8']
expected = tm.convert_rows_list_to_csv_str(expected_rows)
assert result == expected

def test_gz_lineend(self):
# GH 25311
df = pd.DataFrame({'a': [1, 2]})
expected_rows = ['a', '1', '2']
expected = tm.convert_rows_list_to_csv_str(expected_rows)
with ensure_clean('__test_gz_lineend.csv.gz') as path:
df.to_csv(path, index=False)
with tm.decompress_file(path, compression='gzip') as f:
result = f.read().decode('utf-8')

assert result == expected

0 comments on commit 43b949d

Please sign in to comment.