Skip to content

Commit

Permalink
Change example test to parquet from csv
Browse files Browse the repository at this point in the history
Signed-off-by: wmoreiraa <walber3@gmail.com>
  • Loading branch information
wmoreiraa authored and astrojuanlu committed Jun 19, 2023
1 parent 771a33d commit 65c5b61
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions kedro-datasets/kedro_datasets/polars/generic_dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,12 @@ class GenericDataSet(AbstractVersionedDataSet[pl.DataFrame, pl.DataFrame]):
.. code-block:: yaml
cars:
type: polars.GenericDataSet
file_format: csv
filepath: s3://data/01_raw/company/cars.csv
file_format: parquet
filepath: s3://data/01_raw/company/cars.parquet
load_args:
sep = ","
low_memory: True
save_args:
date_format: "%Y-%m-%d"
compression: "snappy"
Example usage for the
`Python API <https://kedro.readthedocs.io/en/stable/data/\
Expand All @@ -62,7 +62,7 @@ class GenericDataSet(AbstractVersionedDataSet[pl.DataFrame, pl.DataFrame]):
>>> data = pl.DataFrame({'col1': [1, 2], 'col2': [4, 5],
>>> 'col3': [5, 6]})
>>>
>>> data_set = GenericDataSet(filepath="test.csv", file_format='csv')
>>> data_set = GenericDataSet(filepath="test.parquet", file_format='parquet')
>>> data_set.save(data)
>>> reloaded = data_set.load()
>>> assert data.frame_equal(reloaded)
Expand Down

0 comments on commit 65c5b61

Please sign in to comment.