Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev - pyarrow version - save_args #1

Merged
merged 3 commits into from
Nov 30, 2022
Merged

Dev - pyarrow version - save_args #1

merged 3 commits into from
Nov 30, 2022

Conversation

heber-urdaneta
Copy link
Owner

Description

Development notes

Checklist

  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the relevant RELEASE.md file
  • Added tests to cover my changes

reuse database and warehouse from credentials if not provided with dataset
@Vladimir-Filimonov Vladimir-Filimonov marked this pull request as ready for review November 30, 2022 13:40
@Vladimir-Filimonov
Copy link
Collaborator

made kedro project (kedro 0.18.3, python 3.8) using this code as extras - works fine except pyarrow dependency (kedro comes with pyarrow 6 while showpark needs 8), had to install it manually.
reading and saving data from/to snowflake with service account and SSO account works good. saving 5 mil rows dataset (350Mb CSV) takes ±20 seconds

@Vladimir-Filimonov Vladimir-Filimonov merged commit c5302d4 into main Nov 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants