Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for get_partitions_metadata #1

Merged
merged 2 commits into from
Oct 21, 2020

Conversation

jtcohen6
Copy link
Collaborator

@jtcohen6 jtcohen6 commented Oct 20, 2020

Description

  • Use get_partitions_metadata in existing integration tests for BigQuery table partition switching
  • Re-add asterisk to raw_execute
  • I started on (but did not complete) using adapter.get_partitions_metadata in the bq_insert_overwrite macro, instead of select max({{ partition_by.field }}) from {{ this }}, to save GBs—our initial goal, after all

Testing

Run this test via Docker + tox. You'll also need a JSON keyfile set to an env var named BIGQUERY_SERVICE_ACCOUNT_JSON in a file called test.env.

docker-compose run test tox -e explicit-py36 -- -s -x -m profile_bigquery test/integration/022_bigquery_test/test_bigquery_changing_partitions.py

@jtcohen6 jtcohen6 requested a review from ran-eh October 20, 2020 13:55
Copy link
Owner

@ran-eh ran-eh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@ran-eh ran-eh merged commit c31ba10 into re-partition-metadata Oct 21, 2020
jtcohen6 pushed a commit that referenced this pull request Oct 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants