-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor load_file tests #194
Comments
@dimberman the description mentions Also, I think it is important to mention that These lines were the first place where we adopted the new integration tests approach: What is missing, IMPOV is to refactor these lines: Finally, I don't think we should be aiming to have a single test function per operator. I'd vote for us to have three independent tests than this type of fixture: We are at risk of overusing fixtures, which can compromise the readability of the tests. |
@dimberman, I think I can partially work on this, refactoring testcases which are having conflicting table names. |
Closes: #194 - [x] Each test should work across all databases - [x] Use test_utils.run_dag - [x] Use PyTest fixtures and parameterize to have a single main test that will validate transform across multiple databases - [x] Loading against all formats (csv, parquet, avro, etc.) -- **Partially done, since it's not required for all tests.** - [x] Loading to a temp_table or named table -- **Partially done, since it's not required for all tests.** - [x] Loading to default schema and to named schema -- **Partially done, since it's not required for all tests.**
Closes: #194 - [x] Each test should work across all databases - [x] Use test_utils.run_dag - [x] Use PyTest fixtures and parameterize to have a single main test that will validate transform across multiple databases - [x] Loading against all formats (csv, parquet, avro, etc.) -- **Partially done, since it's not required for all tests.** - [x] Loading to a temp_table or named table -- **Partially done, since it's not required for all tests.** - [x] Loading to default schema and to named schema -- **Partially done, since it's not required for all tests.**
Description
As a step towards "maturing" the astro DAG authoring project, we must rewrite our tests to ensure that every integration test runs against every database.
This step will simultaneously reduce the number of tests we need to maintain, make testing much simpler as we add new databases, and will make a future refactor much simpler as we can ensure proper coverage.
To do this, we will take advantage of two features in pytest, fixtures and parameterize.
For this ticket, we will update the
append
function.Acceptance criteria
Have a single test file validating
append
across all databases.pytest.marker.integration
:Table
andTempTable
as inputstest_utils.run_dag
fixtures
andparameterize
to have a single main test that will validate transform across multiple databasesThe tests should validate these scenarios:
The text was updated successfully, but these errors were encountered: