-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
test(method): improved performance in coloc tests #536
Conversation
@ireneisdoomed do you mind looking at this one when you have time? Returning a list of dataframes looks a bit off. |
@d0choa will do! |
tests/gentropy/conftest.py
Outdated
def sample_data_for_coloc(spark: SparkSession) -> list[Any]: | ||
"""Sample data for Coloc tests.""" | ||
overlap_df = spark.read.parquet( | ||
"tests/gentropy/data_samples/coloc_test_data.snappy.parquet" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How was this file generated? For semantic tests, it's easier to understand if you create a data subset in the testing module directly.
Instead of reading a file of 500 rows, create a dataframe with 2 overlapping variants, for example.
The same testing function can be parametrised for both scenarios: associations that overlap on multiple SNPs, and on a single SNP.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was directly extracted from the test dataset from the R package
* test(coloc): define fixtures and parametrise coloc tests * test(coloc): compare dfs with assert_frame_equal
@xyg123 Do we need to update the expected results or can this be merged? |
Yes! I think so! |
✨ Context
Prev. implemented coloc unit tests were taking ~140 seconds, now using pytest, the coloc unit test time is ~45 secs.
🛠 What does this PR implement
Read in coloc test data using conftest.py.
🙈 Missing
None.
🚦 Before submitting
dev
branch?make test
)?poetry run pre-commit run --all-files
)?