Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] python UDF tests are leaking #4730

Closed
revans2 opened this issue Feb 9, 2022 · 0 comments · Fixed by #4754
Closed

[BUG] python UDF tests are leaking #4730

revans2 opened this issue Feb 9, 2022 · 0 comments · Fixed by #4754
Assignees
Labels
bug Something isn't working

Comments

@revans2
Copy link
Collaborator

revans2 commented Feb 9, 2022

Describe the bug
As a part of trying to debug host memory usage I found that the UDF tests are leaking badly. Specifically udf_test.py and udf_cudf_test.py

Steps/Code to reproduce bug
Build the plugin and run some of the tests in there with -Dai.rapids.refcount.debug=true then look at the results of what happens.

For example:

PYSP_TEST_spark_master='local[1]' COVERAGE_SUBMIT_FLAGS=-Dai.rapids.refcount.debug=true TEST_PARALLEL=0 SPARK_HOME=$HOME/src/spark_3.2.0/ ./run_pyspark_from_build.sh -k 'udf_test.py and test_group_agg and not more_typesregate_udf and Byte and not more_types' -s 2>&1 | tee single_udf.txt

Showed that 48 column vectors were leaked and one host buffer was leaked. This is just one test, there are 137 tests in those two files and when I run all of them I see over 20,615 items being leaked.

Expected behavior
There are no items that are leaked.

@revans2 revans2 added bug Something isn't working ? - Needs Triage Need team to review and classify labels Feb 9, 2022
@revans2 revans2 self-assigned this Feb 10, 2022
@sameerz sameerz added this to the Jan 31 - Feb 11 milestone Feb 15, 2022
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants