Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] convertDecimal64ToDecimal32Wrapper leaks ColumnView instances #2355

Closed
jlowe opened this issue May 6, 2021 · 0 comments · Fixed by #2392
Closed

[BUG] convertDecimal64ToDecimal32Wrapper leaks ColumnView instances #2355

jlowe opened this issue May 6, 2021 · 0 comments · Fixed by #2392
Assignees
Labels
bug Something isn't working

Comments

@jlowe
Copy link
Member

jlowe commented May 6, 2021

com.nvidia.spark.rapids.ParquetPartitionReaderBase#convertDecimal64ToDecimal32Wrapper instantiates ColumnView instances without closing them. For example, child and newChild are both ColumnView instances and neither are closed.

      } else if (dt == DType.LIST) {
        val child = cv.getChildColumnView(0)
        val newChild = convertDecimal64ToDecimal32(child, precision, newCols)
        if (child == newChild) {
          cv
        } else {
          cv.replaceListChild(newChild)
        }
      } else if (dt == DType.STRUCT) {

Each ColumnView corresponds to a native structure that needs to be closed to release the native memory.

@jlowe jlowe added bug Something isn't working ? - Needs Triage Need team to review and classify labels May 6, 2021
@razajafri razajafri self-assigned this May 7, 2021
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label May 11, 2021
@sameerz sameerz added this to the May 10 - May 21 milestone May 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants