[VL] Managed memory leak in imported Spark UTs

[zhztheplayer]

Errors with "Managed memory leak detected" are being reported from the import Spark UTs.

The errors were ignored since we set "spark.unsafe.exceptionOnMemoryLeak=false" in tests.

We should deal with all of the current reported leak issues then finally set "spark.unsafe.exceptionOnMemoryLeak=true".

[zhztheplayer]

Tracking:

531 / 6341 UT cases are now leaking

https://gist.github.com/zhztheplayer/40cbdd0be9502fa01d8397eaa087e74a

[zhztheplayer]

Fix #1079 submitted for leak caused by iterator unconsumed data

[zhztheplayer]

Add: Using debugging PR #1033 to count on leaked test cases

[zhztheplayer]

Tracking:

531 / 6341 UT cases are leaking within #1079

Although #1079 fixes leak from some corner cases (e.g. TestOperator.scala which is out of this tracking)

https://gist.github.com/zhztheplayer/9baf7566bd03d570286b9281be567043

[zhztheplayer]

Fix #1084 submitted for leak caused by Java side complex Arrow vector creation

[zhztheplayer]

Tracking:

0 / 6341 UT cases are leaking within #1084

https://gist.github.com/zhztheplayer/b61fbf71af47d114eacb365ca7e33332

[zhztheplayer]

PR #1087 submitted to enable leak error reporting (which was previously warning) in Spark UT

[zhztheplayer]

Closing as resolved.