You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
val path = "/tmp/data/"
spark.range(1).selectExpr("id AS c", "id AS f")
.write.mode("overwrite").parquet(s"$path")
spark.read.parquet(s"$path").show()
Analysis
This is because Spark will generate two parquet files. One has one row data. The other has no data, but with meta data.
Velox parquet reader will encounter issue, when parse the second parquet file.
The text was updated successfully, but these errors were encountered:
zhixingheyi-tian
changed the title
VeloxRuntimeError when read parquet file with only meta data
VeloxRuntimeError when reading parquet file with only meta data
Dec 27, 2022
Describe the bug
This issue is exposed from :
Error:
To Reproduce
Analysis
This is because Spark will generate two parquet files. One has one row data. The other has no data, but with meta data.
Velox parquet reader will encounter issue, when parse the second parquet file.
The text was updated successfully, but these errors were encountered: