You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi,
We would like to add the support for timestamp in Velox Parquet reader. It was noticed there were two kinds of timestamp types used in Parquet files, the INT96 type and the INT64 type (for TIMESTAMP_MILLIS or TIMESTAMP_MICROS). Which one is Velox's plan on timestamp reader for Parquet?
Although INT96 is deprecated in Parquet (see PARQUET-323), Spark uses it by default for timestamp during Parquet writing. Therefore, we firstly drafted this PR(#4680) to support a timestamp reader to read from INT96-typed timestamps, and would like to hear your opinions and suggestions. Thanks!
The text was updated successfully, but these errors were encountered:
@rui-mo we definitely need to support INT96 timestamp type for legacy Parquet writers and Parquet files. I will review the PR. Thanks for raising this issue.
Description
Hi,
We would like to add the support for timestamp in Velox Parquet reader. It was noticed there were two kinds of timestamp types used in Parquet files, the INT96 type and the INT64 type (for TIMESTAMP_MILLIS or TIMESTAMP_MICROS). Which one is Velox's plan on timestamp reader for Parquet?
Although INT96 is deprecated in Parquet (see PARQUET-323), Spark uses it by default for timestamp during Parquet writing. Therefore, we firstly drafted this PR(#4680) to support a timestamp reader to read from INT96-typed timestamps, and would like to hear your opinions and suggestions. Thanks!
The text was updated successfully, but these errors were encountered: