You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
yes please @alamb , it would be great if you could; I have been working on more improvements for the parquet arrow reader so my local repo isn't very clean.
Describe the bug
Earlier this week I found that predicate push-down into parquet for Date32 columns was broken in PR #426
I found that this was caused by missing branches in
impl TryFrom<&DataType> for ScalarValue
here https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/scalar.rs#L924which is used in
get_min_max_values
here https://github.com/apache/arrow-datafusion/blob/master/datafusion/src/physical_plan/parquet.rs#L508I also found that adding the following lines into the
try_from
method resolves the issue:To Reproduce
Expected behavior
Statistics column(s) generated for Date32 columns from a parquet data source should not be all null
Additional context
n/a
@alamb
The text was updated successfully, but these errors were encountered: