You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are adding APIs to efficiently convert the data stored in Parquet's "PageIndex" into ArrayRefs -- which will make it significantly easier to use this information for pruning and other tasks.
Describe the solution you'd like
Add support to StatisticsConverter::min_page_statistics and StatisticsConverter::max_page_statistics for the types above
Is your feature request related to a problem or challenge?
Part of #10922
We are adding APIs to efficiently convert the data stored in Parquet's "PageIndex" into
ArrayRef
s -- which will make it significantly easier to use this information for pruning and other tasks.Describe the solution you'd like
Add support to
StatisticsConverter::min_page_statistics
andStatisticsConverter::max_page_statistics
for the types abovedatafusion/datafusion/core/src/datasource/physical_plan/parquet/statistics.rs
Lines 637 to 656 in a923c65
Describe alternatives you've considered
You can follow the model from @Weijun-H in #10931
Check::Both
, following the model oftest_int64
datafusion/datafusion/core/tests/parquet/arrow_statistics.rs
Lines 506 to 529 in a923c65
get_datapage_statistics
:datafusion/datafusion/core/src/datasource/physical_plan/parquet/statistics.rs
Line 624 in 459afbb
datafusion/datafusion/core/src/datasource/physical_plan/parquet/statistics.rs
Line 90 in 2f43476
Typically the change to the test looks like
Additional context
No response
The text was updated successfully, but these errors were encountered: