You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Pushdown filtering was prototyped but never fully implemented. One of the challenges is that pushdown filtering requires an "initialization" phase which could be combined as part of opening the file or separate. The reasoning for making it separate is that we might not know, at file open, which fields we need to access and use in filters.
We also need to introduce some kind of metadata cache that schedulers / decoders have access to. This would be populated on initialization (or first read) and could be reused for future reads.
The text was updated successfully, but these errors were encountered:
Pushdown filtering was prototyped but never fully implemented. One of the challenges is that pushdown filtering requires an "initialization" phase which could be combined as part of opening the file or separate. The reasoning for making it separate is that we might not know, at file open, which fields we need to access and use in filters.
We also need to introduce some kind of metadata cache that schedulers / decoders have access to. This would be populated on initialization (or first read) and could be reused for future reads.
The text was updated successfully, but these errors were encountered: