You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Dataset API requires implementations to fully apply the filters before returning batches to the caller. The current Vortex implementation instead tends to error if there are unrecognized expressions.
If this fails, we run a full scan with post-filtering.
Partition into filters that can / cannot be pushed down
Run any filters that cannot be pushed down using PyArrow prior to returning.
We should also make the Dataset API configurable as to whether or not it uses "view" types like StringView and ListView. We should disable this by default until PyArrow compute has better support for these arrays.