Commits


Andy Grove authored and Krisztián Szűcs committed 54fcb060428
ARROW-4602: [Rust] [DataFusion] Integrate query optimizer with ExecutionContext Instead of registering `DataSource` with the context, we now register `DataSourceProvider`. This trait has a `scan()` method where we can pass the projection. `ExecutionContext` calls the optimizer rule for all SQL queries now, so that only the necessary columns are loaded from disk. There is also a simpler API for registering CSV files with the context, with a `register_csv` method. I added some criterion benchmarks too but they are not ideal since they load from disk each time. I am working on another PR to add support for running queries against RecordBatches already loaded into memory and will update the benchmarks to use these as part of that PR. Author: Andy Grove <andygrove73@gmail.com> Closes #3678 from andygrove/ARROW-4602 and squashes the following commits: 80ea3c99 <Andy Grove> Integrate projection push down rule with ExecutionContext