Commits

Andy Grove authored f4ccceb2536
ARROW-10995: [Rust] [DataFusion] Limit ParquetExec concurrency when reading large number of files Before this PR, we were launching one thread per Parquet file in ParquetExec. This works well when the number of files is small but does not scale well when there are hundreds of files. This PR respects the context's `max_concurrency` configuration option and uses it to split the files up into lists per partition/thread. Closes #9029 from andygrove/limit-parquet-concurrency Authored-by: Andy Grove <andygrove73@gmail.com> Signed-off-by: Andy Grove <andygrove73@gmail.com>