Commits


Weston Pace authored and David Li committed 61d7cd37046
ARROW-11883: [C++] Add ConcatMap, MergeMap, and an async-reentrant version of Map These items can all stand on their own and they are used by the async datasets conversion. MergeMap - Given AsyncGenerator<AsyncGenerator<T>> return AsyncGenerator<T>. This method flattens a generator of generators into a generator of items. It may reorder the items. ConcatMap - Same as MergeMap but it will only pull items from one inner subscription at a time. This reduced parallelism allows items to be returned in-order. Async-reentrant Map - In some cases the map function is slow. Even if the source is not async-reentrant this map can still be async-reentrant by allowing multiple instances of the map function to run at once. The resulting mapped generator is async reentrant but it will not pull reentrantly from the source. Vector utilities - In order to make migrating from Iterator code to vector code easier I added some map style utilities. These copy the vectors (where an iterator wouldn't) so some care should be taken but they can still be useful. Moved Future/AsyncGenerator into top level type_fwd. This is needed for the RecordBatchGenerator alias in the same way Iterator is needed at the top level. Added `IsEnd` to `IterationTraits`. This allows non-comparable types to be iterated on. It allows us to create an AsyncGenerator<AsyncGenerator<T>> since AsyncGenerator is std::function and we can use an empty instance as an end token even though std::function is not comaprable. Closes #9643 from westonpace/feature/arrow-11883 Authored-by: Weston Pace <weston.pace@gmail.com> Signed-off-by: David Li <li.davidm96@gmail.com>