Public
  1. Public

arrow

Public

Commits

Will JonesJonathan Keane
e64480db51f
Will Jones authored and Jonathan Keane committed f0544403b36
ARROW-14029: [R] Repair map_batches()

Updating `map_batches()` function to use `RecordBatchReader` instead of `Scanner$ScanBatches()` so that only one record batch is in memory at a time.

~As part of this, I refactored `do_exec_plan` to always return a RBR instead of a materialized Table.~ I don't think I can refactor `do_exec_plan` to always return a RBR until we get `arrange`, `head`, and `tail` operations to work outside of a sink node. See: https://issues.apache.org/jira/browse/ARROW-15271

Closes #11894 from wjones127/ARROW-14029-r-map-batches

Authored-by: Will Jones <willjones127@gmail.com>
Signed-off-by: Jonathan Keane <jkeane@gmail.com>