Commits


Antoine Pitrou authored and Wes McKinney committed 9f0c70c8337
ARROW-2447: [C++] Device and MemoryManager API Add an abstraction layer to allow safe handling of buffers residing on different devices (the CPU, a GPU...). The layer exposes two interfaces: * the `Device` interface exposes information a particular memory-holding device * the `MemoryManager` allows allocating, copying, reading or writing memory located on a particular device The `Buffer` API is modified so that calling `data()` fails on non-CPU buffers. A separate `address()` method returns the buffer address as an integer, and is allowed on any buffer. The API provides convenience functions to view or copy a buffer from one device to the other. For example, a on-GPU buffer can be copied to the CPU, and in some situations a zero-copy CPU view can also be created (depending on the GPU capabilities and how the GPU memory was allocated). An example use in the PR is IPC. On the write side, a new `SerializeRecordBatch` overload takes a `MemoryManager` argument and is able to serialize data either to any kind of memory (CPU, GPU). On the read side, `ReadRecordBatch` now works on any kind of input buffer, and returns record batches backed by either CPU or GPU memory. It introduces a slight complexity in the CUDA namespace, since there are both `CudaContext` and `CudaMemoryManager` classes. We could solve this by merging the two concepts (but doing so may break compatibility for existing users of CUDA). Closes #6295 from pitrou/ARROW-2447-device-api-memory-manager and squashes the following commits: c665f6177 <Antoine Pitrou> ARROW-2447: Device and MemoryManager API Authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Wes McKinney <wesm+git@apache.org>