Commits


Pearu Peterson authored and Antoine Pitrou committed d904106b259
ARROW-3451: [C++/Python] pyarrow and numba CUDA interop This PR implements the following features: 1. `pyarrow.cuda.Context.handle` property returns `CUcontext` value 2. `pyarrow.cuda.Context.from_numba` method to create `Context` that shares numba context handler 3. `pyarrow.cuda.Context.to_numba` method to create a numba context instance that shares `Context` handler 4. `pyarrow.cuda.foreign_buffer` method to create CudaBuffer of device memory defined by address and size. Resolved ARROW-3451. 5. `pyarrow.cuda.CudaBuffer.to_numba` method to create a numba `MemoryPointer` instance so that `CudaBuffer` memory is used within the arguments to numba jitted functions. 6. `pyarrow.cuda.CudaBuffer.from_numba` method to create a CudaBuffer view of device data represented by numba `MemoryPointer`. 7. `CudaDeviceManager::CreateSharedContext` used in 2. Resolves ARROW-1423. 8. `CudaContext::View` used in 4. Author: Pearu Peterson <pearu.peterson@gmail.com> Author: Kouhei Sutou <kou@clear-code.com> Author: Antoine Pitrou <pitrou@free.fr> Author: Pindikura Ravindra <ravindra@dremio.com> Closes #2732 from pearu/arrow-cuda and squashes the following commits: b503b0ca <Antoine Pitrou> Update docstring for GetContext b6f4ede2 <Pearu Peterson> Remove contexts_ cache and related methods CreateNewContext and CreateSharedContext. 8457e006 <Pearu Peterson> Use C++ style cast. fc5f5058 <Pearu Peterson> Expose CudaContext device number for CudaBufferWriter in buffering mode. 39d3f69f <Pearu Peterson> Introduce CloseIpcBuffer, fixes IPC test. 767df424 <Pearu Peterson> Introduce device argument to AllocateHost. Use sync in IPC test. 3e99126e <Pearu Peterson> Implemented context push-pop pattern. 2nd try. 1b0c19ee <Pearu Peterson> Implemented context push-pop pattern. f9e69b2c <Pearu Peterson> Fix check-format issues. 415cbd6e <Pearu Peterson> Introduce synchronize methods. c546ce51 <Pearu Peterson> Synchronize before copying device data to host. cfb1e74a <Pearu Peterson> Fix lint issue. a6d53765 <Pearu Peterson> Introduce ContextSaver. Fix doc strings. Set fixed seed for cuda tests. cc87f3b4 <Pearu Peterson> Fix lint issues. f27796a1 <Pearu Peterson> Restore previous context after cuCtxSetCurrent. Remove use_numba_context. c1b57177 <Pearu Peterson> Fix bug that enabled using numba context manager always. 42fc2324 <Pearu Peterson> Fix make lint failures. bdc00cee <Pearu Peterson> Introduce GetSharedContext and use_numba_context. f5764747 <Pearu Peterson> Fix compiler warning re casting pointers with different sizes. 1bf7856c <Pearu Peterson> Moved numba jitted function inside the test function. 740e4747 <Pearu Peterson> Apply feedback from pitrou cbe73382 <Pearu Peterson> Fix lint-clang formatting issues. 0920105a <Pearu Peterson> lost it git messages 14789098 <Pearu Peterson> Implement a view of device memory (CudaBuffer.foreign_buffer) 1107aade <Pearu Peterson> pyarrow and numba CUDA interop 3a1e5645 <Pearu Peterson> Fix lint issues. a42ffd1a <Pearu Peterson> Merge branch 'arrow-cuda' of github.com:Quansight/arrow into arrow-cuda ef69f35a <Pearu Peterson> Implement interop between pyarrow and numba. cba9bcf5 <Pearu Peterson> Merging with upstream/master 50aebe65 <Pearu Peterson> Implement interop between pyarrow and numba. 4e20aa9e <Kouhei Sutou> ARROW-3438: Fix too much Markdown escape in CHANGELOG 5284ad07 <Pindikura Ravindra> ARROW-3331: Add re2 to toolchain cf3aebe6 <Kouhei Sutou> ARROW-3432: Expand variables in commit message b6294015 <Kouhei Sutou> ARROW-3431: Include Gemfile to archive f16148f4 <Kouhei Sutou> ARROW-3430: Add workaround to verify 0.11.0