Commits


Benjamin Kietzman authored and Joris Van den Bossche committed 9d7dca643fe
ARROW-8074: [C++][Dataset][Python] FileFragments from buffers and NativeFiles Adds `ds.FileSource`, which represents an openable file and may be initialized from a `path, filesystem`, a `Buffer`, or any python object which can be wrapped by `NativeFile`. `test_parquet.py` now uses `BytesIO` as the roundtrip medium for non legacy `ParquetDataset` instead of resorting to a mock filesystem. Other than that the integration with Python is somewhat haphazard; I'm thinking we need to rewrite some of the APIs to be less magical about figuring out what is a selector, path, list(paths), etc since we will be adding buffers and `NativeFile`s to the mix. Closes #7156 from bkietz/8047-FileFragments-from-NativeFile Lead-authored-by: Benjamin Kietzman <bengilgit@gmail.com> Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com> Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>