Public
  1. Public

arrow

Public

Commits

fjetterWes McKinney
b1e56a2f5d3
fjetter authored and Wes McKinney committed ec32013fd6d
ARROW-1417: [Python] Allow more generic filesystem objects to be passed to ParquetDataset

This way, the `ParquetDataset` accepts both `S3FileSystem` and `LocalFileSystem` objects as they are used in `dask`. By using `issubclass`, external libraries may write their own FS wrappers by inheriting from the arrow FS.

I tested the integration with dask and this will fix the issue blocking https://github.com/dask/dask/pull/2527

Author: fjetter <florian.jetter@blue-yonder.com>

Closes #1032 from fjetter/ARROW-1417 and squashes the following commits:

75f18a5 [fjetter] Remove isinstance check in _ensure_filesystem
302b644 [fjetter] Perform check for type object before issubclass
ed111c9 [fjetter] Allow more generic filesystems to be passed