Commits
Wes McKinney authored af2aeafca6b
ARROW-1213: [Python] Support s3fs filesystem for Amazon S3 in ParquetDataset cc @yackoa Author: Wes McKinney <wes.mckinney@twosigma.com> Closes #916 from wesm/ARROW-1213 and squashes the following commits: f8a0aff1 [Wes McKinney] Add HDFS section to API docs c54302df [Wes McKinney] Add deprecation warning for HdfsClient 4d3e7222 [Wes McKinney] Auto-wrap s3fs filesystem when using ParquetDataset 0be33bb8 [Wes McKinney] Implement os.walk emulation layer for s3fs 719f806d [Wes McKinney] Progress toward supporting s3fs in Parquet reader bbd664ed [Wes McKinney] Refactor HdfsClient into pyarrow/hdfs.py. Add connect factory method. Rename to HadoopFilesystem. Add walk implementation for HDFS, base Parquet directory walker on that 4984a9d4 [Wes McKinney] Refactoring slightly 4c0bcf4a [Wes McKinney] Start on Dask filesystem wrapper, S3-Parquet dataset test case