Commits
Krisztián Szűcs authored and Wes McKinney committed 23475eeddac
ARROW-3367: [INTEGRATION] Port Spark integration test to the docker-compose setup
This incorporates the fixes from https://github.com/apache/arrow/pull/3254
Running it on a local "Docker for Mac" takes ages because of its volume virtualization and the excessive IO usage of maven. So currently I need to run it on travis...
Crossbow builds [here](https://github.com/kszucs/crossbow/branches/all?utf8=%E2%9C%93&query=spark-integration)
Author: Krisztián Szűcs <szucs.krisztian@gmail.com>
Author: Kousuke Saruta <sarutak@oss.nttdata.com>
Closes #3300 from kszucs/spark_integration and squashes the following commits:
c594cd38 <Krisztián Szűcs> fix wrong patch; spark version tag pointed to a different commit...
e13d2277 <Krisztián Szűcs> fix a pyspark test again
4de2c853 <Krisztián Szűcs> fix another pyspark test issue
68756b63 <Krisztián Szűcs> mvn fu
70028d22 <Krisztián Szűcs> java requires integration folder
2c8334d1 <Krisztián Szűcs> setup.py fuuuuu
cb8a438b <Krisztián Szűcs> use rsync...
31ce522d <Krisztián Szűcs> custom build dir for for java
bb30fe1a <Krisztián Szűcs> download spark into the image
fd54e8a2 <Krisztián Szűcs> avoid bind mounts
05a934f6 <Krisztián Szűcs> remove spark_integration entry from the old docker-compose.yml
11b41abf <Krisztián Szűcs> warn level
f1b74e9c <Krisztián Szűcs> silenting
20cf16cc <Krisztián Szűcs> cleanup spark dockerfile
3c088edb <Krisztián Szűcs> compile only sql/core and assemply spark packages; don't parallelize mvn builds
c0b24e60 <Krisztián Szűcs> patch spark 2.4.0
c0aa8ebc <Krisztián Szűcs> less verbose maven
927eac60 <Krisztián Szűcs> fix docker-compose syntax error
60171459 <Krisztián Szűcs> remove notes from docker-compose
b91eae08 <Krisztián Szűcs> remove commented lines
6a6e1d67 <Krisztián Szűcs> Revert "silent docker build outputs on nightly builds"
341adb44 <Krisztián Szűcs> silent installation steps
d6543661 <Krisztián Szűcs> silent docker build outputs on nightly builds
7703b503 <Krisztián Szűcs> add nightly test entry
cdb4f414 <Krisztián Szűcs> spark integration test
41df92d8 <Kousuke Saruta> Fix spark_integration/{Dockerfile,spark_integration.sh} for spark-integration test to work