Commits


Mark Hildreth authored and François Saint-Jacques committed 17404e161a5
ARROW-8648: [Rust] Optimize Rust CI Workflows The general goal is to decrease the real time needed to run the Rust workflows. See below for details, but in general the build/test part of a workflow can be completed in around 1/3 of the typical time. ## Change examples to run without `--release` flag. During testing, examples were being run with the `--release` flag. Up until that point, everything had been built as debug, and so the examples could not use the existing build artifacts. Thus, an entirely new rebuild (in the slow release profile) would occur. The change is to not use `--release` for examples so that we don't need to rebuild the entire project. ## Ensure all cargo commands share same RUSTFLAGS. A single `RUSTFLAGS="-D warnings"` was being set to build all targets. Then, any subsequent cargo command did not have this flag, meaning cargo would assume it would need a rebuild due to having differing compiler flags. This would mean things like running the tests would cause a rebuild. The change is to ensure that all cargo commands use the same RUSTFLAGS value. ## Just check (don't build) with --no-default-features The `rust_build.sh` script builds `arrow` both normally and with the `--no-default-features` flag. However, `rust_test.sh` will just use whatever the latest artifacts are. Thus, it will run test and examples using the more recent build (`--no-default-features`). I _believe_ what was intended was that building with `--no-default-features` was just to make sure it compiled. Thus, I changed this to use `cargo check` rather than `cargo build`. This will therefore just run the static checking of the crate so as to raise compiler errors, but not actually try to fully compile/link the crate together, gaining some savings. ## Results I tested the improvement locally using the following script in the root: ``` rm -rf build sh ci/scripts/rust_build.sh $PWD $PWD/build sh ci/scripts/rust_test.sh $PWD $PWD/build ``` Here were my results (note: as these were local, they did not take downloading dependencies into account, whereas in CI we do download dependencies every build) | Build Setup (Local Dev Computer) | Avg Three Tries (MM:SS) | | ------------------------------------|--------------------------| | master (no optimizations) | 10:35 | | Examples run in debug mode | 06:04 | | Use consistent RUSTFLAGS | 03:50 | | Only `check` —no-default-features | 03:35 | Here are the results from the CI. * "Time" refers to the sum of the "build" and "test" steps for Mac & Windows, and for the "Docker Run" step for Debian. * "Master" times pulled from a recent master CI build. Specifically, c546eef41e6ab20c4ca29a2d836987959843896f. | Build Setup (CI - Debian) | Time (MM:SS) | | ------------------------------------|--------------- | | Master | 22:50 | | Examples run in debug mode | 13:53 | | Use consistent RUSTFLAGS | 09:07 | | Only `check` —no-default-features | 7:57 | | Build Setup (CI - Mac) | Time (MM:SS) | | ------------------------------------|----------------| | Master | 18:08 | | Examples run in debug mode | 11:03 | | Use consistent RUSTFLAGS | 08:49 | | Only `check` —no-default-features | 06:03 | | Build Setup (CI - Windows) | Time (MM:SS) | | ------------------------------------|----------------| | Master | 28:41 | | Examples run in debug mode | 16:41 | | Use consistent RUSTFLAGS | 10:39 | | Only `check` —no-default-features | 10:17 | Closes #7072 from markhildreth/ARROW-8648 Authored-by: Mark Hildreth <mark.k.hildreth@gmail.com> Signed-off-by: François Saint-Jacques <fsaintjacques@gmail.com>