Commits


Kevin Gurney authored and Sutou Kouhei committed 98e0b714dd7
ARROW-13100: [MATLAB] Integrate GoogleTest with MATLAB Interface C++ Code ## Overview This Pull Request: 1. **Adds additional options for integrating the Arrow C++ libraries with the MATLAB interface.** 1.1. Users can supply a custom `ARROW_HOME` value to make use of their own, pre-built `libarrow` library. 1.2. If no information is provided, then CMake will attempt to find a system-installed Arrow distribution (e.g. installed via a package manager). 1.3. If no system Arrow libraries are found by `find_package`, and no additional information is provided, then the Arrow C++ libraries will automatically be built from source using `ExternalProject_Add`. 2. **Adds support for GoogleTest to the CMake build system for the MATLAB interface.** 2.1. A user can supply a custom [`GTEST_ROOT`](https://cmake.org/cmake/help/latest/module/FindGTest.html) value to CMake in order to make use of pre-built GoogleTest binaries. 2.2. If no information is provided, then CMake will attempt to find a system-installed GoogleTest distribution (e.g. installed via a package manager). 2.3. If no system GoogleTest libraries are found by `find_package`, and no additional information is provided, then the CMake build system will use the bundled GoogleTest binaries that are built as part of the Arrow C++ libraries ( `ARROW_BUILD_TESTS` is set to `ON` under the hood). 3. **Adds a `MATLAB_BUILD_TESTS` option to the CMake build system for the MATLAB interface.** 3.1. This allows users to enable building of the C++ tests for the MATLAB interface with `cmake .. -D MATLAB_BUILD_TESTS=ON`. 4. **Allows the MATLAB interface and the C++ tests to be built and run using a consistent set of commands on Windows, macOS, and Linux.** ```bash $ cmake -S . -B build -D MATLAB_BUILD_TESTS=ON $ cmake --build build --config Release $ ctest --test-dir build ``` ## Implementation This implementation uses [`ExternalProject_Add`](https://cmake.org/cmake/help/latest/module/ExternalProject.html#command:externalproject_add) to automatically build the Arrow C++ libraries and bundled GoogleTest binaries from source. We took heavy inspiration from the CMake patterns used to create imported library targets in [`ThirdpartyToolchain.cmake`](https://github.com/apache/arrow/blob/master/cpp/cmake_modules/ThirdpartyToolchain.cmake). We followed the [same approach used in the CMake build system for the Arrow C++ libraries](https://github.com/apache/arrow/blob/e2238582e2a2bf20a68a967145fe1a7b2337a997/cpp/cmake_modules/ThirdpartyToolchain.cmake#L1701) and added an automatic step which copies the `gtest.dll` and `gtest_main.dll` runtime libraries to the same folder as the C++ tests for the MATLAB interface, so that they can be found by the [Windows load-time dynamic linker](https://docs.microsoft.com/en-us/windows/win32/dlls/load-time-dynamic-linking). ## Testing 1. These changes were qualified against the following platform/compiler configurations: 1.1. Windows 10 with Visual Studio 2019 1.2. macOS Big Sur (11.2.3) with GNU Make 3.81 1.3. Debian 10 with GNU Make GNU 4.2.1 2. We ran the CMake build with a custom `GTEST_ROOT` value that points to GoogleTest binaries that were built via the CMake `googletest_ep` target from the Arrow C++ libraries. In other words, we built GoogleTest with `cmake -DARROW_BUILD_TESTS=ON ..; make googletest_ep`. 3. We ran the CMake build without specifying a `GTEST_ROOT` value and the GoogleTest binaries that are bundled with the Arrow C++ libraries (when built with `-D ARROW_BUILD_TESTS=ON`) were automatically used. 4. We ran the CMake build with `MATLAB_BUILD_TESTS=ON` and verified that an Arrow installation found by `find_package` (e.g. when an explicit `ARROW_HOME` is specified) is used, instead of the Arrow library built from source, using `ldd` on Linux, `otool -l` and `otool -L` on macOS, and the [`/VERBOSE:LIB`](https://docs.microsoft.com/en-us/cpp/build/reference/verbose-print-progress-messages?view=msvc-160#remarks) linker flag on Windows. We also verified that deleting the `arrow.lib` import library from a custom `ARROW_HOME` resulted in a linker failure on Windows. 5. We built and ran the tests on Windows, macOS, and Linux using the following commands: ``` $ cmake -S . -B build -D MATLAB_BUILD_TESTS=ON $ cmake --build build --config Release $ ctest --test-dir build ``` ## Future Directions 1. As follow up work to this PR, a major priority is to integrate the MATLAB interface with the Arrow CI ecosystem, so that both C++ and MATLAB tests will be automatically run with every pull request. 2. Pending acceptance of this pull request, the MATLAB interface will have C++ testing infrastructure in place. This will allow us to shift focus towards incremental delivery of C++ feature code, along with associated tests. 3. We plan to update the MATLAB interface documentation to clearly explain how to build and run the C++ tests with and without a custom `GTEST_ROOT` value. This work is captured in [ARROW-13204](https://issues.apache.org/jira/browse/ARROW-13204). 4. The current implementation requires the C++ tests for the Arrow C++ libraries to be built in order to use the bundled GoogleTest binaries. Building the Arrow C++ tests takes a nontrivial amount of time and isn't strictly necessary to validate the correctness of the MATLAB interface. In the future, we may want to consider fetching and building the GoogleTest binaries from GitHub separately. One approach to achieve this would be to factor out the [`build_gtest` macro](https://github.com/apache/arrow/blob/e2238582e2a2bf20a68a967145fe1a7b2337a997/cpp/cmake_modules/ThirdpartyToolchain.cmake#L1602) from `ThirdpartyToolchain.cmake` and make it reusable by the MATLAB interface. 5. Automatically copying `gtest.dll` and `gtest_main.dll` to the same folder as the C++ tests during the build process helps new developers get started with less friction on Windows. Copying `arrow.dll` to the same folder as the MEX files may similarly be useful. This would prevent users from having to manually add `arrow.dll` to the Windows `%PATH%`. This would also require us to keep all the MEX files in the build folder. There would be some trade-offs with this approach concerning MEX file organization and discoverability on the MATLAB Path, so this would require a bit more thought. 6. This PR doesn't support debug builds of GoogleTest when building the bundled GoogleTest binaries. We may want to support this in the future. 7. When specifying a custom `GTEST_ROOT`, `gtest.dll` and `gtest_main.dll` won't be automatically copied to the location of the C++ tests. Therefore, they won't be discoverable by the Windows load-time dynamic linker by default. It might be nice to support automatic copying for this use case in the future. ## Notes 1. Thank you to @lafiona for helping me with this pull request! Closes #10614 from kevingurney/ARROW-13100 Authored-by: Kevin Gurney <kgurney@mathworks.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>