Commits


Dane Pitkin authored and GitHub committed 35019647b67
GH-36069: [Java] Ensure S3 is finalized on shutdown (#36934) ### Rationale for this change Java datasets can implicitly create an S3 filesystem, which will initialize S3 APIs. There is currently no explicit call to shutdown S3 APIs in Java, which results in a warning message being printed at runtime: `arrow::fs::FinalizeS3 was not called even though S3 was initialized. This could lead to a segmentation fault at exit` ### What changes are included in this PR? * Add a Java runtime shutdown hook that calls `EnsureS3Finalized()` via JNI. This is a noop if S3 is uninitialized or already finalized. ### Are these changes tested? Yes, reproduced with: ``` import org.apache.arrow.dataset.file.FileFormat; import org.apache.arrow.dataset.file.FileSystemDatasetFactory; import org.apache.arrow.dataset.jni.NativeMemoryPool; import org.apache.arrow.dataset.source.DatasetFactory; import org.apache.arrow.memory.BufferAllocator; import org.apache.arrow.memory.RootAllocator; public class DatasetModule { public static void main(String[] args) { String uri = "s3://voltrondata-labs-datasets/nyc-taxi-tiny/year=2022/month=2/part-0.parquet"; try ( BufferAllocator allocator = new RootAllocator(); DatasetFactory datasetFactory = new FileSystemDatasetFactory(allocator, NativeMemoryPool.getDefault(), FileFormat.PARQUET, uri); ) { // S3 is initialized } catch (Exception e) { e.printStackTrace(); } } } ``` I didn't think a unit test was worth adding. Let me know if you think otherwise. Reasoning: * We can't test the actual shutdown since thats a JVM thing. * We could test to see if the hook is registered, but that involves exposing the API and having access to the thread object registered with the hook. Or using reflection to obtain it. Not worth it IMO. * No need to test the functionality inside the hook, its just a wrapper around a single C++ API with no params/retval. ### Are there any user-facing changes? No * Closes: #36069 Authored-by: Dane Pitkin <dane@voltrondata.com> Signed-off-by: David Li <li.davidm96@gmail.com>