Commits


Matt Corley authored and Antoine Pitrou committed 1018a4f50fb
ARROW-9517: [C++/Python] Add support for temporary credentials to S3Options ### Background AWS provides a mechanism for using temporary credentials to access AWS resources. When accessing AWS resources with a set of temporary credentials,[ users must provide a session token](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html) in addition to the usual access key id and secret access key. This PR adds support for providing a session token when initializing an S3FileSystem. Additionally, this PR adds support for auto-refreshing temporary credentials via STS AssumeRole: instead of passing explicit credentials, users supply the arn of a role to assume, and an `STSAssumeRoleCredentialsProvider` will be created to handle fetching temporary credentials by assuming this role. ### Changes #### C++ - updated `S3Options.FromAccessKey` and `S3Options.ConfigureAccessKey` to accept an optional `session_token` argument (defaulting to empty string, in accordance with the convention of [Aws::Auth::AWSCredentials](https://sdk.amazonaws.com/cpp/api/0.12.9/d4/d27/class_aws_1_1_auth_1_1_a_w_s_credentials.html) as implemented in the AWS C++ SDK.) - updated `S3FileSystem` implementation to initialize S3Client with CredentialsProvider instead of directly passing AwsCredentials (enabling auto-refreshing of AwsCredentials from provider) - added `S3Options.GetSessionToken` method - added `S3Options.FromAssumeRole` and `S3Options.ConfigureAssumeRoleCredentials` to support auto-refreshing temporary credentials via STS AssumeRole. #### Python - updated cdef `CS3Options` to reflect updates to C++ library - added optional `session_token` argument to `S3FileSystem.__init__`. ### Testing - updated `test_s3_options` python unittest to include a mock session_token when initializing S3FileSystem. - updated `test_s3_options` python unittest to include relevant scenarios for provided a `role_arn` when initializing S3FileSystem. - updated `s3fs_test.cc` c++ tests to cover changes to FromAccessKey as well as the newly added FromAssumeRole. - successfully ran ctest suite and pytest suite with `--enable-s3` option. Closes #7803 from corleyma/dev/ARROW-9517-add-s3-option-session-token Authored-by: Matt Corley <mattc@23andme.com> Signed-off-by: Antoine Pitrou <antoine@python.org>