Commits


Weston Pace authored and GitHub committed fc950192ae7
GH-34410: [Python] Allow chunk sizes larger than the default to be used (#34435) ### Rationale for this change We changed the default chunk size from 64Mi rows to 1Mi rows. However, it turns out that this property was being treated not just as the default but also as the absolute max. So it was no longer possible to specify chunk sizes larger than 1Mi rows. This change separates those two things and restores the max to 64Mi rows. ### What changes are included in this PR? Pyarrow will now set the `ParquetWriterProperties::max_row_group_length` to 64Mi when constructing a parquet writer. ### Are these changes tested? Yes. Unit tests are added. ### Are there any user-facing changes? No. The previous change #34281 changed two defaults (absolute max and default). This PR restores the absolute max back to what it was before. So it is removing a user-facing change. * Closes: #34410 Lead-authored-by: Weston Pace <weston.pace@gmail.com> Co-authored-by: Will Jones <willjones127@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>