S3 destination makes multiple parts of a parquet file

Hi guys!

I’m checking that before the last pull request for S3 destination #15394 the connection is creating multiple parts of the parquet file.

I don’t find in the configuration the option to change it to defined number of parts or a single file.

The option of part_size_mb was deprecated in https://github.com/airbytehq/airbyte/pull/13753 this was necessary to achieve parallel stream syncs to S3 and other bucket services.

1 Like