Cannot load gzip compressed csv files from S3

Hi, everyone

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Mac
  • Memory / Disk: 250GB
  • Deployment: Docker
  • Airbyte Version: 0.39.1-alpha
  • Source name/version: S3
  • Destination name/version: Clickhouse
  • Step: The issue is happening when creating the connection.
  • Description: I have gzip files(CSV) in a S3 bucket that I want to sync to my Clickhouse using Airbyte. I successfully created an S3 source in the Airbyte UI, although I was unable to select GZIP as “File Format (The format of the files you’d like to replicate)”. I tried selecting CSV, parquet and Avro but (logically) my sync returned 0 records.

Hello and welcome to the Airbyte community, @Sergei_Kapochkin! It looks like gzip as a format is not available for the S3 source connector. Would you be interested in making a feature request on our GitHub?