Issue with S3 connector in pyairbyte

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

When using pyairbyte to connect the S3 source connector and extract .csv files from AWS S3, the entire sync fails if one CSV file cannot be read by Airbyte. Is there a way to handle this issue?


Question

Hi !subteam^S01SHT8BSCU|@dev-zazmic, I am trying to connect the source S3 connector using pyairbyte and I am trying to extract the contents of all the .csv files in AWS S3. It’s mostly working fine, but if I have a CSV file which cannot be read by Airbyte, the entire sync is failing instead of just that one file failing and rest of the CSV files still syncing fine. Am I missing something? :slightly_smiling_face:


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["s3-connector", "pyairbyte", "csv-files", "sync-failure", "aws-s3"]