Failed to fetch streams from S3 Connector

Connector Questions & Issues
1

  • OS Version / Instance: Ubuntu

  • Deployment: Kubernetes ?

  • Airbyte Version: 0.35.31-alpha

  • Source name/version: S3/0.1.10

  • Destination name/version: Redshift/0.3.28

  • Step: Discover Schema while making Connection

  • Description:
    We are using S3 as a source connector, source gets configured successfully but it’s failing on fetching the streams (discover schema). While debugging the issue, found the following ERROR in
    worker logs (Block Size is set to Default Value - 10000):

    2022-05-19 11:52:39 ERROR i.a.w.p.a.DefaultAirbyteStreamFactory(internalLog):95 - Detected mismatched datatype on column 'id', in file 'myfile.csv'. Should be 'integer', but found 'string'.

But if we set block size to 100 or 1000, it works perfectly fine.

Thanks!

2

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.