Is this your first time deploying Airbyte : No
OS Version / Instance : EC2 t3a.xlarge
Memory / Disk : 16Gb
Deployment : Docker
Airbyte Version : 0.35.64-alpha
Source name/version : File 0.2.9
Destination name/version : Redshift 0.3.28
Step : Setting new connection, source / On sync
Description : Multiple sources from SFTP over to Redshift are failing on first sync. These do work just fine to a MySQL destination, but trying to move them over to Redshift now. Does not effect every SFTP connection, but a large number of them are now failing. Any ideas what might be wrong
logs-5132.txt (21.5 KB)
It seems that if I delete the connection, and then set it up fresh, and instead of full refresh / overwrite, I choose to append, it seems to run. Any thoughts on what might cause that?
Hi @krishagel,
Did you ran a refresh source schema after changing your destination?
According to this error in the logs:
java.lang.IllegalArgumentException: Message contained record from a stream that was not in the catalog. the schema used by the destination is not up to date with the source schema.
When deleting the connection you basically achieved the same thing.
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.