Summary
User encountered issues with wrong files in Google Drive folder affecting Airbyte stream’s primary keys, seeking help on troubleshooting and avoiding the need to create a new Google Drive folder each time.
Question
Airbyte Cloud: Source Connector: Google Drive
Issue: I dumpped wrong files in the Google Drive folder used by my Google Drive Source Connector, and got wrong fields in my Airbyte stream which completely forbade me to run my stream incrementaly (Primary keys fields were missing in the wrong files)
Troubleshoot done:
• I created a new GDrive source connector on Airbyte using the same GDrive folder URL → Somehow the wrong fields were still there and still could not run incremental refresh :man-facepalming:
Solution (Quick fix):
• what fixed my issue was to create a new GDrive folder!
Questions:
• Did anyone experienced a similar issue?
• Did you solved it the same way or did something else?
• Anyone know if GDrive somehow cache files in the same folder even after removing them ?
• Is Airbyte somehow saving data or fields from a given GDrive URL ? I cannot explain why creating a new GDrive source connector (with same folder url) did not solve the issue
Also a more proper solution would be appreciate because I would like to avoid creating new Gdrive folder each time someone make a mistake
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.