Data Loading Issue with Custom Connector in Airbyte Cloud

Summary

Data loading from a custom connector in Airbyte Cloud is taking longer than expected, job remains in ‘Running’ state despite successful data write to database. Troubleshooting needed.


Question

Hi - I’m using Airbyte Cloud to load data from a custom connector (nothing too special, simple REST API with basic auth, no pagination or rate limiting). I have a data source with <100 records that is taking over an hour to run, which is delaying some downstream data processing that are dependent on jobs finishing. The logs indicate that data is successfully written to my database, but the job persists in a ‘Running’ state.
I’ve tried deleting and recreating the connection, and cancelling the job is not an option as it will not allow me to exit a running state. I’ve also done extensive connector and source testing, where I have not had any issues.

Any pointers on troubleshooting this? :grimacing:



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["airbyte-cloud", "custom-connector", "data-loading", "troubleshooting"]