Source MSSQL connector hang forever intermittently (NPE)

Connector Questions & Issues
,
1
  • Is this your first time deploying Airbyte?: Yes
  • OS Version / Instance: EKS
  • Memory / Disk: 4-8GB
  • Deployment: Kubernetes deployment
  • Airbyte Version: 0.39.16
  • Source name/version: source-mssql 0.4.2
  • Destination name/version: [destination-snowflake] 0.4.28
  • Step: The issue is happening during sync intermittently and it runs forever until manually cancel and rerun the job.
  • Description:I have created CDC sync from mssql to snowflake and job just hang sometime with nullpointer exception. Could you please shed some light on this ?

Expected:

Job should not fail or fail with some meaningful error and then retry

Error:

at java.lang.Thread.run(Thread.java:833) [?:?]

Caused by: io.airbyte.workers.exception.WorkerException: Error while getting checking connection.
at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:86) ~[io.airbyte-airbyte-workers-0.39.16-alpha.jar:?]
at io.airbyte.workers.general.DefaultCheckConnectionWorker.run(DefaultCheckConnectionWorker.java:29) ~[io.airbyte-airbyte-workers-0.39.16-alpha.jar:?]
at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.16-alpha.jar:?]
… 1 more
Caused by: java.lang.NullPointerException
at java.io.Reader.(Reader.java:168) ~[?:?]
at java.io.InputStreamReader.(InputStreamReader.java:112) ~[?:?]
at io.airbyte.commons.io.IOs.newBufferedReader(IOs.java:119) ~[io.airbyte-airbyte-commons-0.39.16-alpha.jar:?]

2

Hey wasn’t this resolved in https://discuss.airbyte.io/t/mssql-cdc-connector-hangs-for-long-time/1323/7

3

Previous issue is resolved after upgrade. This looks different issue as rerun works fine and error messages are not same as well . I am rerunning the job if it stuck with null pointer exception (cancel and syn now). Any pointer which case this exception occurs. I faced 2 times in a day and still monitoring the job.

4

Got it. Could you share the complete logs? Also let us know if it succeeds after you rerun

1 Like
5

logs-5892.txt (8.6 KB)

Rerun gets success hence, I am using airflow to monitor the run and if it runs more than 30 minutes then cancel the job as there is. no timeout feature in airbyte scheduler ( it suppose to run couple of minutes)

6

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.