NullPointerException: "schedule" is null

Connector Questions & Issues
,
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: CentOS Linux 7 (Core)
  • Memory / Disk: 12Gb / 1Tb+
  • Deployment: Kubernetes
  • Airbyte Version: 0.40.4 (recently updated from 0.35.10 alpha)
  • Source name/version: MSSQL 0.4.20 (but similar problems with other connectors as well)
  • Destination name/version: Snowflake 0.4.38
  • Step: during sync

Problem
Some of the connections running on our Kubernetes cluster have been failing over the past few days, logging the same exception at different stages of the sync process:

2022-09-30 04:28:57 ERROR i.a.c.l.Exceptions(swallow):66 - Swallowed error.
java.lang.NullPointerException: Cannot invoke "io.airbyte.config.Schedule.getTimeUnit()" because "schedule" is null

Question
Does anyone know what the root cause of this issue could be and how we can solve this?
Additionally, how I could have found the solution myself? Where should I have looked or which documentation pages should I have read?
Thanks in advance for any help.

Example 1
In this example, it failed immediately with mentioned exception during attempt 1. (However, attempt 2 got a different exception, but I would like to focus on mentioned exception first)

logs-6(3).txt (632.8 KB)

image
image1133×458 42.5 KB

image
image1761×459 50.5 KB

Example 2
Finally, here is another example of one of our connections having the same issue:

logs-3826.txt (885.1 KB)

image
image1758×599 59.5 KB

Additional context
We have roughly 20 active connections running on our production kubernetes cluster. All connections are scheduled daily within a 4 hour timeframe.

Normal sync times range between 30 seconds to 2 hours depending on the size of the source data set.

A few weeks ago we updated from v0.35.10(alpha) to v0.40.4. However, the problems I lay out here have only started a few days ago.

All of our connections use a Snowflake destination, but we use multiple different source connectors, including custom ones.

The issue does not seem to lie with a specific source connector, as we have (for example) some MSSQL->Snowflake connections that succeed but also some MSSQL->snowflake that fail.

2

Hello there! You are receiving this message because none of your fellow community members has stepped in to respond to your topic post. (If you are a community member and you are reading this response, feel free to jump in if you have the answer!) As a result, the Community Assistance Team has been made aware of this topic and will be investigating and responding as quickly as possible.
Some important considerations that will help your to get your issue solved faster:

  • It is best to use our topic creation template; if you haven’t yet, we recommend posting a followup with the requested information. With that information the team will be able to more quickly search for similar issues with connectors and the platform and troubleshoot more quickly your specific question or problem.
  • Make sure to upload the complete log file; a common investigation roadblock is that sometimes the error for the issue happens well before the problem is surfaced to the user, and so having the tail of the log is less useful than having the whole log to scan through.
  • Be as descriptive and specific as possible; when investigating it is extremely valuable to know what steps were taken to encounter the issue, what version of connector / platform / Java / Python / docker / k8s was used, etc. The more context supplied, the quicker the investigation can start on your topic and the faster we can drive towards an answer.
  • We in the Community Assistance Team are glad you’ve made yourself part of our community, and we’ll do our best to answer your questions and resolve the problems as quickly as possible. Expect to hear from a specific team member as soon as possible.

Thank you for your time and attention.
Best,
The Community Assistance Team

3

Sorry the delay here Boris, I saw this error in the past but it was fixed in a more recent version. Do you mind upgrading to latest both docker-composeand .envfile?

4

Thank you for the reply @marcosmarxm.

Do I only need to update to the latest docker-compose and .env file, or do I also have to upgrade the content of the kube/resources folder in order to fix this issue? (we deploy to Kubernetes)

5

Oh, you must update all components to latest version (if you’re using Helm or kustomize)