Summary
Error related to null value for temporal connection timeout in Airbyte logs and setting ephemeral storage requests/limits for Airbyte pods in Kubernetes.
Question
Hey all, I’ve got a couple of possibly unrelated questions.
We are deploying Airbyte on kubernetes via the helm charts.
I’m trying to track down an error in our logs based on what I believe is a default value not getting set for the temporal connection timeout. Below is the error, and I’ve attached the logs from the worker pod. Jobs have completed with this error, but I’m trying to track down if it is indicative of a problem we’ll see later.
Caused by: java.lang.NullPointerException: Cannot invoke "java.time.Duration.toMinutes()" because the return value of "io.airbyte.workers.temporal.check.connection.CheckConnectionActivity.getCheckConnectionTimeout()" is null
I was thinking a quick solution could be to pass a connection timeout so there would be a default value.
My other question is how to pass ephemeral-storage requests and limits to pods created during jobs such as source-postgres-check, destination-snowflake-write and other pods. Without a resource request the Airbyte pods are scheduled on nodes with very little ephemeral storage and they get evicted.
Any help would be greatly appreciated, thanks!
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.