Multiple failed attempts to Snowflake destination


Airbyte jobs experiencing multiple failed attempts to Snowflake destination, errors include WorkerException, NullPointerException, and pod creation failure. Splitting job into multiple connections resolves the issue. Seeking insights on job volumes and duration.


Hi friends, our Airbyte jobs keep going through multiple attempts to Snowflake destination, we’re on version 0.50.48 and Snowflake version 3.4.2. Last 3 syncs all had 5 failed attempts. Anyone else experiencing this?
Errors are mostly this first error
message='io.temporal.serviceclient.CheckedExceptionWrapper: io.airbyte.workers.exception.WorkerException: Running the launcher replication-orchestrator failed', type='java.lang.RuntimeException', nonRetryable=false
but can also be:
Activity with activityType='RunWithJobOutput' failed: 'Activity task failed'. scheduledEventId=12, startedEventId=13, activityId=81d55ac3-1cbb-3557-8a02-3d636608c78e, identity='1@airbyte-production-worker-7b4cc46864-4hj6q', retryState=RETRY_STATE_MAXIMUM_ATTEMPTS_REACHED
io.airbyte.workers.exception.WorkerException: Failed to create pod for write step
Splitting this job out into 2 or 3 connections seems to cause the issues to go away, but curious what volumes people are pushing through Airbyte and how long your jobs take

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["airbyte", "snowflake-destination", "failed-attempts", "workerexception", "nullpointerexception", "pod-creation-failure", "job-volumes", "job-duration"]