Specifying start_date for incremental sync in Airbyte connector


When trying to perform an incremental load via Airbyte connector from Snowflake to BigQuery, the user is facing issues specifying a start_date for incremental sync due to memory errors. They previously encountered similar problems with full load. Looking for possible resolutions.


Hello, we are using an airbyte connector to load data from a snowflake table to BigQuery.
We have an XS environment on Snowflake and a table with 95GB of data. Loading was taking too long and gets interrupted with a memory error. We understand that the environment configuration is too small, and we decided to do a full load separately.
Now that we want to make an incremental load via Airbyte, it seems that we have the same issue. We cannot see where to specify a start_date for incremental sync.
Have you already had this issue before ? Any possible resolutions ?

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["airbyte-connector", "incremental-sync", "snowflake", "bigquery", "start-date", "memory-error"]