Summary
When trying to perform an incremental load via Airbyte connector from Snowflake to BigQuery, the user is facing issues specifying a start_date for incremental sync due to memory errors. They previously encountered similar problems with full load. Looking for possible resolutions.
Question
Hello, we are using an airbyte connector to load data from a snowflake table to BigQuery.
We have an XS environment on Snowflake and a table with 95GB of data. Loading was taking too long and gets interrupted with a memory error. We understand that the environment configuration is too small, and we decided to do a full load separately.
Now that we want to make an incremental load via Airbyte, it seems that we have the same issue. We cannot see where to specify a start_date for incremental sync.
Have you already had this issue before ? Any possible resolutions ?
Thanks
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.