Data loss in BigQuery destination when updating connection settings

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

When updating a connection setting like ‘sync frequency’ in BigQuery destination, data inserted by other connections is cleared along with the data from the updated connection. Is this normal behavior or a bug?


Question

Hello all,

My issue is that I have many connections to a bigquery destination, whenever I update a connection like “sync frequency” (no stream changes involved) , it clears the data on the destination, even the data inserted by other connections, is this the normal behavior or it should be reported as a bug?

NOTE: all of our production data was cleared.


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["data-loss", "bigquery-connector", "sync-frequency", "data-clearing"]
2

You should try to create separate dataset for each airbyte connection, except if your airbyte data should be in the same dataset.