Multiple source sync to same destination table

Hello, I have multiple postgres sources which I want to sync to same Bigquery Destination Namespace and same table names. I am appending data to the same table names from all those sources. I see that some tmp tables are getting created in between and are then getting deleted by the connection. As the name for the tmp tables from different sources will be same as the table names in those connections are same, I was just wondering can there be an issue while concurrently running these connections. Like one connection deletes tmp tables for another connection as the names will be same. Or can there be any other concurrency issue because the table names are same?

Hello @SP271,
As you described, there will indeed be concurrency issues. I would suggest you use different datasets (namespaces) for each of your connections and create a view that gathers the records from all of these datasets on BigQuery.

Hello @alafanechere Is there any other way to avoid it instead of creating different datasets? Like maybe using BigQuery(Denormalized) or any other way on your mind? There are too many connections that we have to make that will indeed make too many datasets.

Is there any other way to avoid it instead of creating different datasets?

The only workaround I can currently suggest is to make sure a single sync is running anytime to avoid concurrency issues. You can set the MAX_SYNC_WORKERS value to 1.

I raised this feedback to our product team and I hope that we will soon be able to tackle this limitiation.