Issue with dbt after deleting legacy table in BigQuery

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

Migrating to destinations V2 on BigQuery, facing downtime on final table in dbt after deleting legacy table. Questioning if downtime is intended and how to orchestrate accordingly.


Question

Hey team, we’re migrating to destinations V2 (2.3.25 so far) on BigQuery, it went completely smooth up until i decided to delete a legacy table containing the raw data.
It seems that we now have “downtime” on the final table, as dbt isn’t able said table from time to time.
Is it intended ? Should we expect downtime and orchestrate accordingly ?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["migrating", "destinations-v2", "bigquery", "legacy-table", "downtime", "dbt", "orchestration"]
2

cc <@U0540CQ23N0> <@U021H4HQ3PW>

3

Do you have some logs to share maybe?

4

Yes, here they are for airbyte(check out the timestamps)

2024-02-29 12:10:30 destination &gt; 2024-02-29 12:10:30 INFO i.a.i.b.d.t.TypeAndDedupeTransaction(executeTypeAndDedupe):36 - Attempting typing and deduping for airbyte_hubspot.association_deals with suffix
2024-02-29 12:10:52 destination &gt; 2024-02-29 12:10:52 INFO i.a.i.b.d.t.DefaultTyperDeduper(lambda$typeAndDedupeTask$4):244 - Allowing other threads to proceed for airbyte_hubspot.association_deals
2024-02-29 12:10:52 destination &gt; 2024-02-29 12:10:52 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onCloseFunction$4):179 - Cleaning up destination started for 1 streams```
Attached is the bq timestamp, and then the dbt error timestamp