BigQuery Materialized Views Error After Airbyte Full Refresh

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

After a Full Refresh in Airbyte, BigQuery queries using materialized views yield an error due to table recreation. Looking for a workaround solution.


Question

Hi folks -

My team uses Airbyte to sync to BigQuery. We want to utilize BigQuery materialized views to improve query cost and performance. Problem is, after a Full Refresh airbyte sync, BigQuery queries using mat views will yield the error Materialized view references table XXX which was deleted and recreated. The view must be deleted and recreated as well. This results in a service disruption. I’m curious if anyone here has figured out a clever workaround for this. Thanks in advance!


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["bigquery", "materialized-views", "full-refresh", "airbyte-sync", "workaround"]
2

Maybe a custom transform? I use Redshift late binding views to get around this, not sure if BigQuery has something similar

3

Thanks for the suggestion, Jeff