Enviroment
- Airbyte version : 0.29.15-alpha
- OS Version / Instance : MacOs BigSur
- Deployment : Docker
- Source Connector and version : Microsoft Sql Server
- Destination Connector and version : BigQuery
- Severity : High
- Step where error happened : Full refresh | Override
Current Behavior
I want to use an existing dataset on BigQuery. After sync starts, the connector is able to create the tmp table into the existing dataset and just after, I receive Dataset not found
Any idea how to fix that ?
2021-09-08 15:35:44 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-09-08 15:35:44 e[32mINFOe[m i.a.i.d.b.BigQueryUtils(createTable):112 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=deli, tableId=_airbyte_tmp_auc_ppm_stagingbi_T_Operations}} created successfully
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - Exception in thread "main" com.google.cloud.bigquery.BigQueryException: 404 Not Found
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - POST https://www.googleapis.com/upload/bigquery/v2/projects/veepee-data-developer-sandbox/jobs?uploadType=resumable
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - {
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - "error": {
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - "code": 404,
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - "message": "Not found: Dataset veepee-data-developer-sandbox:deli",
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - "errors": [
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - {
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - "message": "Not found: Dataset veepee-data-developer-sandbox:deli",
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - "domain": "global",
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - "reason": "notFound"
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - }
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - ],
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - "status": "NOT_FOUND"
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - }
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - }
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 -
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.translate(HttpBigQueryRpc.java:115)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.open(HttpBigQueryRpc.java:655)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.bigquery.TableDataWriteChannel$2.call(TableDataWriteChannel.java:87)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.bigquery.TableDataWriteChannel$2.call(TableDataWriteChannel.java:82)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.bigquery.TableDataWriteChannel.open(TableDataWriteChannel.java:81)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.bigquery.TableDataWriteChannel.<init>(TableDataWriteChannel.java:41)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at com.google.cloud.bigquery.BigQueryImpl.writer(BigQueryImpl.java:1388)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at io.airbyte.integrations.destination.bigquery.BigQueryDestination.getConsumer(BigQueryDestination.java:234)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:147)
2021-09-08 15:35:44 ERROR () LineGobbler(voidCall):85 - at io.airbyte.integrations.destination.bigquery.BigQueryDestination.main(BigQueryDestination.java:357)