Error during BigQuery source stream selection

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

User encountered an HTTP 504 error while selecting streams for the BigQuery source in a self-hosted Airbyte deployment. Logs indicate issues with MetricClient initialization and an unknown keyword in the JSON schema.


Question

Hi everyone!
Here are the steps i’ve done and the problem i got:

  1. Airbyte was deployed self-hosted
  2. All tests were successfully passed when i was configuring BigQuery source
  3. At “Select Streams” steps i got an error: “An unknown error occurred. (HTTP 504)”
  4. After checking the log i found the following:
    2024-11-28 12:04:44,279 [main] WARN i.a.m.l.MetricClientFactory(getMetricClient):43 - MetricClient has not been initialized. Must call MetricClientFactory.CreateMetricClient before using MetricClient. Using a dummy client for now. I
    gnore this if Airbyte is configured to not publish any metrics.
    2024-11-28 12:04:44,346 [main] INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):314 - INFO i.a.i.s.b.BigQuerySource(main):219 starting source: class io.airbyte.integrations.source.bigquery.BigQuerySource
    2024-11-28 12:04:44,347 [main] INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):314 - INFO i.a.c.i.b.IntegrationCliParser(parseOptions):126 integration args: {discover=null, config=/config/connectionConfiguration.json}
    2024-11-28 12:04:44,347 [main] INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):314 - INFO i.a.c.i.b.IntegrationRunner(runInternal):132 Running integration: io.airbyte.integrations.source.bigquery.BigQuerySource
    2024-11-28 12:04:44,347 [main] INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):314 - INFO i.a.c.i.b.IntegrationRunner(runInternal):133 Command: DISCOVER
    2024-11-28 12:04:44,348 [main] INFO i.a.w.i.VersionedAirbyteStreamFactory(internalLog):314 - INFO i.a.c.i.b.IntegrationRunner(runInternal):134 Integration config: IntegrationConfig{command=DISCOVER, configPath=‘/config/connectionCon
    figuration.json’, catalogPath=‘null’, statePath=‘null’}
    2024-11-28 12:04:44,348 [main] WARN i.a.w.i.VersionedAirbyteStreamFactory(internalLog):311 - WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword

Has anybody already encountered with the same problem? I also checked “baseUrl” error, which was discussed, but didn’t it in my logs


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['bigquery', 'http-504', 'metricclient', 'json-schema', 'airbyte']
2

The logs you shared doesn’t contain an ERROR message. Do you mind upload the file to further investigation?