Issue with OSS Helm Chart setup for Airbyte BigQuery source creation

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

The user is facing a 502 error while testing a BigQuery source creation from the UI in Airbyte. The issue seems to be related to the webserver failing to find the airbyte-minio-svc. The user tried to reapply the chart with custom values, but encountered errors while trying to create a bucket with Minio. The logs show errors related to ‘no such host’ when trying to access airbyte-minio-svc.


Question

Hi all,

Question about the OSS helm chart setup. I am creating a bigquery source from the UI, but when I test it the progress bar eventually throws a 502 error. When I inspect the logs, I see that the webserver fails to find the airbyte-minio-svc. I tried to reapply the chart with my values and it hangs there trying to create a bucket with minio (not sure which bucket it’s refering to), but when I look at the create-bucket pod, I see these errors

...waiting...
mc: &lt;ERROR&gt; Unable to initialize new alias from the provided credentials. Get "<http://airbyte-minio-svc:9000/probe-bsign-fmthuefpcg3dsz1uoeozvknhm1lu4v/?location=>": dial tcp: lookup airbyte-minio-svc on 34.118.224.10:53: no such host.
...waiting...
mc: &lt;ERROR&gt; Unable to initialize new alias from the provided credentials. Get "<http://airbyte-minio-svc:9000/probe-bsign-gofcrij24b4em6gz13ih2bq5k3zwko/?location=>": dial tcp: lookup airbyte-minio-svc on 34.118.224.10:53: no such host.
...waiting...```
and I still don't see the minio-svc
```╰─ kubectl get svc                                                                                                                                  ─╯
NAME                                           TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)        AGE
airbyte-airbyte-connector-builder-server-svc   NodePort       34.118.239.32    &lt;none&gt;         80:32219/TCP   2d16h
airbyte-airbyte-server-svc                     ClusterIP      34.118.235.62    &lt;none&gt;         8001/TCP       2d16h
airbyte-airbyte-webapp-svc                     LoadBalancer   34.118.233.33    &lt;external_ip&gt;  80:32351/TCP   2d16h
airbyte-temporal                               ClusterIP      34.118.239.188   &lt;none&gt;         7233/TCP       2d16h
airbyte-workload-api-server-svc                ClusterIP      34.118.225.22    &lt;none&gt;         8007/TCP       2d16h```
This is the log from the helm upgrade command
```╰─ helm upgrade --debug --install airbyte airbyte/airbyte --values values.yaml                                                                      ─╯
history.go:56: [debug] getting history for release airbyte
upgrade.go:158: [debug] preparing upgrade for airbyte
upgrade.go:166: [debug] performing update for airbyte
upgrade.go:364: [debug] creating upgraded release for airbyte
...
wait.go:104: [debug] beginning wait for 1 resources to be deleted with timeout of 5m0s
client.go:142: [debug] creating 1 resource(s)
client.go:712: [debug] Watching for changes to Pod airbyte-minio-create-bucket with timeout of 5m0s
client.go:740: [debug] Add/Modify event for airbyte-minio-create-bucket: ADDED
client.go:799: [debug] Pod airbyte-minio-create-bucket pending
client.go:740: [debug] Add/Modify event for airbyte-minio-create-bucket: MODIFIED
client.go:801: [debug] Pod airbyte-minio-create-bucket running
upgrade.go:476: [debug] warning: Upgrade "airbyte" failed: post-upgrade hooks failed: 1 error occurred:
	* timed out waiting for the condition


Error: UPGRADE FAILED: post-upgrade hooks failed: 1 error occurred:
	* timed out waiting for the condition```
Would appreciate any pointers

cc - <@U07C8CCC68Y> :pray:

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1723822709201969) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["oss-helm-chart", "airbyte-bigquery-source", "502-error", "airbyte-minio-svc", "minio-create-bucket", "post-upgrade-hooks-failed"]
</sub>
2

Were you planning on using GCS or was that just to work around the Minio issue?

3

Was planning to use it eventually, so all good. That being said, I found some bugs / misinformation in the chart, cause the GCS storage secret name was being ignored, and the service was looking for a hardcoded secret name instead, even though I specified my own, and that’s how I ended up using credentialsJson as opposed to credentialsPath Anyways, I will collect more evidence and submit an issue, but I’m unblocked now

4

ok I will take a look at that