We have an airbyte implementation on K8 that is deployed using Kustomize. Occassionally the web takes a very long time to load anything,(connections page, sources page, etc.) We noticed this issue on more than 1 cluster.
Hey could you help with more information like airbyte version, number of connections you are using?
We are using the latest version, 0.39.38-alpha , we have around 100 Connections right now. However this issue was there when we had 3 connections as well. We also compared docker-compose with K8 using 100 connections and there was no delay in the UI at all. On K8 it occasionally takes a min to load sources or connections pages.
Note that the server/pod utilization is very low
Hey do you find any errors in the console?
I can see a lot of warnings in the webapp pod such as
2022/08/01 17:17:35 [warn] 38#38: *8182 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/4/51/0000000514 while reading upstream
And i can see a lot of this as well on the server pod, even though the server has internet access and can successfully ping github
2022-08-01 19:40:34 WARN i.a.s.s.AirbyteGithubStore(getLatestSources):68 - Unable to retrieve latest Source list from Github. Using the list bundled with Airbyte. This warning is expected if this Airbyte cluster does not have internet access.
java.net.http.HttpTimeoutException: request timed out
So does the k8s cluster have internet access
Yup it does, several connectors that we are using rely on internet connectivity to get the data and they’re working fine
Got it. Would suggest you to redo the deployment by deleting the pods and see if that works. Also once that is done could you share the logs file here