K8s No logs available

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: EKS m5n.xlarge
  • Memory / Disk: 16Gb memory and whatever the default pvc for airbyte is
  • Deployment: Kubernetes
  • Airbyte Version: 0.38.2-alpha
  • Source name/version: Postgres 0.4.14
  • Destination name/version: Snowflake 0.4.25
  • Step: During Sync
  • Description: Logs are missing during and after sync. I can see the log file info (e.g. /tmp/workspace/51/0/logs.log) but the only log line returned is “No logs available.” Maybe there’s a pod that needs restarting?

Are you able to access logs in minio?

if I kubectl exec into the minio pod I don’t see anything under the data/ dir. that’s where the logs usually are correct?

actually, if I look at the scheduler logs i’m seeing an XMinioStorageFull error.

kubectl logs deployments/airbyte-scheduler

Cannot publish to S3: Storage backend has reached its minimum free disk threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull;

You should increase the space for minio and the path is /storagesee example below:

thanks, i’ll give it a try

I finally got around to increasing the minio storage and it works fine now. upping the spec/resources/requests/storage to 2000Mi in kube/resources/airbyte-minio.yaml did the trick. I did have to delete the minio deployment first losing the older logs as the pvc for minio isn’t dynamic.

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.