K8s No logs available

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: EKS m5n.xlarge
  • Memory / Disk: 16Gb memory and whatever the default pvc for airbyte is
  • Deployment: Kubernetes
  • Airbyte Version: 0.38.2-alpha
  • Source name/version: Postgres 0.4.14
  • Destination name/version: Snowflake 0.4.25
  • Step: During Sync
  • Description: Logs are missing during and after sync. I can see the log file info (e.g. /tmp/workspace/51/0/logs.log) but the only log line returned is “No logs available.” Maybe there’s a pod that needs restarting?

Are you able to access logs in minio?

if I kubectl exec into the minio pod I don’t see anything under the data/ dir. that’s where the logs usually are correct?

actually, if I look at the scheduler logs i’m seeing an XMinioStorageFull error.

kubectl logs deployments/airbyte-scheduler

Cannot publish to S3: Storage backend has reached its minimum free disk threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull;

You should increase the space for minio and the path is /storagesee example below:

thanks, i’ll give it a try

I finally got around to increasing the minio storage and it works fine now. upping the spec/resources/requests/storage to 2000Mi in kube/resources/airbyte-minio.yaml did the trick. I did have to delete the minio deployment first losing the older logs as the pvc for minio isn’t dynamic.