- Is this your first time deploying Airbyte?: No
- OS Version / Instance: EKS m5n.xlarge
- Memory / Disk: 16Gb memory and whatever the default pvc for airbyte is
- Deployment: Kubernetes
- Airbyte Version: 0.38.2-alpha
- Source name/version: Postgres 0.4.14
- Destination name/version: Snowflake 0.4.25
- Step: During Sync
- Description: Logs are missing during and after sync. I can see the log file info (e.g. /tmp/workspace/51/0/logs.log) but the only log line returned is “No logs available.” Maybe there’s a pod that needs restarting?
Are you able to access logs in minio?
if I kubectl exec into the minio pod I don’t see anything under the data/ dir. that’s where the logs usually are correct?
actually, if I look at the scheduler logs i’m seeing an XMinioStorageFull error.
kubectl logs deployments/airbyte-scheduler
Cannot publish to S3: Storage backend has reached its minimum free disk threshold. Please delete a few objects to proceed. (Service: Amazon S3; Status Code: 507; Error Code: XMinioStorageFull;
You should increase the space for minio and the path is
/storagesee example below:
thanks, i’ll give it a try
I finally got around to increasing the minio storage and it works fine now. upping the spec/resources/requests/storage to 2000Mi in kube/resources/airbyte-minio.yaml did the trick. I did have to delete the minio deployment first losing the older logs as the pvc for minio isn’t dynamic.