- Is this your first time deploying Airbyte?: Yes
- OS Version / Instance: Ubuntu
- Memory / Disk: 32GB
- Deployment: Are you using Docker or Kubernetes deployment? Docker
- Airbyte Version: What version are you using now? 0.40.3
- Source name/version: Local CSV file
- Destination name/version: Postgres local install
- Step: The issue is happening during sync, creating the connection or a new source? Sync
I am trying to sync a large (32M rows) CSV local file (/tmp/airbyte_local) into a Postgres table also on a local (non-docker) instance.
Postgres has been working fine for several other loads including both small CSV files as well as remote connections to Azure Synapse, Shopify and others.
But I cannot load this large local CSV file successfully. I’m seeing “Canceling Destination” message after several million records.
Any ideas ?
logs-345.txt (4.3 MB)