logs-4105.txt (138.5 KB)
- Is this your first time deploying Airbyte: No / Yes
- OS Version / Instance: Ubuntu 18.04, Mac OS, Windows, GCP , EC2 micro.a4
- Memory / Disk: 16Gb / 1Tb SSD
- Deployment: Docker / Kubernetes
- Airbyte Version: 0.35.15-alpha
- Source name/version: File 0.24
- Destination name/version: Amazon S3
- Step: Problem occurs on-sync.
I’ve got a connector here that failed to sync a few times and the logs report that the Java VM is out of memory. The VM that Airbyte is running on has 8 GB of RAM and that’s been enough for us since we started.
I’d like to rule out the possibility of poor coding on my part by figuring out exactly how much RAM the connector is using while syncing.
I’ve never profiled anything Python before, is there a “preferred” set of tools / methods that should be used when profiling Airbyte connectors?
@marcosmarxm yes we can. it might take some time because ops is a little pre-occupied but i’ll get back to you.
Its worth noting that the last couple of syncs with this connector have succeeded without error so this may have been an anomaly.
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.