Summary
Error message ‘Sort exceeded memory limit’ in MongoDB connector for Airbyte. Need to set allowDiskUse:true. Error log provides details.
Question
Hi, I am new to Airbyte and I am trying to create a connector from MongoDB to Snowflake. I am getting error Command failed with error 292 (QueryExceededMemoryLimitNoDiskUseAllowed): 'PlanExecutor error during aggregation :: caused by :: Sort exceeded memory limit of 33554432 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.' Where do I set allowDiskUse:true ? Extract from error log for more details: 2024-05-14 17:02:53 [46mplatform[0m > ERROR main i.a.c.i.b.AirbyteExceptionHandler(uncaughtException):30 Something went wrong in the connector. See the logs for more details. com.mongodb.MongoCommandException: Command failed with error 292 (QueryExceededMemoryLimitNoDiskUseAllowed): 'PlanExecutor error during aggregation :: caused by :: Sort exceeded memory limit of 33554432 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.' on server ********mongodb.net:27017. The full response is {"ok": 0.0, "errmsg": "PlanExecutor error during aggregation :: caused by :: Sort exceeded memory limit of 33554432 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.", "code": 292, "codeName": "QueryExceededMemoryLimitNoDiskUseAllowed", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1715706174, "i": 21}}, "signature": {"hash": {"$binary": {"base64": "***=", "subType": "00"}}, "keyId": ***}}, "operationTime": {"$timestamp": {"t": 1715706174, "i": 21}}}
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.