Issue fetching schemas from MongoDB source resulting in badGateway error

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Encountering badGateway error while fetching schemas from MongoDB source to BigQuery target. Checked environment variables and user permissions. Seeking suggestions for resolution.


Question

Hi everyone,
I hope you are all doing well!

I am encountering an issue while fetching schemas from a MongoDB source. The process takes around 5 minutes and results in the error: errors.http.badGateway.
This issue occurs while creating a connection between MongoDB (source) and BigQuery (target).
I have checked the environment variables, and BASIC_AUTH_PROXY_TIMEOUT is set to 900, so that doesn’t seem to be the problem.
I also tried limiting user permissions regarding data sources, which worked, but I need to fetch all schemas from the source.

Has anyone encountered and resolved this issue before? I am open to any suggestions.

Thanks in advance.


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["fetching-schemas", "mongodb-source", "bigquery-target", "badgateway-error", "user-permissions"]
2

Search for HTTP_IDLE_TIMEOUT, READ_TIMEOUT environment variables in Slack

3

<@U05JENRCF7C> Thank you. After I increased this value, it worked for 30 minutes. However, after 30 minutes, I am getting the error ‘Failed to run schema discovery’. I think this might be related to the MongoDB setting logicalSessionTimeoutMinutes=30. Do you experience such an error before? And why it takes so long?

4

No, I haven’t used MongoDB connector at all