Failed Connection between mongodb and Bigquery

  • Is this your first time deploying Airbyte?
    : No
  • OS Version / Instance
    : Amazon Linux 2 AMI (HVM) / AWS EC2 t2.medium
  • Memory / Disk
    : 4GB / 50GB
  • Deployment
    : Docker (Following this document β†’ On AWS (EC2) | Airbyte Documentation)
  • Airbyte Version
    : 0.39.35-alpha
  • Source name/version
    : mongodb / 0.1.15
  • Destination name/version
    : Bigquery / 1.1.11
  • Step
    : It happens creating the connection between existing source and destination.
  • Description

Hi, I got this error message creating connection mongodb to Bigquery.

2022-07-11 08:21:59 e[33mWARNe[m o.h.v.i.p.j.JavaBeanExecutable(getParameters):216 - HV000254: Missing parameter metadata for LogType(String, int, String), which declares implicit or synthetic parameters.
 Automatic resolution of generic type information for method parameters may yield incorrect results if multiple parameters have the same erasure. To solve this, compile your code with the '-parameters' flag.

For testing the error on which state, I made two different databases on the same mongoDB Server. Connection succeed if source database has few collections, But When I changed source database to bigger db, it failed. And connection failure message pop up over a hour later.

Original Source Database size is 21GiB and has 28 collections I need to connect to Bigquery.

No error when I added source Database (at first step) and there aren’t error messages on MongoDB log. (there were lots of slow query on mongodb log)

What do I have to check other configuration?
I add airbyte server log for solve. Thanks. : )

server-logs-airbyte.txt (46.8 KB)

Hi @boochy, welcome to the Airbyte community! I am triaging this to make sure I can help you in the best way possible, should have suggestions for you tomorrow!

1 Like