Failed to fetch schema for Oracle where schema has large number of tables

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu 22.04 on AWS EC2 with 8 cores 32 GB RAM
  • Memory / Disk: 250GB
  • Deployment: Docker Deployment
  • Airbyte Version: 0.39.32-alpha
  • Source name/version: Oracle DB 0.3.17
  • Destination name/version: Redshift 0.3.46
  • Step: creating the connection
  • Description: Connection setup fails with the error - Failed to fetch schema. Please try again
    When trying to use a schema with up to 5000 tables, the setup works. The setup fails for this particular schema having 13541 tables.

Hello and welcome to the Airbyte community, @vaibhavpals! Could you show me the server logs that you’re getting? We’ve gotten a few similar issues to this one recently, and this might help me point you to a solution.

Hi,
Can you guide me as to exactly what logs i need to be looking at?
Which container should I monitor? And detailed logging i need to enable?

Hi,

I believe this to be the error :
io.grpc.StatusRuntimeException: RESOURCE_EXHAUSTED: grpc: received message larger than max (4676324 vs. 4194304)

Thanks for posting the error. Currently Airbyte supports up to 6000 tables; it’s on our roadmap to expand that number, but I can’t give you a definitive date on when 6000+ tables will be supported yet. If you’d like to keep an eye on this, here is the related issue:
https://github.com/airbytehq/airbyte/issues/11114