Error 513 when setting up connection to source DB with over 200+ tables

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

A new release was made to fix the HTTP 513 error when setting up a connection to a source DB with over 200+ tables, but the error still persists after updating abctl and trying to set up the connection again.


Question

Hello people. As per PR https://github.com/airbytehq/abctl/pull/129#issue-2539571801, a new release was cut yesterday to fix the HTTP 513 error when setting up a connection to a source DB with over 200+ tables. After getting the new version of abctl (using curl -LsfS https://get.airbyte.com | bash -) and running the abctl local install, we went back into the UI and tried setting up a connection to the schema with over 200 tables, and still ran into the 513 error. Anybody able to get it working? <@U07FH2Y34A1> thank you for the help thus far.


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["http-513-error", "source-db", "abctl", "connection-setup", "200-tables"]
2

…from a Team Member’s memory

3

hm, ok.

well, let’s make sure your ingress has the right config map anyway:
docker exec -it airbyte-abctl-control-plane kubectl -n ingress-nginx get cm -l <http://app.kubernetes.io/name=ingress-nginx|app.kubernetes.io/name=ingress-nginx> -oyaml

4

what should I expect the output to be?

5

mine looks like

items:
- apiVersion: v1
  data:
    allow-snippet-annotations: "false"
    proxy-body-size: 10m
    proxy-read-timeout: "600"
    proxy-send-timeout: "600"
...etc...```
6

followed your instructions and just waiting on the install to finish :hourglass_flowing_sand:

7

you ran an uninstall + install?

8

took the call coz we are on a Dev environ today. So no fear of crashing it :slightly_smiling_face:

9

<@U07FH2Y34A1> the install finished and above is the output to the config map

10

Nice, that looks good. The other ingress resources should have been recreated during the install, so it should work now

11

trying to set up a new connection to the source with 200+ tables now :crossed_fingers:

12

I think uninstall+install was the way to get the new release in. We were able to successfully get a connection set up against a source DB with over 200+ tables :tada:

13

however, my question is - will we run into it in the future if the source DB adds say 500 more tables tomorrow?

14

is there a config that we can change on our side?

15

We match the platform configs now at 10MB, which is giant. If you needed to go larger than that, I’m not sure what would need to happen. It’s possible, but you’d need to dig into the ingress config and also the airbyte app configs (e.g. server config)

16

that is helpful input. We’ll capture these findings for when we “might” run into the issue

17

@Alex Buchanan another finding is that running abctl local uninstall --persisted did not remove all the database configuration. This environment was a test machine so we were expecting all config to be erased

18

Could you clarify what you mean by “database configuration”?

uninstall --persisted should remove local directories where the database state is stored when using the built-in database (e.g. ~/.airbyte/abctl/data/airbyte-volume-db)

19

To clarify the understanding we had was that the postgres database config would be wiped out when using the --persisted flag. after running with --persisted we found all previous connections sere still present

20

Are you using an external postgres database? Or the one that installs automatically with abctl?