Does running multiple connections to the same data source help with parallelisation/performance?

Q&A
#1

We have a postgres database with a few 500GB+ tables and hundreds of smaller (0-10GB range) ones.

Would setting up separate connections (CDC) for:

  • one connection per big table
  • one connection for all the remaining small tables
    help for performance/parallelisation?

For reference:
Airbyte v0.39.32-alpha
Postgres Source 0.4.30

#2

Yes, for now it would be better have different connections and (if possible) diferrent CDC slots to each connection

1 Like