Summary
The user is experiencing sync failures when the number of tables in the source schema runs into hundreds. The state table is missing offset and per stream records, causing full snapshots on every sync. They are inquiring about any restrictions or size limits on the number of tables in the source schema and if it can be increased.
Question
My connection is syncing from MySQL to S3. I observed when the number of tables in source schema runs into hundreds, the sync fails to save the state. State table doesn’t have the offset record. State table also doesn’t have per stream records. Every sync does a full snapshot of the tables it is configured to sync.
Is there a restriction on number of tables the source schema can have? Can this restriction/size limit be increased?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.