Summary
Hubspot source incremental sync is taking 3-5 hours, suspecting full refresh instead of incremental updates. How to improve sync time?
Question
Hello everyone im facing the issue that I have a Hubspot source that takes 3-5 hours to update on incremental sync. Im pretty sure that its just refreshing everything on each run so not sure how I can improve this time :'c
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["hubspot-source", "incremental-sync", "sync-time", "full-refresh"]
Are all of the streams you’re pulling set to incremental, or are any of them locked to Full Refresh?
Only pulling companies, deals and contacts all of them on incremental sync
Are the row counts high enough to be a full sync? And are the logs showing it sitting at a certain phase of syncing for a long time?
What version of Airbyte and what destination? If you’re on an older version of a pre-Destinations V2 connector, it could be slow runs of normalization (which was removed in Destinations V2). If that’s the case, just note that there are breaking changes with Destinations V2, so you will need to plan accordingly.