Summary
User reports a performance issue with slow data sync from Sentry to BigQuery, specifically observing rapid extraction of events but prolonged loading times, with logs showing repeated entries. Seeking advice on potential causes and resolutions.
Question
Hello,
I am having trouble with a very slow sync between Sentry and BigQuery
The sync extracts events records from Sentry very quickly, but seems to take a very long time loading them to the destination. In previous syncs, it took ~8 hours to load ~1000 records.
I am using
Incremental | Append + Deduped
for the sync mode
Looking at the logs, the same log entries keep appearing a couple of times every minute.
Any advice on the cause and possible fix for this would be much appreciated.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.