Encountered error while extracting large amount of data from HubSpot using Airbyte
Hi everyone, I’m extracting a large amount of data from HubSpot with 2.4 million records. However, after connecting with Airbyte, I encountered this error. Can anyone help me?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
How can I extract contact list data? Currently, I’m only receiving contact list metadata from Airbyte
<@U06UPU56ADU> You want the contacts stream for that, not contact_lists . If you don’t see this stream, then the authenticated user most likely doesn’t have permissions to pull from that object.
What stream are you trying to sync from HubSpot? The most likely answer is that the user you’re authenticating with doesn’t have the correct permissions for that object.
I would start by logging in with that user (if you’re using oAuth) and making sure you can list the object you’re trying to sync. There may also be other settings at play, since I’m not sure how the export security controls interact with the API when using oAuth.
If the issue were size/resources, you’d get a different error in the 5xx range. We sync HubSpot accounts into the billions of records (largely due to the noisy email_events stream)—but we have instances with 7-10 million Contacts alone and are able to sync them just fine.