Summary
Exploring the possibility of using the REST or GraphQL API instead of the Bulk API in the Shopify connector to improve sync speed for shops with large amounts of data.
Question
Hi all, we’re looking into using Airbyte + its Shopify connector to pull all historical data from certain streams. Shopify offers the Bulk API which seems to be what the connector has been migrating to lately for most streams. Is it possible to switch over the REST API with cursor-based pagination or the GraphQL API and have the connector NOT use the Bulk API? Main reason being the Bulk API is often SLOW for shops with a lot of data. Syncs can take hours (due to Shopify generating the exports) when they would take <30 min via other methods.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.