Summary
The user is experiencing slow performance when fetching historical data from Shopify using Airbyte on a 32GB EC2 instance. They are seeking advice on optimizing parameters like GraphQL BULK Date Range in Days, BULK Job checkpoint, and BULK Job termination threshold.
Question
Hello everyone,
The shopify historical data fetch is taking a lot of time to get all the data. We have installed Airbyte on 32GB EC2. The job is waiting for a lot of time to get the data streams using GraphQL BULK API.
Here is our configuration of Shopify connection.
Could you please let us know what would be the ideal values to be set to the following parameters to fetch the data quickly?
• GraphQL BULK Date Range in Days
• BULK Job checkpoint (rows collected)
• BULK Job termination threshold
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.