Shopify connector facing problems during ingestion

Connector Questions & Issues
,
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu
  • Memory / Disk: 14GB RAM, 20GB
  • Deployment: Docker on GCP GCE
  • Airbyte Version: 0.39.39-alpha
  • Source name/version: Shopify 0.1.37
  • Destination name/version: BigQuery 1.1.11
  • Step: Happens during the sync
  • Description: Facing a problem with the description of it mentioning “Failure Origin: source, Message: Something went wrong within the source connector” when connecting Shopify to BigQuery. The logs are uploaded for reference. I don’t understand the problem because I have Admin permission to Shopify hence I would not come into problem when calling the API for the Shopify. Please help. Thanks.
    logs-24.txt (422.4 KB)
2

Hi @thongseng188,

I found a related post with someone using the same Shopify connector version and getting the same error:
https://discuss.airbyte.io/t/failed-to-pull-data-from-shopify/1549

This is the error I see in your logs:
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: ``https://montigo-bottle.myshopify.com/admin/api/2021-07/shopify_payments/balance/transactions.json?limit=250&order=id+asc&since_id=0

Could you please try to make a manual cURL call using your API token to hit this endpoint and see the headers/response from Shopify? If the issue persists then, I will internally link your post to this GitHub issue:
https://github.com/airbytehq/airbyte/issues/14065

3

@natalyjazzviolin hello, I found out the problem already. The source connector always stops at requests.exceptions.HTTPError: 404 Client Error: Not Found for url: ``https://montigo-bottle.myshopify.com/admin/api/2021-07/shopify_payments/balance/transactions.json?limit=250&order=id+asc&since_id=0 .

The reason is in Malaysia’s Shopify store (or in some other countries not mentioned here link) do not support balance transactions on Shopify Payment. Other tables are not ingested when encountered with this problem. Hence it failed. Tried the manual cURL as well, the same result.

Regards,
Seng.

4

Thanks for the clarification! I’ll escalate this to GitHub as this seems to be an oversight regarding using the Shopify connector in countries that do not support balance transactions!