Initial Syncs Run, Subsequent All Fail (GitHub and Jira)

  • Is this your first time deploying Airbyte?: Yes
  • OS Version / Instance: Ubuntu
  • Memory / Disk: you can use something like 8Gb / 60GB
  • Deployment: Docker on EC2
  • Airbyte Version: What version are you using now? 0.36.9
  • Source name/version: GitHub & Jira (testing Zendesk Now) Latest
  • Destination name/version: Snowflake Latest
  • Step: Synce
  • Description: When I add a new source with everything set to overwrite full-refresh, it will run correclty the first time, but all subsequent runs fail. If I do any incremental runs on the GitHub connector, it will fail as well.

github_failed_json_validation.txt (314.0 KB)
jira_failed_second_run.txt (803.6 KB)

The only error I can see from the logs is that the worker fails.

Every single run, regardless of success shows “json validation failed”

Exakt same issue here, but for a slightly different setup.

Airbyte 0.38.1-alpha
Destination bigquery : 1.1.4
Source: MSSQL 0.3.21

These errors only happen to connections that are normalizing data to a tabular format.

Log from first successful run:
logs-17 (1).txt (714.5 KB)

Log from following unsucessful run. Please note that all seems to have gone well. It’s just the run status that says it failed.
logs-26.txt (746.6 KB)

I haven’t been able to figure anything out as to why. There’s plenty of disk space and memory - running on a t2.large as recommended in the setup guide. Not sure what else it could be.

Garret for the Jira problem please take a look in this issue: https://github.com/airbytehq/airbyte/issues/9218

For the Github connector the issue is:

(airbyte_cdk.sources.streams.http.exceptions.UserDefinedBackoffException: Request URL: https://api.github.com/repos/heap/heap/issues/comments?per_page=10&page=4&since=2013-09-29T06%3A42%3A15Z, Response Code: 502, Response Text: { "message": "Server Error"})

Maybe it’s a transient error from Github server. Do you mind break the topic in two, one for Jira and other for Github?

Max Krog your issue is this one https://discuss.airbyte.io/t/bigquery-details-400-bad-request/1049 please follow discussion there.