Failed to convert JSON to Avro

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu, GKE n1-standard-4
  • Deployment: Kubernetes via Helm
  • Airbyte Version: 0.36.2-alpha
  • Source name/version: Zendesk Support 0.2.6
  • Destination name/version: GCS 0.2.4
  • Step: On initial sync job
  • Description: Getting this error when running a sync Failed to convert JSON to Avro: Could not evaluate union, field events is expected to be one of these: NULL, ARRAY. If this is a complex type, check if offending field: events.previous_value adheres to schema.
    Destination is parquet file, snappy compression

Seems similar to this issue.

logs-15.txt (67.5 KB)

If you change to JSON it works?

Yep, using JSONL format seems like it works. Doesn’t error out at events like above.

Edit: Added logs,
logs-21.txt (125.9 KB)

I raised the problem to our connector team Kemp Po, using JSONL it’s possible as workaround for now?

Sounds great @marcosmarxm, Thank you. Yeah actually using csvs currently as a workaround and just have a transform job behind the scenes running to convert to parquets

Great Kemp Po, team will work in the future to solve the issue. I’ll let you know about any update

1 Like