Failed to convert JSON to Avro

Connector Questions & Issues
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu, GKE n1-standard-4
  • Deployment: Kubernetes via Helm
  • Airbyte Version: 0.36.2-alpha
  • Source name/version: Zendesk Support 0.2.6
  • Destination name/version: GCS 0.2.4
  • Step: On initial sync job
  • Description: Getting this error when running a sync Failed to convert JSON to Avro: Could not evaluate union, field events is expected to be one of these: NULL, ARRAY. If this is a complex type, check if offending field: events.previous_value adheres to schema.
    Destination is parquet file, snappy compression

Seems similar to this issue.

Screen Shot 2022-04-25 at 11.38.02 PM
Screen Shot 2022-04-25 at 11.38.02 PM803×714 82.3 KB

logs-15.txt (67.5 KB)

2

If you change to JSON it works?

4

Yep, using JSONL format seems like it works. Doesn’t error out at events like above.

Edit: Added logs,
logs-21.txt (125.9 KB)

5

I raised the problem to our connector team Kemp Po, using JSONL it’s possible as workaround for now?

6

Sounds great @marcosmarxm, Thank you. Yeah actually using csvs currently as a workaround and just have a transform job behind the scenes running to convert to parquets

7

Great Kemp Po, team will work in the future to solve the issue. I’ll let you know about any update

1 Like
8

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.