Jira issues -> issues_fields normalization not populating any data

  • Deployment: Kubernetes
  • Airbyte Version: 0.38.4-alpha
  • Source name/version: Jira
  • Destination name/version: Redshift
  • Step: Normalization
  • Description: Raw JSON in issues.fields is correct. Normalization step succeeds without errors according to the logs, but no data besides the default Airbyte columns show up in the normalized issues_fields table.

Hi @caseywhoop,

Could you please share:

  • Your source jira connector verion
  • a sample of your Raw JSON for the issue_fields stream. Do you confirm that you have a raw table for issue_fields only? The connector exposes an IssueFields streams. But the Issue stream also has a fields property. This might be the cause of the normalization problem.
  • The sync log
  • Your catalog.json file (instructions on how to extract it here

Is it also possible for you to try a different destination such as Postgres? If the error does not happen on Postgres it would help us nail down the problem.
I’m afraid that this is probably related to a bug in the source connector and that I’ll ask you to file an issue with the details I asked forabove.

1 Like

Hi - thanks for the reply.

1: source jira connector version: 0.2.20
2: I am not sure I follow, but hopefully this+the screenshots gives some better context. The only sync I have turned on is Issues. It loads 3 tables into my schema: _airbyte_raw_issues, issues, and issues_fields. The issues table has a column called fields that contains a large JSON of all the useful data. I was under the impression the table issues_fields would contain the information from that issues.fields JSON, but normalized. It only contains the standard airbyte default columns.
3. Sync logs attached
4. I think this one is a little tricky b/c the kubernetes pod that builds this is only live for ~0.5s?
5. Will try w/ Postgres soon and report back

airbyte_jira_fields_json.txt (2.9 KB)

airbyte_jira_fields_json.txt (2.9 KB)

I just tried a freshly setup postgres db destination with the same exact result

Thank you for the thorough testing!
So I think the bug resides in the unnesting process the normalization run on the fields field of the issues stream.
Could you please open a bug report on our GitHub repo with all the details you shared and share its link here? I’ll make sure to triage it properly so that it reaches our connector team backlog.

Bug reported here: JIRA Issues Normalization Fails Without Error · Issue #13969 · airbytehq/airbyte · GitHub