Jira issues -> issues_fields normalization not populating any data

  • Deployment: Kubernetes
  • Airbyte Version: 0.38.4-alpha
  • Source name/version: Jira
  • Destination name/version: Redshift
  • Step: Normalization
  • Description: Raw JSON in issues.fields is correct. Normalization step succeeds without errors according to the logs, but no data besides the default Airbyte columns show up in the normalized issues_fields table.

Hi @caseywhoop,

Could you please share:

  • Your source jira connector verion
  • a sample of your Raw JSON for the issue_fields stream. Do you confirm that you have a raw table for issue_fields only? The connector exposes an IssueFields streams. But the Issue stream also has a fields property. This might be the cause of the normalization problem.
  • The sync log
  • Your catalog.json file (instructions on how to extract it here

Is it also possible for you to try a different destination such as Postgres? If the error does not happen on Postgres it would help us nail down the problem.
I’m afraid that this is probably related to a bug in the source connector and that I’ll ask you to file an issue with the details I asked forabove.

1 Like

Hi - thanks for the reply.

1: source jira connector version: 0.2.20
2: I am not sure I follow, but hopefully this+the screenshots gives some better context. The only sync I have turned on is Issues. It loads 3 tables into my schema: _airbyte_raw_issues, issues, and issues_fields. The issues table has a column called fields that contains a large JSON of all the useful data. I was under the impression the table issues_fields would contain the information from that issues.fields JSON, but normalized. It only contains the standard airbyte default columns.
3. Sync logs attached
4. I think this one is a little tricky b/c the kubernetes pod that builds this is only live for ~0.5s?
5. Will try w/ Postgres soon and report back

airbyte_jira_fields_json.txt (2.9 KB)



airbyte_jira_fields_json.txt (2.9 KB)

I just tried a freshly setup postgres db destination with the same exact result

Thank you for the thorough testing!
So I think the bug resides in the unnesting process the normalization run on the fields field of the issues stream.
Could you please open a bug report on our GitHub repo with all the details you shared and share its link here? I’ll make sure to triage it properly so that it reaches our connector team backlog.

Bug reported here: JIRA Issues Normalization Fails Without Error · Issue #13969 · airbytehq/airbyte · GitHub

Thanks!

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.