- Deployment: Kubernetes
- Airbyte Version: 0.38.4-alpha
- Source name/version: Jira
- Destination name/version: Redshift
- Step: Normalization
- Description: Raw JSON in
issues.fields
is correct. Normalization step succeeds without errors according to the logs, but no data besides the default Airbyte columns show up in the normalizedissues_fields
table.
Hi @caseywhoop,
Could you please share:
- Your source jira connector verion
- a sample of your Raw JSON for the
issue_fields
stream. Do you confirm that you have a raw table forissue_fields
only? The connector exposes anIssueFields
streams. But theIssue
stream also has afields
property. This might be the cause of the normalization problem. - The sync log
- Your
catalog.json
file (instructions on how to extract it here
Is it also possible for you to try a different destination such as Postgres? If the error does not happen on Postgres it would help us nail down the problem.
I’m afraid that this is probably related to a bug in the source connector and that I’ll ask you to file an issue with the details I asked forabove.
Hi - thanks for the reply.
1: source jira connector version: 0.2.20
2: I am not sure I follow, but hopefully this+the screenshots gives some better context. The only sync I have turned on is Issues
. It loads 3 tables into my schema: _airbyte_raw_issues
, issues
, and issues_fields
. The issues table has a column called fields
that contains a large JSON of all the useful data. I was under the impression the table issues_fields would contain the information from that issues.fields JSON, but normalized. It only contains the standard airbyte default columns.
3. Sync logs attached
4. I think this one is a little tricky b/c the kubernetes pod that builds this is only live for ~0.5s?
5. Will try w/ Postgres soon and report back
airbyte_jira_fields_json.txt (2.9 KB)
airbyte_jira_fields_json.txt (2.9 KB)
I just tried a freshly setup postgres db destination with the same exact result
Thank you for the thorough testing!
So I think the bug resides in the unnesting process the normalization run on the fields
field of the issues
stream.
Could you please open a bug report on our GitHub repo with all the details you shared and share its link here? I’ll make sure to triage it properly so that it reaches our connector team backlog.
Bug reported here: JIRA Issues Normalization Fails Without Error · Issue #13969 · airbytehq/airbyte · GitHub
Thanks!
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.