Google Analytics data NULL'ing out in redshift due to bad macros

Connector Questions & Issues
,
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: EC2 t2.large
  • Memory / Disk:
  • Deployment: Docker
  • Airbyte Version: 0.36.2-alpha
  • Source name/version: google-analytics 0.1.19
  • Destination name/version: redshift 0.3.32
  • Step: Normalization
  • Description: Jobs started failing last week when I updated our redshift destination to 0.3.32. I was able to isolate the issue (I think) to an update in the macro library generated by Airbyte to account for the SUPER data type that redshift uses. After incorporating those updates into my version of the macro library (I’m running custom transformations), I’ve got jobs successfully running again.

However, I’ve encountered a new issue with the transformation. Seemingly random fields will NULL out in the materialized table. The raw JSON data coming in from the connector is all correct, but for whatever reason the transformation is inputting NULL values. I suspect there is an issue with one of the macros generated by Airbyte (either configuration.sql or json_operations.sql) that I’m using in my dbt transformation logic, but I’m struggling to troubleshoot further. Anyone else encountering a similar issue?

2

Can you upgrade to latest version? Looks the version 0.3.32 of Redshift added some dependency of version of Airbyte

4

I resolved the issue – it was a different function in one of my tables that didn’t like the new SUPER data type. Ended up not being an airbyte / dbt transformation problem as I originally thought.

5

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.