Unable to replicate Mailchimp data for stream [campaign] if its normalized


I have created a connection using following:
Source: Mailchimp [API Key authentication used]

Destination: Redshift [Copy Strategy]

Stream selected for replication: Campaigns

It works fine if raw data is replicated but the sync fails if normalization is applied.

Logs are attached below:
logs-4494.txt (340.0 KB)

Here is the stream raw data replicated to destination
campaign data.txt (3 KB)
This is the same data that i need to normalize

Hey could you try a fresh source and rerun the sync again ?

Hey @harshith,

I tried a fresh sync where i configured a new source and destination but the sync again fails.
Here’s my connection settings,

Here are the logs for the new sync.
logs-29744.txt (135.8 KB)

My Airbyte version is 0.35.31

Hey could you help me with more details like

  1. Are you using any custom connector?
  2. Could you try a different destination?

Ideally, this error shows up when there are duplicate tables

Hi @harshith

Please find my responses below:
1-No, i am not using any custom connectors, both connectors are the ones that are built by airbyte.

  1. Actually i have only one data ware house currently which is redshift. Therefore i cannot try it out with a different destination.

Also i tried the sync again
Here’s my schema before running the sync, showing no table exists for this stream:
After running the sync, only raw table is created but not the normalized one.

All tries were a failure with following error. In error message you can observe that it says that Oauth constant value is required, but actually i am using API key authentication method. Maybe this can help you guys to debug this issue

Hey could you help us with the version of source and destination. You can also try with postgres as a database and see if that works just so that we can target down

Hi @harshith,

Please find the connector versions below:
Source: Mailchimp - 0.2.11
Destination: Redshift - 0.3.28

Also i tried a connection using postgres as destination & it still failed when normalization was applied on source stream “campaign”

Logs for the sync of connection mailchimp > postgres are attahced below
logs-30199.txt (15.7 KB)

Hey the logs doesn’t have any details regarding the normalisation could you check that and reshare the logs file?

Hi @harshith

Please find the updated logs, apologies for the inconvenience.
logs-1121.txt (132.5 KB)

Hey could you check the schema for this connection? May be share a screenshot or response of this API https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#post-/v1/connections/get

Hi @harshith

I created a new connection with the same setup. Please find the attached details:

logs-310.txt (256.4 KB)

Response of the requested API:
Connection API - Response.txt (135.6 KB)

Hey have created an issue https://github.com/airbytehq/airbyte/issues/16209 so that team can view and help here

Thank you for your quick response, Would be great if I can get notified when the issue is fixed.

Sure feel free to give a +1 or follow up message on the issue so that team can reply/priotise over there