Unable to replicate Mailchimp data for stream [campaign] if its normalized

Connector Questions & Issues
1

Hi,

I have created a connection using following:
Source: Mailchimp [API Key authentication used]

image
image968×695 30.3 KB

Destination: Redshift [Copy Strategy]

Stream selected for replication: Campaigns

It works fine if raw data is replicated but the sync fails if normalization is applied.

Logs are attached below:
logs-4494.txt (340.0 KB)

Here is the stream raw data replicated to destination
campaign data.txt (3 KB)
This is the same data that i need to normalize

2

Hey could you try a fresh source and rerun the sync again ?

3

Hey @harshith,

I tried a fresh sync where i configured a new source and destination but the sync again fails.
Here’s my connection settings,

image
image1425×749 68.7 KB

Here are the logs for the new sync.
logs-29744.txt (135.8 KB)

4

@harshith
My Airbyte version is 0.35.31

5

Hey could you help me with more details like

  1. Are you using any custom connector?
  2. Could you try a different destination?

Ideally, this error shows up when there are duplicate tables

6

Hi @harshith

Please find my responses below:
1-No, i am not using any custom connectors, both connectors are the ones that are built by airbyte.

  1. Actually i have only one data ware house currently which is redshift. Therefore i cannot try it out with a different destination.

Also i tried the sync again
Here’s my schema before running the sync, showing no table exists for this stream:
image
After running the sync, only raw table is created but not the normalized one.

image
image1637×578 111 KB

All tries were a failure with following error. In error message you can observe that it says that Oauth constant value is required, but actually i am using API key authentication method. Maybe this can help you guys to debug this issue
image
image1801×901 134 KB

7

Hey could you help us with the version of source and destination. You can also try with postgres as a database and see if that works just so that we can target down

8

Hi @harshith,

Please find the connector versions below:
Source: Mailchimp - 0.2.11
Destination: Redshift - 0.3.28

Also i tried a connection using postgres as destination & it still failed when normalization was applied on source stream “campaign”

image
image1795×847 131 KB

Logs for the sync of connection mailchimp > postgres are attahced below
logs-30199.txt (15.7 KB)

9

Hey the logs doesn’t have any details regarding the normalisation could you check that and reshare the logs file?

10

Hi @harshith

Please find the updated logs, apologies for the inconvenience.
logs-1121.txt (132.5 KB)

11

Hey could you check the schema for this connection? May be share a screenshot or response of this API https://airbyte-public-api-docs.s3.us-east-2.amazonaws.com/rapidoc-api-docs.html#post-/v1/connections/get

12

Hi @harshith

I created a new connection with the same setup. Please find the attached details:

Logs
logs-310.txt (256.4 KB)

Response of the requested API:
Connection API - Response.txt (135.6 KB)

13

Hey have created an issue https://github.com/airbytehq/airbyte/issues/16209 so that team can view and help here

14

@harshith
Thank you for your quick response, Would be great if I can get notified when the issue is fixed.

15

Sure feel free to give a +1 or follow up message on the issue so that team can reply/priotise over there