Mixpanel connector provides no data

[albanvanrijsewijk1]

Hello,

• Is this your first time deploying Airbyte?: No
• Source name/version: Mixpanel 0.1.23
• Destination name/version: Snowflake
• Step: The issue is happening during sync
• Description: Since last Mixpanel connector update, my sync has no record in it. Can you help me to identify where the issue comes from please ?
  
  

Available to provide any additional information.

Thanks a lot for your help !

[marcosmarxm]

Hello there! You are receiving this message because none of your fellow community members has stepped in to respond to your topic post. (If you are a community member and you are reading this response, feel free to jump in if you have the answer!) As a result, the Community Assistance Team has been made aware of this topic and will be investigating and responding as quickly as possible.
Some important considerations that will help your to get your issue solved faster:

• It is best to use our topic creation template; if you haven’t yet, we recommend posting a followup with the requested information. With that information the team will be able to more quickly search for similar issues with connectors and the platform and troubleshoot more quickly your specific question or problem.
• Make sure to upload the complete log file; a common investigation roadblock is that sometimes the error for the issue happens well before the problem is surfaced to the user, and so having the tail of the log is less useful than having the whole log to scan through.
• Be as descriptive and specific as possible; when investigating it is extremely valuable to know what steps were taken to encounter the issue, what version of connector / platform / Java / Python / docker / k8s was used, etc. The more context supplied, the quicker the investigation can start on your topic and the faster we can drive towards an answer.
• We in the Community Assistance Team are glad you’ve made yourself part of our community, and we’ll do our best to answer your questions and resolve the problems as quickly as possible. Expect to hear from a specific team member as soon as possible.

Thank you for your time and attention.
Best,
The Community Assistance Team

[marcosmarxm]

The 2 last syncs are Resets and they are correct. Did you try to sync again? Can you share the logs from the Cancelled sync?

[albanvanrijsewijk1]

Hello @marcosmarxm, thanks a lot for your answer, it’s really helping. In fact when retrying to sync mixpanel data, it takes a very very long time and I do feel this is not the expected behaviour.

Attached are the logs from the last sync that is running since yesterday.

The error is the following:
166f2fb3_cf5f_4233_b60f_54cec13745a4_logs_600_txt.txt (1.1 MB)
2022-09-26 18:37:58 ERROR c.n.s.DateTimeValidator(tryParse):82 - Invalid date-time: No zone offset information found

Is there something I can do on my end to solve that issue ?

Best,

[marcosmarxm]

Do you have any idea about how many records there are in the Mixpanel source? Some sources limit the ingestion and can take a long time to sync the first attempt. Not sure if the error is actually an error, because didn’t stop the sync. Did you check destination to see if there is data there?

[albanvanrijsewijk1]

Hello @marcosmarxm Thanks again for your reply. It has been a week now that I am trying to import Mixpanel Data into Snowflake so that’s why I did not reply until now. It seems that I have reached a new milestone this week-end.

Only normalization is failing now and I have checked the destination: tables starting with _airbyte_raw_Mixpanel contains some data.

Can you help me interpret the last logs please ? I don’t really know what to do to make the normalization succeed.

Best,
Alban
166f2fb3_cf5f_4233_b60f_54cec13745a4_logs_640_txt.txt (406.0 KB)

[marcosmarxm]

The Engage stream has a wrong data record:

2022-10-03 07:01:45 e[42mnormalizatione[0m > e[31mCompleted with 1 error and 0 warnings:e[0m2022-10-03 07:01:45 e[42mnormalizatione[0m > e[33mDatabase Error in model MIXPANEL_ENGAGE_SCD (models/generated/airbyte_incremental/scd/MIXPANEL/MIXPANEL_ENGAGE_SCD.sql)e[0m2022-10-03 07:01:45 e[42mnormalizatione[0m > 100038 (22018): Numeric value '' is not recognized2022-10-03 07:01:45 e[42mnormalizatione[0m > compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_incremental/scd/MIXPANEL/MIXPANEL_ENGAGE_SCD.sql

Can you check the Replication tab for Engage stream?

[albanvanrijsewijk1]

@marcosmarxm Thanks a lot for your help. I am really sorry but I don’t get how some records could be wrong. Please find below some screen caps of my settings regarding the engage stream:

In which field the empty numeric value is being populated ?

[marcosmarxm]

Alban to solve the issue you must follow the steps:

• update the connector to latest version (if is not)
• Refresh the schema and reset your data… from your screenshot you’re using a old-schema format (group id is number the best data type is string, you can check the latest schema here: https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-mixpanel/source_mixpanel/schemas/engage.json)

[emilija-omnisend]

Hi, we’re having the same problem with incremental updates on engage stream.

Connector is the latest version (atm, it’s 0.1.28).
We’re trying to use the default property as a cursor - last_seen.
Schema was refreshed and data was reset a few times. It didn’t help, we’re still seeing the error “[m c.n.s.DateTimeValidator(tryParse):82 - Invalid date-time: No zone offset information found” and the sync loads full data.

The comment about Mixpanel engage stream taking a long time to load applies to us as well. But that one is caused by https://github.com/airbytehq/airbyte/pull/7439 . Engage api endpoint doesn’t have a limit like export does (it was working perfectly without those pauses until we upgraded Mixpanel connector, now instead of one hour, our sync runs almost 7 since it can’t do incremental updates due to the aforementioned date issue and is always loading everything.

We “solved” the problem by downgrading the connector for now. But it would be much better if these problems were fixed and we could use up to date connectors.

[albanvanrijsewijk1]

Hello @emilija-omnisend,

Thanks for your participation. There is definitely an issue on Airbyte end (my dev told me he was investigating a python issue on their code).

Can you share with me your current version of your mixpanel connector please ? I’ll follow your workaround while we try to find a proper solution.

Thanks

[emilija-omnisend]

Sure, we ended up switching to before the 60 second delay was implemented - version 0.1.1. Although that version doesn’t support incremental updates, but the full-refresh is much faster.

[albanvanrijsewijk1]

Thank you so much, will try on my end and let you know. @marcosmarxm Do you have any news regarding this issue please ?

[marcosmarxm]

Not until now Alban.