Summary
The user is experiencing errors with the Mixpanel source connector versions 2.0.0 and 2.0.1, specifically related to the cohorts endpoint limit being exceeded even when only the export stream is selected. They are seeking assistance on why the connector is trying to access the cohorts endpoint and how to resolve this.
Question
Hello!! I’m having some issues with Mixpanel source connector (versions 2.0.0 and 2.0.1.)
I have a Mixpanel source where I only have selected export stream.
When I try to sync I got errors regarding cohots endpoint limit exceeded.
: Request URL: <https://mixpanel.com/api/2.0/cohorts/list>, Response Code: 429, Response Text: {"request": "/api/2.0/cohorts/list", "error": "Query rate limit exceeded for project_id: <REMOVED> 1/5 queries running concurrently and 61/60 queries running in the last hour. For more information, please consult our documentation at <https://help.mixpanel.com/hc/en-us/articles/115004602563-Rate-Limits-for-Export-API-Endpoints>"}
Does anyone knows why the source connector is trying cohort endpoint (even when I only select export), have this problem or know how to solve this?
Thank you!
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["mixpanel-source-connector", "cohorts-endpoint", "error", "limit-exceeded"]
Hmm, I don’t know enough to give you a confident answer yet. However, based on the information provided, it seems like the Mixpanel connector is hitting the rate limit for the API. This could be due to the connector making too many requests within a certain timeframe. In one of the Github issues related to the Mixpanel connector, it was mentioned that Mixpanel might have different rate limits for various resources, even though this isn’t explicitly documented. It’s also worth noting that in the same issue, users reported that the connector became very slow in newer versions, possibly due to changes in the API rate limits. As a potential workaround, you could try using an older version of the connector (e.g., 0.1.1) as some users reported that it worked for them. However, please note that this might not be a viable long-term solution. For a more accurate solution, I would recommend reaching out to the Airbyte community on their forum or Slack channel. They might be able to provide more insights or potential fixes for this issue.
Hey Christian!
I didn’t look in details, but:
• It’s possible that an endpoint would be hit if exports are substream of cohorts (?)
• In either way, when you hit a rate limit, a connector should try and back off gracefully instead of erroring out — I did not look if we have that in Mixpanel yet
• File a GitHub issue, we’ll take a look and fix it.
• As Kapa here is implying, I don’t how we support per-stream rate limits, but I have a suspicion that we do support that at the CDK level, perhaps just not implemented for Mixpanel specifically.
Either way, fixable.
Doesn’t look like export depends on cohorts in any way, hmm