Limiting data pulled from Posthog in Airbyte Cloud

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Exploring options to limit data pulled from Posthog in Airbyte Cloud to avoid throttling by the Posthog API


Question

Hi! I am attempting to connect posthog <> bigquery via Airbyte. We are testing with Airbyte Cloud. We are getting throttled by the Posthog API due to volume, and posthog doesn’t offer increased rate limits.

Is there a way to limit the data being pulled via posthog within Airbyte? That way we can test the integration works without getting throttled? Thank you!


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["limiting-data", "posthog-connector", "airbyte-cloud", "throttling", "rate-limits"]
2

Hi! I might have a solution for you. I created a custom Posthog connector for the persons stream that includes a backoff strategy of 400 seconds so that you don’t get stuck behind that rate limit error.

I’ll DM you with the YAML so you can give it a whirl!