Summary
Clarification on the Google Ads connector’s incremental fetch window and start date setup for daily incremental fetch.
Question
Question: Google Ads Connector incremental overlap window lookback size
Follow up here from question from <@U05DJ8GSFCJ>
in the Google Ads documentation, it says "For incremental streams, data is synced up to the previous day using your Google Ads account time zone since Google Ads can filter data only by date without time." but Google themselves state that data can change up to seven days in the past.
We looked into the connector code and it is not clear for us the start_date setup for daily incremental fetch,
is it only fetching the last day?
In the log for incremental fetch we see segments.date with one date yesterday so it looks like only one day refetch.
But in the code if no start_date is specified it fetches 2 years of data https://github.com/airbytehq/airbyte/blob/1bd7577be0da3a0dabec0b80c5d51ef16501210f/airbyte-integrations/connectors/source-google-ads/source_google_ads/source.py#L90-L91|https://github.com/airbytehq/airbyte/blob/1bd7577be0da3a0dabec0b80c5d51ef16501210f[…]ations/connectors/source-google-ads/source_google_ads/source.py
I do not see any other date logic around.
TLDR:
does anyone know the google connector incremental overlap fetch window for google?
(some other connectors are more explicit to define when the metrics are finalised e.g. here: https://github.com/airbytehq/airbyte/blob/1bd7577be0da3a0dabec0b80c5d51ef16501210f/airbyte-integrations/connectors/source-snapchat-marketing/source_snapchat_marketing/source.py#L793-L796|https://github.com/airbytehq/airbyte/blob/1bd7577be0da3a0dabec0b80c5d51ef16501210f[…]s/source-snapchat-marketing/source_snapchat_marketing/source.py )
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.