Troubleshooting saving state issue in Google Ads connector

Slack Knowledge Base Q&A Connector Development
, , , ,
1

Summary

The user is facing an issue with saving state in the Google Ads connector after refactoring it to extend from ConcurrentSourceAdapter. The updated connector emits different state messages compared to the unmodified connector.


Question

hello! I’m trying to convert google ads source to concurrent model. I refactored SourceGoogleAds to extend from ConcurrentSourceAdapter and made other required changes. but I’m stuck with inability to save state. updated connector emits state messages like this:
{"type": "STATE", "state": {"type": "STREAM", "stream": {"stream_descriptor": {"name": "custom_ga_query", "namespace": null}, "stream_state": {"segments.date": "2024-03-28"}}, "sourceStats": {"recordCount": 0}}}
but unmodified connector emits following state messages:

{"type": "STATE", "state": {"type": "STREAM", "stream": {"stream_descriptor": {"name": "custom_ga_query", "namespace": null}, "stream_state": {"123444444": {"segments.date": "2024-03-28"}}}, "sourceStats": {"recordCount": 54}}}
here are my changes: https://github.com/airbytehq/airbyte/compare/master...nurikk:airbyte:source-google-ads-concurrency

if anyone can help with troubleshooting it would be very helpfull


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["google-ads-connector", "concurrent-model", "saving-state", "troubleshooting", "sourcegoogleads", "concurrentsourceadapter"]
2

Wow, thank you for tinkering with it! :clap:

3

I’ll show this to the team :wink:

4

Thanks <@U069EMNRPA4>! I’ve spent more time on debugging google ads source and still can’t find what I’m doing wrong. there’s something with stream_state argument passed to read_records method. but damn, it’s so hard to comprehend what’s happening :sweat_smile:

5

okay, I figured out that I have to implement custom state converter and cursor classes to handle multiple customers account. so far thing are moving in the right direction. only one piece is missing, correct start date initialisation. which I’ll implement tomorrow

6

I made it! it works! :slightly_smiling_face: