Error when upgrading Facebook Marketing Source and Postgres Destination

Summary

Encountering a KeyError ‘account_id’ error after upgrading Facebook Marketing Source and Postgres Destination. New sources, destinations, and connectors work fine, but older ones throw this error.


Question

Hey team,

I recently upgraded my Facebook Marketing Source from v1.4.2 to v2.0.0 and Postgres Destination from v0.6.3 to v2.0.4. However, after the upgrade, I’m encountering the following error:

2024-03-09 14:25:26 source > Encountered an exception while reading stream customads_age_gender_action_type
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 126, in read
    yield from self._read_stream(
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 203, in _read_stream
    for record in record_iterator:
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/abstract_source.py", line 239, in _read_incremental
    for record_data_or_message in stream_instance.read_incremental(
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/core.py", line 162, in read_incremental
    stream_state = self.get_updated_state(stream_state, record_data)
  File "/airbyte/integration_code/source_facebook_marketing/streams/base_streams.py", line 249, in get_updated_state
    account_id = latest_record["account_id"]
KeyError: 'account_id'

I’ve attached the complete traceback for reference.
Additionally, here’s the schema for my custom insight:

{
            "name": "ads_age_gender_action_type",
            "level": "ad",
            "fields": [
                "ad_id",
                "ctr",
                "inline_link_click_ctr",
                "inline_link_clicks",
                "outbound_clicks",
                "outbound_clicks_ctr",
                "spend",
                "date_start",
                "date_stop",
                "   ",
                "ad_click_actions",
                "ad_impression_actions",
                "conversions",
                "unique_inline_link_click_ctr",
                "unique_link_clicks_ctr",
                "unique_outbound_clicks_ctr",
                "unique_inline_link_clicks",
                "unique_outbound_clicks",
            ],
            "breakdowns": ["age", "gender"],
            "start_date": start_date,
            "time_increment": 1,
            "action_breakdowns": ["action_type"],
            "action_report_time": "mixed",
            "insights_lookback_window": 28,
        }

Interestingly, creating new sources, destinations, and connectors for different Facebook Account works without any issue. It’s only when upgrading the older ones that this error occurs.

Could anyone please assist me in resolving this issue?



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["facebook-marketing-source", "postgres-destination", "upgrade-error", "keyerror", "account-id"]