Issue with syncing Kafka topic to Postgres

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Syncing Kafka topic to Postgres, value column in final table is null, _airbyte_data column in raw table is empty


Question

Hi All,

I am trying to sync a Kafka topic to Postgres. Things seems to be work, the Source in Airbyte testing successfully and rows are getting written to the raw table and the final target table in Postgres, but the value column in the final table is null and the _airbyte_data column in the raw table are just {}.

Below is the log from the sync show 1 row read and 1 row written. Any help would be appreciated.

-DW


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["kafka-topic", "postgres", "airbyte", "sync", "null-value", "_airbyte_data"]
2

FWIWI to the Airbyte Community,

The following issue logged against the Airbyte Kafka Connector help me understand that the connector requires a specific JSON structure in order to consume and then process my Kafka message into my target.

https://github.com/airbytehq/airbyte/issues/8875

With this knowledge I resolved my own issue. It seems that there is an appetite to improve the Kafka connector.

-DW