Encountering problem mentioned in GitHub issue 8875

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

User is facing a problem mentioned in GitHub issue 8875 and seeking help for a solution.


Question

Hey Everyone, I am encountering the same problem mentioned in this issue: https://github.com/airbytehq/airbyte/issues/8875
Does anyone have the solution for this?? Any help will be appreciated!!


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["encountering-problem", "github-issue-8875", "solution", "help"]
2

make sure the consumer you configured in source kafka airbyte able to the read the messages in the topics and retest the source

3

And how can I do that in the Kafka Source config??? Is there some option???

4

Yup let me share some steps

5

We used kafka as source and postgres as destination we faced a similar issue tables getting created in postgres but messages from kafka not getting synced with postgres

6

Are you using any Ui tool to mange your kafka cluster

7

Sorry for a late response… I currently have Kafka in my localhost

8

Hey <@U067Z18322D>, could you help me on this???

9

Will connect on Monday 1pm.

10

I will DM you the link…

11

Mean while share the the details here

12

If I have time will look into it tomorrow

13

Yeah… My use case is, I want to get data from Kafka topic to Postgres table using Airbyte.
The messages in those topics are in perfect JSON format. But while syncing, I am getting _airbyte_data as {} in the destination for all the messages.
Not sure what is going wrong…

14

Okay Got it. Both your kafka and postgres are in you local system?

15

And I installed a UI tool to monitor my local Kafka… There were no ACLs as well to block Airbyte from reading messages

16

Things to check

1.the consumer you configured in airbyte need to have enough permissions to read the topics from kafka.

2 . Check the destination postgres db creds and access rules.

17

Share the ss of the airbyte kafka configuration

18

Yeah…
I will check the step 1…
But for step 2, I actually tried for a different destination as well (Local JSON), and I am getting the same {} in output

19

Then it’s the issue of kafka Source configuration

20

If u have logs pls attach here if possible will connect tomorrow I hope it not gonna take more than 30-45 mintues