Summary
The synchronization process from MongoDB to Kafka in Airbyte is causing duplicated records and an error message ‘Cannot send message to Kafka. Error: Topic test_topic not present in metadata after 60000 ms.’ despite the topic existing on the Kafka broker server. The user has verified the connection to the Kafka server.
Question
Hello Everyone, i have been able to setup my source as mongoDb, and Destination as Kafka on airbyte.
Connection works fine till its time for synchronization,
Challenge i am facing is the synchronization does not stop till it times out, while syncing i can see it fetching records from mongodb, but it keeps duplicating the records and give an error message
java.lang.RuntimeException: Cannot send message to Kafka. Error: Topic test_topic not present in metadata after 60000 ms.
Kafka is running on a docker instance
However test_topic exists on the kafka broker server, i have verified
Telneting to the bootstrap server works fine and shows that i am connected to the kafka server also
Been on this for 3 days, any help will be appreciated
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.