Hello, everyone. I’m trying to integrate Airbyte into our ETL with Kafka-destination and I faced a problem with Kafka topics naming convention in our company: their names should have dots as a separator. I found out that Airbyte-kafka-connector always replaces them with underscore if put into names. For example, if I write into field “Topic Pattern” something like “airbyte.test” in actual Kafka I get “airbyte_test”. Is there any way to configure Airbyte to create topics with dots in their names?
In this case, the period character might be one of the special characters that the StandardNameTransformer replaces, but I am not 100% certain of this.
Hi, @sajarin, thanks for your answer. I actually already figured out about this being a feature rather than bug. So I just added period in this line - airbyte/Names.java at master · airbytehq/airbyte · GitHub, then updated tests in destination-kafka and create custom image to match our needs. With it I can now create topics with periods in names. I guess that workaround solves my question.
But still, I suggest to think about adding possibility to choose conventional symbols between underscores to periods or even removing this limitation at all: in Kafka topic naming conventions periods are as good as underscores - Kafka Topic Naming Conventions: Best Practices | by Kiran Prabhu | Medium, they just don’t recommend mixing them both in names at the same time
Hey @SAmser93, glad to hear you were able to find a workaround. I’ve sent your suggestion to the product team and hopefully they’ll think about the possibility of allowing periods in topic names or removing the limitation as you stated.