Manual Data Type Definition in MongoDB Connector

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

User inquires about the ability to manually set data types in the schema definition for the MongoDB connector due to non-uniform source documents causing inference issues.


Question

Hello,
I would like to ask if there is a way to manually set a data type in a schema definition for a connection. Specifically, in the MongoDB connector, we want to define the data type manually. This is necessary because our source dataset contains documents that are not uniform, and Airbyte appears to randomly select some source documents to infer the data type.
We are using Airbyte version 1.1 and the MongoDB connector version 1.5.12.
Thank you


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['manual-data-type', 'schema-definition', 'mongodb-connector', 'airbyte-1.1', 'airbyte-connector-1.5.12']
2

https://docs.airbyte.com/integrations/sources/mongodb-v2#schema-enforcement
It looks like there is no option to do that at this moment. You can enforce schema based on sample documents or use schema-less mode.

You might want to consider contributing to this connector.