Summary
Airbyte is facing issues loading JSON files with a list of objects into a single column for a Postgres database due to default schemaless behavior. User is looking for a solution to parse the data using dbt.
Question
So, i am trying to move data from s3 folder to postgres database via airbyte which was working very well with all the configuration about connection setup.
But, now i have came across the situation where new type of json file have arrived which have list of json objects in them like
[{“key1”:“value1”, "key2:“value2”}]
i just want them to be loaded in under single column via airbyte and i can parse them using dbt later on.
But the issue is, the default schemaless behavior of airbyte is `
{“data”:“object”}
`
which leads to the issue of my json file having list of json object not single object.
And if i try to use the airbyte infer schema , it gives me file configuration error.
Have anyone faced this same issue ???
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.