File decompression - zstd

Connector Development
, ,
1

Hi,

I have a file from a 3rd party full of data I need to store in our data warehouse, and am evaluating data loading tools for a new tech team I’m building.

The file is JSON, but is compressed with zstd.

Can this be supported? If so what’s the likelihood this is worked on soon?

Cheers
Morgan

2

In case it’s not clear, the file is on a remote URI like this:

https://bucket.s3.eu-central-1.amazonaws.com/feed/feed.json.zst

3

Today this file format is not supported. I created the issue https://github.com/airbytehq/airbyte/issues/14294. Any updates I’ll return to you.

4

Thanks @marcosmarxm - I’ve subscribed to the issue on Github.