Summary
User inquires about dynamically combining monthly transactional files into a single table in Snowflake, specifically addressing the challenge of syncing files that are generated in the future.
Question
Hi Has anyone tried combining/unioning monthly transactional files dynamically into one single file in the destination warehouse (Snowflake in our case)?
For example,
file_feb_2024,
file_mar_2024,
file_apr_2024,
file_may_2024,
...
...```
All of above should be unioned into one single file called `files_2024`. However, the files for *Mar 2024 onwards* have not been generated yet. When they are generated on the first day of month, they will start syncing. I d want the records to load into the `files_2024` table in snowflake
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1708299009675709) if you want
to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
['snowflake', 'transactional-files', 'dynamic-union', 'data-sync']
</sub>