Automating documentation of synced tables from a Postgres source in Airbyte

Summary

Automating documentation of tables synced from a Postgres source in Airbyte using logs. Tips on maintaining documentation are also welcome.


Question

our team is trying to find a way to maintain documentation of all the tables we are syncing from a postgres source. has anyone figured out a way to automate documenting say, the tables being synced (or their state), by ingesting airbyte logs? any other tips on maintaining this documentation very much welcome too



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["automating-documentation", "synced-tables", "postgres-source", "airbyte-logs", "maintaining-documentation"]

what do you think about terraform?
you can have Airbyte configuration as a code and you have documented connections and streams
https://github.com/airbytehq/terraform-provider-airbyte

in output values you can store some details about connections and streams
I don’t think you can have state of the tables though

ohh this sounds promising, I’m going to look into it. thanks!