Summary
User seeks recommendations for managing alarms and notifications for Airbyte connections in a Kubernetes environment, considering Terraform for codification and potential integration with Airflow.
Question
I’ve got Airbyte up and running on our Kubernetes cluster. I’ve got two connections that sync our prod and dev DB to our singular Redshift cluster. I’m looking to setup alarms for the prod sync, and notifications for the dev sync.
I’ve got the metrics published to Datadog, but I’m not sure of the best approach to at scale, differentiate between connections that should have different alarm/notification configurations.
I plan on using Terraform to codify these connections, but click-ops my first two.
Alternatively, we are in process of setting up airflow. Perhaps I should setup alarms at the airflow level instead of Airbyte. Can anyone provide any recommendations on how to manage alarms?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.