How to update the heartbeat timeout in the connector builder for Airbyte
Hi all, In the connector builder I want to update the heartbeat timeout. How can I do it ?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
You can configure the heartbeat timeout in Airbyte by modifying the flags.yaml file. There are two entries related to the heartbeat: - heartbeat-max-seconds-between-messages: This configures the maximum time allowed between 2 messages. The default is 3 hours. - heartbeat.failSync: Setting this to true will make the syncs to fail if a missed heartbeat is detected. If false no sync will be failed because of a missed heartbeat. The default value is true. Here is the relevant extract from the documentation: yaml # Configuration The heartbeat can be configured using the file flags.yaml through 2 entries: - `heartbeat-max-seconds-between-messages`: this configures the maximum time allowed between 2 messages. The default is 3 hours. - `heartbeat.failSync`: Setting this to true will make the syncs to fail if a missed heartbeat is detected. If false no sync will be failed because of a missed heartbeat. The default value is true. Please note that modifying these settings should be done with caution as it can affect the behavior of your data synchronization.
<@U04SE163WC8> Is the flag.yaml the yaml show in the UI of the connector builder ?
No, the YAML file shown in the UI of the connector builder is not referred to as flag.yaml. It is called the Connector Manifest. This YAML file describes the features and functionality of the connector. You can iterate on this YAML file via the connector builder UI which makes it easy to inspect and debug your connector in greater detail than you would be able to through the command line. The Connector Manifest can be exported into a file and used to build a Docker image containing the connector which can be shared more widely. This YAML can be downloaded by clicking the Download Config button in the bottom-left of the connector builder UI. After creating the connector, you overwrite the contents of airbyte-integrations/connectors/source-<connector name>/source_<connector name>/manifest.yaml with the YAML you created in the UI. You can find more details about this process in the documentation.
Why do you want to increase the timeout or can you give more information of what timeout you want to increase?