Summary
User is experiencing issues with 429 Errors in a low-code connector, where the default backoff strategy is insufficient and custom error handling does not override the default behavior.
Question
Hello! Trying to create a low-code connector but hitting some limits.
On a specific stream, I am getting 429 Errors (Too Many Requests), and the default Backoff Strategy (Exponential, up to 5 times) is not suitable. However, if I set up a custom error handling for 429 Errors, it does not override the default.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
['low-code-connector', '429-errors', 'custom-error-handling', 'backoff-strategy']
Don’t know if it helps - the API is this: https://apidocs.awardforce.com/|AwardForce
Hello Natik! I was able to solve the problem in the meantime. The key takeaway is that if you build Error Handlers for specific streams using the UI builder, these won’t override the default behaviours (e.g. on 429)
The workaround is to act on the YAML itself with a custom block. In this case, the block sits inside each stream and looks like this:
response_filters:
- action: RETRY
http_codes:
- 429
- 500
- 502
- 504
backoff_strategies:
- type: ConstantBackoffStrategy
backoff_time_in_seconds: 60```
If you compare it with the block that is created when you build the error handler using the UI, it is different and does not override the default! It looks like this:
```error_handler:
type: CompositeErrorHandler
error_handlers:
- type: DefaultErrorHandler
max_retries: 5
response_filters:
- type: HttpResponseFilter
action: RETRY
http_codes:
- 429
- 500
- 502
- 504
backoff_strategies:
- type: ConstantBackoffStrategy
backoff_time_in_seconds: 60```
And I think this is more of a bug than a feature 
Don’t know if it helps - the API is this: https://apidocs.awardforce.com/|AwardForce
Hey Miguel!
The problem with error handlers in Builder is that Builder does not do retries. It will run error handlers, but if your action in the handler is exponential backoff, the Builder will not actually do it. It’s because test reads are designed to be fast, and we did not want to keep the user waiting for potentially long time while you’re backing off on rate limit (could be hours).
We will absolutely consider changing this, especially if we make Builder stream records in responses instead of waiting for all of them and presenting them together.
But for now, for you to test backoff, the easiest way is to publish the connector to your workspace and run a test sync and look at the logs. When you publish it, then at sync time backoff will be respected.
Hello Natik! I was able to solve the problem in the meantime. The key takeaway is that if you build Error Handlers for specific streams using the UI builder, these won’t override the default behaviours (e.g. on 429)
The workaround is to act on the YAML itself with a custom block. In this case, the block sits inside each stream and looks like this:
response_filters:
- action: RETRY
http_codes:
- 429
- 500
- 502
- 504
backoff_strategies:
- type: ConstantBackoffStrategy
backoff_time_in_seconds: 60```
If you compare it with the block that is created when you build the error handler using the UI, it is different and does not override the default! It looks like this:
```error_handler:
type: CompositeErrorHandler
error_handlers:
- type: DefaultErrorHandler
max_retries: 5
response_filters:
- type: HttpResponseFilter
action: RETRY
http_codes:
- 429
- 500
- 502
- 504
backoff_strategies:
- type: ConstantBackoffStrategy
backoff_time_in_seconds: 60```
And I think this is more of a bug than a feature