Issues with API calls to Airbyte Cloud

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , ,
21

Also, the Airflow task was working just fine, but started failing sporadically as ya’ll were rolling out changes:

22

Hi Chris, thanks for providing additional details, that really helps. I now thinks it has something to do with security policy configuration. Your request are getting rejected by the LB which is why I did not see anything on the server telemetry.
I’m looking further into it now

23

If that is possible, could you try setting the user agent to something custom for your client?

24

Is that done via a header option?

25

The weird thing is when I run the example curl command from your docs on my local machine it actually works

26

It’s just the Python one that doesn’t work

27

Yes it is a header “User-Agent”

28

Okay, yeah. This worked:


url = "<https://api.airbyte.com/v1/jobs>"

payload = {
    "jobType": "sync",
    "connectionId": "9d23019a-d983-4802-829d-ba13c88ba68c"
}

headers = {
    "accept": "application/json",
    "content-type": "application/json",
    "authorization": "Bearer <redacted>",
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)```
that was just the user-agent that co-pilot recommended
29

but I can’t easily pass a user-agent to the Airflow Operator, that’s all abstracted as part of the pip packaging

30

Thanks for confirming! We’re gonna work on fixing those security rules on our end but that is gonna take a bit more time. I do not think we’ll be able to fix that over the weekend.

31

I cannot pass a header to the Airflow Operator
(unless I make another open-source contribution) -<https://github.com/apache/airflow/commit/25fd66c4b48fd940b11b09d3b590ab9d002cda11| I already did that for you guys this week> :smile:

32

probably makes more sense for you guys to fix it on your end because who knows who else this is impacting.

33

It definitely does, I just want to be careful when modifying those stuff. I’ll keep you posted on the resolution

34

Following up here in case anyone else was experiencing that issue. This is now resolved.