Summary
Unable to build a custom connector due to SSL certificate verification error. Attempted to install root CA in the airbyte-connector-builder-server pod but facing issues with custom docker image creation and pod startup.
Question
Hi team,
We deployed Airbyte Open Source Edition with official Helm chart on AWS EKS cluster. Currently we are not able to build custom connector to connect to our internal service. Here is the logs from the pod:
HTTPSConnectionPool(host='<http://api.dev|api.dev>.<<http://our_domain_name.com|our_domain_name.com>>', port=443): Max retries exceeded with url: /v1/analytics/proposal (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))
As I understand the error, I need to install the root CA in the airbyte-connector-builder-server pod.
Below are the steps I took to try to fix the error.
I tried to create a custom docker image based on the official airbyte/connector-builder-server image, then copy my root CA file, install the ca-certificates OS package and run the update-ca-trust OS command. After helm upgrade with my custom image, my pod just won’t start with error 127 in kubernetes logs and an error in the logs of the pod itself - /bin/bash: airbyte-app/bin/airbyte-connector-builder-server: No such file or directory.
I’ll be appreciate to any help.
Thanks
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["airbyte-open-source-edition", "aws-eks", "custom-connector", "ssl-certificate", "root-ca", "docker-image", "pod-startup"]
Hi guys, I will be appreciated to any advice here.
Yep yep yep. I’m in transit to airport, but I definitely have seen this before, so we should figure this out AND make a little note in our documentation about this.
<@U047ANT3J84> do you remember the last chat we had with Evan about this? We had to tell Python to trust a self-signed cert, but I don’t remember how, I think we ended up passing a path to root certs in an environment variable?
Ah yeah, helped with this with custom connectors but not builder connectors - so the work involved was in Airbyte CI and packaging the cert in the connector image itself, but didn’t require any Airbyte infra changes. The process sounds similar to what you did with creating your custom connector-builder-server image though. Can you share the dockerfile for the override?
That’s where things get into “and now you’re gonna deal with gradle” territory real quick. Let me talk with <@U04197GAK9R> today, see if folks can give us an env var knob to add root certs for the whole stack. Seems reasonable for OSS deploys.
<@U01MMSDJGC9> got some more details to share about the SSL question.
<@U069EMNRPA4> saw your react, do you think this is something Extensibility could help to answer?
Hello guys! 
Do you have any updates on this issue? At our company, we are also unable to adopt the connector builder due to this issue. As far as I know, patching the airbyte-connector-builder-server image with the certificates will only work for connector development and testing.
When the connector built using the connector builder actually runs, it uses a different image (airbyte/source-declarative-manifest) that is hardcoded and we can’t patch it with the certificates, so the problem is still going to happen. Or do you see any workarounds? 
FYI we have airbyte deployed with the official helm charts on EKS as well.