Using Airbyte 0.40.4 locally on Windows 10 using Docker Desktop
I am trying to set up my Airbyte to use Google Analytics (Uiniversal Analytics) connector
I can successfully set this connector up from my personal Macbook and also from the cloud version
However, as soon as I try to run it from my Windows 10 machine in the office it fails. I suspect this is something to do with the zScaler certificate not being copied
The message looks as below:-
2022-09-13 19:22:27 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Backing off _send(…) for 80.0s (requests.exceptions.SSLError: HTTPSConnectionPool(host=‘www.googleapis.com’, port=443): Max retries exceeded with url: /analytics/v3/metadata/ga/columns (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)’))))
2022-09-13 19:22:27 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Caught retryable error ‘HTTPSConnectionPool(host=‘www.googleapis.com’, port=443): Max retries exceeded with url: /analytics/v3/metadata/ga/columns (Caused by SSLError(SSLCertVerificationError(1, ‘[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)’)))’ after 5 tries. Waiting 80 seconds then retrying…
My guess is that I somehow need to copy the zScaler root certificate into the Dockerfile but I’m not sure how to go about that. I have made some changes to the Dockerfile but they do not seem to reflect when the container starts
Any guidance on how to resolve this issue or how to copy the zScaler root certificate from my windows machine into the docker container so the connector can then use it?
Many thanks in advance
Hello there! You are receiving this message because none of your fellow community members has stepped in to respond to your topic post. (If you are a community member and you are reading this response, feel free to jump in if you have the answer!) As a result, the Community Assistance Team has been made aware of this topic and will be investigating and responding as quickly as possible.
Some important considerations that will help your to get your issue solved faster:
- It is best to use our topic creation template; if you haven’t yet, we recommend posting a followup with the requested information. With that information the team will be able to more quickly search for similar issues with connectors and the platform and troubleshoot more quickly your specific question or problem.
- Make sure to upload the complete log file; a common investigation roadblock is that sometimes the error for the issue happens well before the problem is surfaced to the user, and so having the tail of the log is less useful than having the whole log to scan through.
- Be as descriptive and specific as possible; when investigating it is extremely valuable to know what steps were taken to encounter the issue, what version of connector / platform / Java / Python / docker / k8s was used, etc. The more context supplied, the quicker the investigation can start on your topic and the faster we can drive towards an answer.
- We in the Community Assistance Team are glad you’ve made yourself part of our community, and we’ll do our best to answer your questions and resolve the problems as quickly as possible. Expect to hear from a specific team member as soon as possible.
Thank you for your time and attention.
The Community Assistance Team
I’ve found some documentation on how to install the cert into Docker here: https://help.zscaler.com/zia/adding-custom-certificate-application-specific-trusted-store#docker
Hopefully that is helpful, let me know if it doesn’t resolve the issue and we can take a closer look.
Thanks for this @sh4sh , it has helped so I think I have made some progress on this but not quite all the way there yet
I followed the instructions on your link and I know that works as if I try to do a docker build wthout the certificate copied any commands that go out to the internet won’t work copy the .crt file across then they will
I then tried to run the app using docker compose up and it still doesn’t work which I guess is because airbyte-worker is pulling pre-built images from somewhere
After that I tried to build my own images to supersede those ones and build the platform. I’m using GitBash on Windows 10 and that builds every single image apart from the webapp which comes up with an error of
Task :airbyte-webapp:npm_run_build FAILED
ƒì╗ Start orval v6.8.1 - A swagger client generator for typescript
ƒÄë api - Your OpenAPI spec has been converted into ready to use orval!
BUILD_PATH=‘./build/app’ craco build
‘BUILD_PATH’ is not recognized as an internal or external command,
operable program or batch file.
I’m at a bit of loss of what to do with that - although I’m hopeful if I do sort it out it will sort out the entire problem
This looks like an environment variable that the task is trying to set, but the error
‘BUILD_PATH’ is not recognized as an internal or external command, operable program or batch file. is a Windows error. I believe you can set this environment variable in the
.env file in the repo, or if that doesn’t work for some reason you can see about adding it to Windows however I would not recommend that route if you can avoid it.