cwood32
September 7, 2022, 8:49pm
1
Are there instructions that are available of how to load custom certificates? I’ve seen post like and this branch these below:
You’ve probably already come up with your own solution, but just in case, I had this problem once Elasticsearch starting securing their connections by default (v8). I came up with a workaround by making a new “version” of the destination-elasticsearch connector by writing a Dockerfile that builds from the existing version, but adds the extra CA into the correct place for the JVM:
FROM airbyte/destination-elasticsearch:0.1.3
COPY ./http_ca.crt /usr/local/openjdk-17/lib/security/
COPY ./gd_bundle…
airbytehq:master ← awill1:awill1/add-documentdb-tls-support
opened 07:47PM - 09 Mar 22 UTC
## What
Solves #10388
This PR adds support to the MongoDB source connecto… r for Amazon DocumentDB when TLS is enabled. Prior to this PR, the connection could not be made because of a certificate validation error.
A new MongoDb Instance Type was added called *Amazon DocumentDB*.
Here is a screenshot of the new *Amazon DocumentDB* MongoDb Instance Type in the MongoDB source UI.
## How
Installs Amazon DocumentDB CA certs on the MongoDB source docker image in the default Java keystore. Having these certs installed in the docker image allows support for Amazon DocumentDB sources with TLS enabled.
A new MongoDb Instance Type was added called *Amazon DocumentDB*. This type was added because the jdbc connection string is slightly different as recommended by Amazon. Specifically it is connecting to a replica set and should use the cluster host.
## Recommended reading order
1. `airbyte-db/lib/src/main/java/io/airbyte/db/mongodb/MongoUtils.java`
2. `airbyte-integrations/connectors/source-mongodb-v2/src/main/resources/spec.json`
3. `airbyte-integrations/connectors/source-mongodb-v2/src/main/java/io.airbyte.integrations.source.mongodb/MongoDbSource.java`
4. `airbyte-integrations/connectors/source-mongodb-v2/Dockerfile`
## User Impact
There should be no breaking changes. This solution extends the existing MongoDb Instance Types and adds certs to the default keystore. No changes to existing code, only addition of code.
A new MongoDb Instance Type was added called *Amazon DocumentDB* in the MongoDB source UI..
## Pre-merge Checklist
Expand the relevant checklist and delete the others.
<details><summary><strong>Updating a connector</strong></summary>
### Community member or Airbyter
- [x] Grant edit access to maintainers ([instructions](https://docs.github.com/en/github/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork#enabling-repository-maintainer-permissions-on-existing-pull-requests))
- [x] Secrets in the connector's spec are annotated with `airbyte_secret`
- [ ] Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run `./gradlew :airbyte-integrations:connectors:<name>:integrationTest`.
- [ ] Code reviews completed
- [ ] Documentation updated
- [ ] Connector's `README.md`
- [ ] Connector's `bootstrap.md`. See [description and examples](https://docs.google.com/document/d/1ypdgmwmEHWv-TrO4_YOQ7pAJGVrMp5BOkEVh831N260/edit?usp=sharing)
- [ ] Changelog updated in `docs/integrations/<source or destination>/<name>.md` including changelog. See changelog [example](https://docs.airbyte.io/integrations/sources/stripe#changelog)
- [x] PR name follows [PR naming conventions](https://docs.airbyte.io/contributing-to-airbyte/updating-documentation#issues-and-pull-requests)
### Airbyter
If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.
- [ ] Create a non-forked branch based on this PR and test the below items on it
- [ ] Build is successful
- [ ] Credentials added to Github CI. [Instructions](https://docs.airbyte.io/connector-development#using-credentials-in-ci).
- [ ] [`/test connector=connectors/<name>` command](https://docs.airbyte.io/connector-development#updating-an-existing-connector) is passing
- [ ] New Connector version released on Dockerhub by running the `/publish` command described [here](https://docs.airbyte.io/connector-development#updating-an-existing-connector)
- [ ] After the new connector version is published, connector version bumped in the seed directory as described [here](https://docs.airbyte.io/connector-development#publishing-a-connector)
- [ ] Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described [here](https://docs.airbyte.io/connector-development#publishing-a-connector)
</details>
## Tests
<details><summary><strong>Unit</strong></summary>
*Put your unit tests output here.*
</details>
<details><summary><strong>Integration</strong></summary>
*Put your integration tests output here.*
</details>
<details><summary><strong>Acceptance</strong></summary>
*Put your acceptance tests output here.*
</details>
I’m using the Airbyte OSS and connecting to cloud databases that require certificates.
Or is there a time or backlog items for adding certificate just like the postgres connector?
sajarin
September 8, 2022, 6:51pm
2
Hi @cwood32 , I don’t believe there are currently instructions on how to load custom certificates for the Elasticsearch and MongoDB connectors. It’s been added to the team’s backlog but I don’t believe it’s a high priority item at the moment. We always encourage community contributions if you’re willing to add support. Otherwise, you’ll have to wait a bit longer. Hope this helps.