Clarification on process for creating new source and destination connectors for Astra DB

Slack Knowledge Base Q&A Connector Development
, , , ,
1

Summary

The user is seeking clarification on the process for creating new source and destination connectors for Astra DB with its new vector store features and JSON API.


Question

Hey, myself and a team at Anant Corporation have been working with Datastax to get new source and destination connectors create for Astra DB, specifically its new vector store features and its JSON API. We have pull requests up here https://github.com/airbytehq/airbyte/pull/34058 and here https://github.com/airbytehq/airbyte/pull/34052. Can you clarify our process from here? <@U069XPZR6H3>


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["datastax", "astra-db", "source-connector", "destination-connector", "vector-store", "json-api", "pull-request", "process"]
2

Sure, I’d be happy to clarify the process for you. Once you’ve submitted your pull requests, here’s what you can expect: 1. Wait for a review: Your pull requests will be reviewed by a community maintainer or a member of our team. This is to ensure that your contribution aligns with the existing works and the design orientation of the product. 2. Integration tests: To run integration tests, Airbyte needs access to a test account/environment. You should coordinate with an Airbyte engineer (via the PR) to add test credentials so that they can run tests for the integration in the CI. Once the config is stored in Github Secrets, you’ll need to edit .github/workflows/test-command.yml and .github/workflows/publish-command.yml to inject the config into the build environment. You’ll also need to edit the airbyte/tools/bin/ci_credentials.sh script to pull the script from the build environment and write it to secrets/config.json during the build. 3. Documentation: Don’t forget to update the documentation in docs/integrations/&lt;connector-name&gt;.md. 4. Pull Request Guidelines: Make sure your pull request follows the title convention and includes a link to an existing issue. Also, update the description as necessary. If you have any questions about a step in the process, you can include it in your PR or ask it on the #help-connector-development channel on Slack. Please note that due to project priorities, not all contributions may be accepted at this time. The team is currently prioritizing bug fixes, features, and enhancements to existing API source connectors, among other things. Contributions outside of these will be evaluated on a case-by-case basis by the engineering team. For more details, you can refer to the Code Contributions and Submit a New Connector sections in the Airbyte documentation.

3

Hi <@U06A0ECEGCB> , Thank you so much for opening these! I will take a look asap

4

I believe I have addressed the comments on the Destination PR. Please take another look when you have time. Planning to make applicable changes to the source PR as well, but please look that over as well in case there are source specific concerns that we need to address.

5

Thanks, I will check it ASAP

6

Any updates on the Astra Source PR?

7

<@U01MMSDJGC9> - I think you were the person who commented on the PR <@U06A0ECEGCB> is referencing: https://github.com/airbytehq/airbyte/pull/34052