Configuring ADLS Gen2 storage in Airbyte deployed in Azure AKS

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

The user is trying to configure ADLS Gen2 storage in Airbyte deployed in Azure AKS but is facing issues as there is no configuration option for ADLS Gen2 on the official website. They have provided their current storage configuration code.


Question

We recently deployed Airbyte in Azure AKS and wanted to configure the status and log storage in ADLS Gen2, but there is no configuration option for ADLS Gen2 on the official website. The following is our actual code. We have not been able to successfully upgrade it. Can you please help me see where the problem lies?

    type: "S3"
    storageSecretName: airbyte-logs-secrets # Name of your Kubernetes secret.
    bucket: ## S3 bucket names that you've created. We recommend storing the following all in one bucket.
      log: airbyte-logs-bucket
      state: airbyte-state-bucket
      workloadOutput: airbyte-workload-output-bucket

    S3:
      region: "westus2"  ## e.g. us-east-1
      authenticationType: credentials ## Use "credentials" or "instanceProfile"
      accessKeyIdSecretKey: AWS_ACCESS_KEY_ID
      secretAccessKeySecretKey: AWS_SECRET_ACCESS_KEY
      endpoint: <https://adls4airbyte4prod.dfs.core.windows.net>```


<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1721900725275429) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["azure-aks", "adls-gen2", "storage-configuration", "airbyte-logs-secrets", "s3", "westus2", "credentials", "aws-access-key-id", "aws-secret-access-key", "adls4airbyte4prod"]
</sub>