Deprecate minio from gke deployment

Platform, Deploy & Infra Issues
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: GKE
  • Memory / Disk: 20 GB
  • Deployment: K8S
  • Airbyte Version: 0.35.10-alpha
  • Source name/version: bq
  • Destination name/version: bq
  • Step: Remove minio from gke deployment
  • Description: I am currently trying to deprecate minio from the gke k8s airbyte deployment and only use gcs for logging. When I remove the aws/s3 specific envars from the worker pod I get the following error:
Appender java.lang.IllegalStateException: No factory method found for class com.van.logging.log4j2.Log4j2Appender                                                                                                                             
        at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.findFactoryMethod(PluginBuilder.java:234)                                                                                                                          
        at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:134

As far as I can tell, this is due to the the logging plugin unable to find the directory to log in, are there any other envar I need to set? These are the ones I removed from the worker.yaml envars

- name: S3_LOG_BUCKET
  valueFrom:
    configMapKeyRef:
      name: airbyte-env
      key: S3_LOG_BUCKET
- name: S3_LOG_BUCKET_REGION
  valueFrom:
    configMapKeyRef:
      name: airbyte-env
      key: S3_LOG_BUCKET_REGION
- name: AWS_ACCESS_KEY_ID
  valueFrom:
    secretKeyRef:
      name: airbyte-secrets
      key: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
  valueFrom:
    secretKeyRef:
      name: airbyte-secrets
      key: AWS_SECRET_ACCESS_KEY
- name: S3_MINIO_ENDPOINT
  valueFrom:
    configMapKeyRef:
      name: airbyte-env
      key: S3_MINIO_ENDPOINT
- name: S3_PATH_STYLE_ACCESS
  valueFrom:
    configMapKeyRef:
      name: airbyte-env
      key: S3_PATH_STYLE_ACCESS
2

Hey I think rather removing those variables you have to leave them empty. I hope this helps On Kubernetes (Beta) | Airbyte Documentation

1 Like
4

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.