Issues with sending logs to GCS and setting data retention in Airbyte deployment on GCP

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

User is facing issues with sending logs to GCS and setting data retention in Airbyte deployment on GCP. Modifications made following documentation steps are not working as expected.


Question

Hi! I have an airbyte deployment with abctl running in GCP.

I have two issues:

  1. I cannot send logs to a GCS bucket. I modified the yaml file following the steps in the https://docs.airbyte.com/deploying-airbyte/integrations/storage|documentation. However, logs doesn’t get send to GCS. I don’t know where is the issue as some files are uploaded (workloads) while others don’t (logs)
  2. In relation with the previous step. As I cannot set logging in GCS I want to set a data retention of 5 days. I followed the https://docs.airbyte.com/operator-guides/configuring-airbyte|docs and I set the env TEMPORAL_HISTORY_RETENTION_IN_DAYS = 5 in the values.yml file. I checked that the var is in all the pods but it doesn’t seem to work.
    Thanks in advance!

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["airbyte-deployment", "gcp", "logs", "gcs", "data-retention", "abctl", "yaml-file", "values.yml", "temporal-history-retention-in-days"]