Issues with logs in UI and AWS S3 exceptions

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

User is facing issues with logs in UI and AWS S3 exceptions in Airbyte


Question

Hi everyone! New to Airbyte, facing a few issues with logs in UI as well as some AWS S3 exceptions. Will provide details in thread. Thanks!


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["issues", "logs-in-ui", "aws-s3", "exceptions", "airbyte-platform"]
2

I’ve deployed Airbyte using helm to EKS.
Upon setting up some test connections we noticed exceptions in DD as well as absence of some logs in UI

Exceptions like these:


Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: Unable to find a region via the region provider chain. Must provide an explicit region in the builder or setup environment to supply a region., com.amazonaws.auth.profile.ProfileCredentialsProvider@5209f548: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@576a634e: Unauthorized (Service: null; Status Code: 401; Error Code: null; Request ID: null; Proxy: null)]

com.amazonaws.AmazonServiceException: Unauthorized (Service: null; Status Code: 401; Error Code: null; Request ID: null; Proxy: null)```
They all come from ‘airbyte-container-orchestrator’ 
The connections/jobs went through just fine. I suspect that these errors are from Airbyte trying to push the logs to S3?

Here is my helmfile
```global:
  serviceAccountName: "airbyte"
 
  # -- Environment variables
  env_vars: 
    AWS_REGION: us-west-2
    DD_SERVICE: airbyte-job
    DD_SOURCE: airbyte-job

  storage:
    type: s3
    bucket:
      log: data-airbyte-external-logs
      state: data-airbyte-external-logs
      workloadOutput: data-airbyte-external-logs
      activityPayload: data-airbyte-external-logs

    s3:
      region: "us-west-2" 


serviceAccount:
  create: false
  annotations: 
    eks.amazonaws.com/role-arn: arn:aws:iam::117314662416:role/airbyteAppIrsa
  name: airbyte

Is there anything I’ve missed here?
When I check the bucket in AWS I see the following:
3 folders: / , app-logging/ , job-logging/
Each of them have folders/objects inside. So Airbyte clearly was able to push objects using the IRSA role.

The other problem is that in the Airbyte UI in the logs view I see only platform logs. Nothing for the rest (source/destinations, etc)
Not sure if this is related somehow.

I’m using the 0.263.0 helm release.
I haven’t set any AWS_KEYs vars in any workloads since I intend to use an IRSA role and not a user.

3

Also, setting AuthenticationType like mentioned https://docs.airbyte.com/deploying-airbyte/integrations/storage|here doesn’t seems to make any difference. No diff shows when running “helmfile apply”