Redshift destination setup fails with S3 staging

  • Is this your first time deploying Airbyte?: Yes
  • OS Version / Instance: AMI Linux
  • Memory / Disk: 8GB / 60 GB
  • Deployment: Docker
  • Airbyte Version: * 0.38.2-alpha
  • Source name/version:
  • Destination name/version: Redshift/0.3.32
  • Step: The issue is happening during setting up the destination
  • Description: I am setting up redshift as a destination and getting error while opting for the s3 stage COPY method .

Error :
Could not connect to the staging persistence with the provided configuration. The request signature we calculated does not match the signature you provided. Check your key and signing method. (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: ; S3 Extended Request ID: =; Proxy: null)

While I checked the permissions for iam user it’s fine and also I could see a file written the stage bucket _airbyte_connection_test_286e4be858df49269f0c64581aa3e803

logs

2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550) ~[aws-java-sdk-core-1.12.6.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530) ~[aws-java-sdk-core-1.12.6.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5437) ~[aws-java-sdk-s3-1.12.6.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5384) ~[aws-java-sdk-s3-1.12.6.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at com.amazonaws.services.s3.AmazonS3Client.getAcl(AmazonS3Client.java:4050) ~[aws-java-sdk-s3-1.12.6.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1273) ~[aws-java-sdk-s3-1.12.6.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at com.amazonaws.services.s3.AmazonS3Client.getBucketAcl(AmazonS3Client.java:1263) ~[aws-java-sdk-s3-1.12.6.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExistV2(AmazonS3Client.java:1401) ~[aws-java-sdk-s3-1.12.6.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.s3.S3StorageOperations.doesBucketExist(S3StorageOperations.java:101) ~[io.airbyte.airbyte-integrations.connectors-destination-s3-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.s3.S3StorageOperations.createBucketObjectIfNotExists(S3StorageOperations.java:88) ~[io.airbyte.airbyte-integrations.connectors-destination-s3-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.s3.S3Destination.attemptWriteAndDeleteS3Object(S3Destination.java:152) ~[io.airbyte.airbyte-integrations.connectors-destination-s3-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.s3.S3Destination.attemptS3WriteAndDelete(S3Destination.java:143) ~[io.airbyte.airbyte-integrations.connectors-destination-s3-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.s3.S3Destination.attemptS3WriteAndDelete(S3Destination.java:133) ~[io.airbyte.airbyte-integrations.connectors-destination-s3-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.redshift.RedshiftCopyS3Destination.checkPersistence(RedshiftCopyS3Destination.java:63) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.jdbc.copy.CopyDestination.check(CopyDestination.java:49) [io.airbyte.airbyte-integrations.connectors-destination-jdbc-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.jdbc.copy.SwitchingDestination.check(SwitchingDestination.java:56) [io.airbyte.airbyte-integrations.connectors-destination-jdbc-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:138) [io.airbyte.airbyte-integrations.bases-base-java-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:105) [io.airbyte.airbyte-integrations.bases-base-java-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$0):61 - 	at io.airbyte.integrations.destination.redshift.RedshiftDestination.main(RedshiftDestination.java:71) [io.airbyte.airbyte-integrations.connectors-destination-redshift-0.36.1-alpha.jar:?]
2022-05-12 08:30:40 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...```

Though I am able to setup redshift destination  without s3 staging . 
Thanks in advance!

Can you share the complete log file Adnan?

Onsetting again and reviewing the iam permisssions it got Resolved

Awesome @Adnan! Could you share what you need to change or use for permissions in Redshift?