BigQuery GCS staging failed

Troubleshooting
#1
  • Is this your first time deploying Airbyte?: No
  • OS Version: debian-10
  • Memory / Disk: 16GB
  • Deployment: Docker
  • Airbyte Version: 0.36.1-alpha
  • Source name/version: Hubspot 0.1.51
  • Destination name/version: BigQuery 1.1.4
  • Step: Loading Method GCS staging doesn’t run
  • Description:
    I tried moving BQ staging methood from standard insert to GCS staging but I’m getting very strange error

com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden

When configurating BigQuery settings the destination test passed successuly however, when I trigger a sync it doesn’t work.
standard insert works fine.
logs-911.txt (111.0 KB)

1 Like
Bigquery details: 400 Bad Request
Bigquery details: 400 Bad Request
#2

any ideas about this @marcosmarxm ? :slight_smile:

#3

Hi @ivan,
You receive errors from the AWS S3 client because S3 client is compatible with GCS, this is why we re-used the same client for integrations with S3 and GCS.
I think your service account is missing a set of permissions that are required for this loading method. Could you make sure the service account has permission to list, read objects (and metadata), and write in the bucket?

#4

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.

#5