Migrating Airbyte from AWS to GCP

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

Guide on migrating Airbyte from AWS to GCP


Question

Hello guys,
I want to migrate Airbyte from AWS to GCP. Can anyone help me with the steps or any documentation where I can get detail about it.


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["migrate", "airbyte", "aws", "gcp", "documentation"]
2

The big things will be:

  1. Moving the postgres database (including both Airbyte and Temporal DBs, usually there are 3 total) and restoring to your DB on GCP (we use Cloud SQL to externalize this)
  2. Update your values.yaml to map AWS-specific settings to their GCP counterparts (e.g. if you’re using S3 to externalize logs and state, you probably want to move this to GCS)
  3. Figure out how you want to connect—ingress is usually the trickiest part, but you’ll find several threads on here that talk about how we’re all handling that (usually this will be a Cloud Load Balancer, but how to deploy that is a bigger discussion)
    Some of the specifics also depend on whether you’re deploying to GKE or a single node using something like abctl. (We use GKE Autopilot and it’s worked really well for us.)