Extracting all tables from source in Terraform provider

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

User is asking if it’s possible to extract all tables from a source in Terraform provider and apply a specific sync_mode.


Question

Hi,
Just wondering if in the terraform provider we can say, extract all tables from the source, and apply this sync_mode ?
Thanks


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["extract-all-tables", "source", "terraform-provider", "sync-mode"]
2

You can’t do that with Airbyte terraform provider. Configuring streams is on airbyte_connection resource level.

For what source do you want to do that? You could use data sources from different provider and pass them to configuration of airbyte_connection resource.

3

you can experiment with some like this, I don’t guarantee that it works, but you might give it a try

https://registry.terraform.io/providers/petoju/mysql/latest/docs/data-sources/tables

  database = "my_awesome_app"
}

resource "airbyte_connection" "my_awesome_connection" {

  configurations = {
    streams = [
      for table in data.mysql_tables.app.tables :
      {
        name      = table
        sync_mode = "incremental_append"
      }
    ]
  }
  
}```
4

sidenote: to make it work, wherever you run your terraform, it has to have access to MySQL database to fetch list of tables

5

if the list of tables doesn’t change that often, I would consider just static local variable

  tables = [...]
}```