Using Terraform for Marketplace Connectors

Slack Knowledge Base Q&A Connector Development
, , , ,
1

Summary

User inquires about the possibility of utilizing Terraform for deploying Marketplace connectors, specifically referencing the MixMax integration documentation.


Question

Is there a way to use terraform for Marketplace connectors? Looking at MixMax: https://docs.airbyte.com/integrations/sources/mixmax


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

['terraform', 'marketplace-connectors', 'mixmax', 'airbyte']
2

Airbyte team generates and adds new connectors from time to time to terraform provider. If something is not supported with a special resource yet, you can use “generic” one

https://registry.terraform.io/providers/airbytehq/airbyte/latest/docs/resources/source_custom
https://registry.terraform.io/providers/airbytehq/airbyte/latest/docs/resources/destination_custom

:warning: I think there might be an error in docs
if configuration = { "user" : "charles" } doesn’t work, you need something like configuration = jsonencode({ "user" : "charles" })

3

I’m trying it for gong, get errors Inappropriate value for attribute "configuration": string required.

  configuration = {"gong_access_key": "abcd1234", "gong_access_secret": "hadslkfjhsadllkjhlasjdhf"}
  name          = "gong" ...```
4

adding the jsonencode:

│ 
│ Invalid value for "str" parameter: string required.
╵```
5

you didn’t use jsonencode

6

https://registry.terraform.io/providers/airbytehq/airbyte/latest/docs/resources/source_custom
• definition_id (String) The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Let me know if configuration works with definition_id = "32382e40-3b49-4b99-9c5c-4076501914e7"
https://github.com/airbytehq/airbyte/blob/5e17bbc604fc20b3d18e7205d437a85d8c419755/airbyte-integrations/connectors/source-gong/metadata.yaml#L9|metadata.yaml#L9

  configuration = jsonencode({
    access_key        = "..."
    access_key_secret = "..."
  })

  name = "gong"
  
  definition_id = "32382e40-3b49-4b99-9c5c-4076501914e7"

  # ...
}```
7

Another option is with sourceType in configuration

  configuration = jsonencode({
    access_key        = "..."
    access_key_secret = "..."
    sourceType        = "gong"
  })

  name = "gong"

  # ...
}```
8

Airbyte team generates and adds new connectors from time to time to terraform provider. If something is not supported with a special resource yet, you can use “generic” one

https://registry.terraform.io/providers/airbytehq/airbyte/latest/docs/resources/source_custom
https://registry.terraform.io/providers/airbytehq/airbyte/latest/docs/resources/destination_custom

:warning: I think there might be an error in docs
if configuration = { "user" : "charles" } doesn’t work, you need something like configuration = jsonencode({ "user" : "charles" })

9

I’m trying it for gong, get errors Inappropriate value for attribute "configuration": string required.

  configuration = {"gong_access_key": "abcd1234", "gong_access_secret": "hadslkfjhsadllkjhlasjdhf"}
  name          = "gong" ...```
10

adding the jsonencode:

│ 
│ Invalid value for "str" parameter: string required.
╵```
11

you didn’t use jsonencode

12 
  configuration = jsonencode({"gong_access_key": "...", "gong_access_secret": "..."})
  name          = "gong"
}```
13

│ unknown status code returned: Status 422
│ {“status”:422,“type”:“https://reference.airbyte.com/reference/errors#unprocessable-entity”,“title”:“unprocessable-entity”,“detail”:“The body of the request was not
│ understood”,“documentationUrl”:null,“data”:null}

14

the result using jsonencode :woman-facepalming:

15

every connector has specification about fields and their names

in case of source-gong connector https://github.com/airbytehq/airbyte/blob/5e17bbc604fc20b3d18e7205d437a85d8c419755/airbyte-integrations/connectors/source-gong/manifest.yaml#L252-L263|manifest.yaml#L252-L263
I think configuration will look like this

  configuration = jsonencode({
    access_key        = "..."
    access_key_secret = "..."
  })

  name = "gong"
  
  # ...
}```
16

still getting

│ unknown status code returned: Status 422
│ {"status":422,"type":"<https://reference.airbyte.com/reference/errors#unprocessable-entity>","title":"unprocessable-entity","detail":"The body of the request was not
│ understood","documentationUrl":null,"data":null}
╵```
17

https://registry.terraform.io/providers/airbytehq/airbyte/latest/docs/resources/source_custom
• definition_id (String) The UUID of the connector definition. One of configuration.sourceType or definitionId must be provided. Requires replacement if changed.
Let me know if configuration works with definition_id = "32382e40-3b49-4b99-9c5c-4076501914e7"
https://github.com/airbytehq/airbyte/blob/5e17bbc604fc20b3d18e7205d437a85d8c419755/airbyte-integrations/connectors/source-gong/metadata.yaml#L9|metadata.yaml#L9

  configuration = jsonencode({
    access_key        = "..."
    access_key_secret = "..."
  })

  name = "gong"
  
  definition_id = "32382e40-3b49-4b99-9c5c-4076501914e7"

  # ...
}```
18

Another option is with sourceType in configuration

  configuration = jsonencode({
    access_key        = "..."
    access_key_secret = "..."
    sourceType        = "gong"
  })

  name = "gong"

  # ...
}```
19

that worked! thank you, now I know where to get the definition id!