Airbyte API Unprocessable Entity Error After Connector Upgrade

Slack Knowledge Base Q&A API, Terraform and Other Topics
, , , ,
1

Summary

User is encountering unprocessable entity errors when configuring a GCS source via the Airbyte API after upgrading a connector. They are seeking documentation for the required payload structure.


Question

HI, I have an Airbyte OSS instance. I am trying to use the Airbyte API to configure sources and destinations. It worked until I upgraded one of my connectors. Now I’m getting unprocessable entity errors for the gcs source. Is there a swagger docs endpoint for me to look at the required structure?

Checking https://reference.airbyte.com/reference/createsource doesn’t help as it has the same structure I was using before the connector was updated.

Code causing the error for reference:

            "name": stream,
            "workspaceId": workspace_id,
            "configuration": {
                "sourceType": "gcs",
                "service_account": service_account_key,
                "bucket": bucket_name,
                "streams": [
                    {
                        "name": streams[stream],
                        "globs": [f"**{streams[stream]}.parquet"],
                        "validation_policy": "Emit Record",
                        "schemaless": False,
                        "format": {"filetype": "parquet", "decimal_as_float": False},
                    }
                ],
            },
        }```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1731667441180559) if you want
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
['airbyte-api', 'gcs-source', 'unprocessable-entity', 'swagger-docs', 'connector-upgrade']
</sub>
2

> until I upgraded one of my connectors
which one? GCS source connector? from which version to which version?

When asking for help, please be precise as much as possible, because details matter

3

GCS Source Connector, can’t remember what version it was previously, and upgraded to the latest available.
I just figured out the change of structure now by inspecting the request that was sent when creating the source via UI. There is a new credentials block inside the configuration block, instead of simply passing the service account key and value.

In any case, the question stands, is there a swagger endpoint for the OSS users of the public/v1 api?

4

As far as I know there is only internal api /api/v1/source_definition_specifications/get
but you can use Reference section on connector’s page https://docs.airbyte.com/integrations/sources/gcs#reference

5

Thanks will take a look!

6

> until I upgraded one of my connectors
which one? GCS source connector? from which version to which version?

When asking for help, please be precise as much as possible, because details matter

7

GCS Source Connector, can’t remember what version it was previously, and upgraded to the latest available.
I just figured out the change of structure now by inspecting the request that was sent when creating the source via UI. There is a new credentials block inside the configuration block, instead of simply passing the service account key and value.

In any case, the question stands, is there a swagger endpoint for the OSS users of the public/v1 api?

8

As far as I know there is only internal api /api/v1/source_definition_specifications/get
but you can use Reference section on connector’s page https://docs.airbyte.com/integrations/sources/gcs#reference

9

Thanks will take a look!