Summary
User is experiencing an error 500 when trying to create a connector, mentioning issues with the Airbyte API and Terraform provider. Seeking help and suggestions.
Question
Hey there. Does anyone know if the Airbyte API and the terraform provider are working correctly? Because I am trying to create a connector now, and it is not working properly, returning an error 500, saying it cannot invoke API. I am using Airbyte version 0.63.12, terraform version 1.9, and airbyte terraform provider 0.6.1. Is anyone having the same problem? Any suggestions of wha to do?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["airbyte-api", "terraform-provider", "connector", "error-500", "version-0.63.12", "version-1.9", "version-0.6.1"]
Check this thread https://airbytehq.slack.com/archives/C021JANJ6TY/p1723677658921699?thread_ts=1723658239.285079&cid=C021JANJ6TY
As you’re using quite recent version, then it looks like you need this variant of configuration
server_url = "<https://your-airbyte-server/api/public/v1>"
} ```
Here you can find details that this configuration is needed due to changes in paths of endpoints
<https://reference.airbyte.com/reference/standalone-server-deprecation-and-migration-to-airbyte-server>
If you get an error like this:
│ HTTP/1.1 400 Bad Request
│ Content-Length: 301
│ Connection: keep-alive
│ Content-Type: application/json
│ Date: Thu, 15 Aug 2024 19:04:51 GMT
│
│ {"status":400,"type":"<https://reference.airbyte.com/reference/errors#resource-not-found>","title":"resource-not-found","detail":"The requested resource could not be
│ found.","documentationUrl":null,"data":{"resourceType":"STANDARD_SOURCE_DEFINITION","resourceId":"b0b68d5f-b4b9-49e2-b7dc-2833c76bbd55"}}```
you might also need to set `username` and `password` (works for me locally for Airbyte OSS)
or `client_id` and `client_secret` (doesn't work for me locally for Airbyte OSS)
Thank you, man! It is not 400 though, it is 500, and I think it already have the server url in this format. Any other ideas?
Yeah, one idea. Providing details.
Providing details about configuration (hide sensitive data). Providing details about error from terraform. Logs from terraform. What kind of deployment do you use or is it Cloud. Maybe also logs from your deployment if you use Airbyte OSS.
Without details is just guessing game what went wrong.
So true. Thank you! Yeah, I am using the OSS version. When I try to create a connection(and connection only) it returns the following error. I suspect that it maybe because either my database credentials are wrong, or because I have my postgres containerized in the same environment as the airbyte itself, but not within the same network, which may be causing this error. Do you think that’s a reasonable prediction?
╷
│ Error: failure to invoke API
│
│ with airbyte_connection.mysql_postgres,
│ on <http://connections.tf|connections.tf> line 1, in resource "airbyte_connection" "mysql_postgres":
│ 1: resource "airbyte_connection" "mysql_postgres" {
│
│ unknown status code returned: Status 500
│ {"message":"Internal Server Error:
│ null","exceptionClassName":"java.lang.NullPointerException","exceptionStack":["java.lang.NullPointerException","\tat
│ io.airbyte.server.apis.publicapi.services.SourceServiceImpl.getSourceSchema(SourceService.kt:210)","\tat
│ io.airbyte.server.apis.publicapi.controllers.ConnectionsController.publicCreateConnection$lambda$3(ConnectionsController.kt:97)","\tat
│ io.airbyte.server.apis.publicapi.apiTracking.TrackingHelper.callWithTracker(TrackingHelper.kt:107)","\tat
│ io.airbyte.server.apis.publicapi.controllers.ConnectionsController.publicCreateConnection(ConnectionsController.kt:96)","\tat
│ io.airbyte.server.apis.publicapi.controllers.$ConnectionsController$Definition$Exec.dispatch(Unknown
│ Source)","\tat
│ io.micronaut.context.AbstractExecutableMethodsDefinition$DispatchedExecutableMethod.invokeUnsafe(AbstractExecutableMethodsDefinition.java:461)","\tat
│ io.micronaut.context.DefaultBeanContext$BeanContextUnsafeExecutionHandle.invokeUnsafe(DefaultBeanContext.java:4326)","\tat
│ io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:271)","\tat
│ io.micronaut.web.router.DefaultUriRouteMatch.execute(DefaultUriRouteMatch.java:39)","\tat
│ io.micronaut.http.server.RouteExecutor.executeRouteAndConvertBody(RouteExecutor.java:490)","\tat
│ io.micronaut.http.server.RouteExecutor.lambda$callRoute$6(RouteExecutor.java:467)","\tat
│ io.micronaut.core.execution.ExecutionFlow.lambda$async$1(ExecutionFlow.java:87)","\tat
│ io.micronaut.core.propagation.PropagatedContext.lambda$wrap$3(PropagatedContext.java:211)","\tat
│ java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)","\tat
│ java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)","\tat
│ java.base/java.lang.Thread.run(Thread.java:1583)"],"rootCauseExceptionStack":[]}```
Are you able to create this connection manually? Yeah, your prediction is reasonable, but it might be another issue. In stacktrace there is getSourceSchema, so something with source connector?