Summary
Multiple users are experiencing the same issue. Has anyone created a Github issue for this problem?
Question
<@U06UUGA4S1X> <@U040BPFQE7P> <@U06DQ4RHPUJ> <@U076UAVRD9P> <@U076Y4LLXUY> Looks like we’re all having the same issue , did anyone create a Github issue yet?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["multiple-users", "same-issue", "github-issue"]
<@U074J2Q04GL> what is the issue?
When trying to create a new Connection from a Source, the Schema discovery step when selecting Streams fails
For all of us, across different Connectors/Sources
Are you deploying a new instance or upgrade from a previous one. I saw some discusison happening https://github.com/airbytehq/airbyte/issues/38854. I already raised the issue to the platform team but would be really helpful to share all steps and information to make the issue reproducible
I did update the Facebook Connector yesterday but this doesn’t relate to that Connector in case that’s relevant. The issue right now has to do with the Files connector however
I’m running airbyte on a docker container and have deleted and spun back up the container a few times so i can tell you exactly what i’ve done :
• Add a sheets connector with a google service account
• Having a separate container running postgres, i create a destination connect to postgres with the ip, db name, etc.
let me know if you need other details <@U01MMSDJGC9>
For me, it was for a new environment… I set up a new airbyte in my AWS EC2 instance with docker compose… Configured both source and destination as postgres… And got the same issue when setting up the connector…
<@U040BPFQE7P> are you deploying with docker compose as well? I am running the ./run-ab-platform.sh
script on my local and its working without any issues. Might be a docker issue <@U01MMSDJGC9>
<@U040BPFQE7P> is this a new instance too?
Yes, running using docker compose with ./run-ab-platform.sh
. New EC2 instance. Same error on Amazon Linux AMI and Ubuntu.
Same issue (schema discovery failed while creating new connection) for me too.
Fresh installation. Amazon Linux, Docker Compose v2.27.1
with ./run-ab-platform.sh
on the latest airbyte version 0.61.0
, Postgres source v3.4.10
+ Postgres destination v2.0.10
. Same issue with mongo source v1.3.15
+ Postgres destination v2.0.10
as well.
Stacktrace using docker logs:
airbyte-worker | 2024-06-04T18:57:44.919372179Z Caused by: io.temporal.failure.ApplicationFailure: message='baseUrl is invalid.', type='java.lang.IllegalStateException', nonRetryable=false
I just tried building a new image starting from Ubuntu:jammy-20240427 and i got past this error. You guy should try the same if you can.
I am still stuck on my other error i mentioned earlier though, if anyone has seen it any help would be appreciated :
2024-06-04 19:12:12 destination > docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration: cannot enter cgroupv2 "/sys/fs/cgroup/docker" with domain controllers -- it is in threaded mode: unknown.
good luck
So for everyone using run-ab-platform
script. Please make sure whatever you run run-ab-platform -r
it actually updates all files (.env
, docker-compose.yaml
). It looks the script doesn’t update if you manually changed a file, for that reason you can run something like ls -la
and see if both files where updated, if not I strongly recommend to compare the existing file with the latest one in Github and see if something is missing.
I ran into the same problem locally but in a new folder running the command it worked as expected.
<@U06UUGA4S1X> <@U040BPFQE7P> <@U06DQ4RHPUJ> <@U076UAVRD9P> <@U076Y4LLXUY> did this solve your problem?
I’m also seeing this ‘Discovering schema failed’ issue now when trying load a local file for source. Also only seeing this error on Windows OS 11?
<@U074J2Q04GL> I did. You can currently fix by downgrading to 0.60.1 or when using helm charts set globalAirbyteUrl. See this issue for info
https://github.com/airbytehq/airbyte/issues/38854|https://github.com/airbytehq/airbyte/issues/38854
But your issue maybe different, even though it was saying Failed to discover schema, the root issue for me was “baseUrl was invalid” (from docker logs).
Yep, <@U076UAVRD9P> downgrading worked for me too…
Thanks <@U076UAVRD9P>, tested this yesterday and seems to be working now!