Summary
The user is trying to connect Airbyte (abctl) to a MySQL instance running in a different container on Docker Desktop. They are looking for a way to specify the Docker Network when starting abctl.
Question
Running the abctl container on a Mac with Docker Desktop. Also have a MySQL Server running in a different container.
Tried adding the MySQL container to the “kind” network but can’t get Airbyte to connect to the MySQL instance. Is there a way to specify the Docker Network when starting abctl?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["abctl", "docker-network", "mysql-container", "docker-desktop"]
you tried docker network connect kind <mysql container name> , right?
I used --network kind in the docker run command
I run those commands:
$ docker inspect mysql | jq ".[0].NetworkSettings.Networks.kind.DNSNames"
[
"mysql",
"6cbe5a49d0db"
]```
then I used host: `mysql`, port: `3306`, username: `root`, password: `password` and it works for me just fine
Completely understand. The destination saved and tested successfully but I have yet to get a sync to work. It doesn’t seem to be able to write data and I’m unsure why.
What destination connector are you using? Do you get any errors in logs?
For testing you can use End-to-End Testing Destination https://docs.airbyte.com/integrations/destinations/e2e-test
Thank you Przemysław.
My basic mistake was that I was using the -p option to set a local port and was giving that port to Airbyte when, on the docker network, the default port was what was needed.
Not comfortable using root when I don’t have to so here’s my setup minus the passwords.
docker run -d --name test-mysql -e MYSQL_ROOT_PASSWORD=<<REDACTED_ROOT_PASS>> --network kind -p 33066:3306 mysql
mysql --host=127.0.0.1 --port=33066 -u root -p
CREATE USER ‘airbyte’@‘%’ IDENTIFIED BY ‘<<REDACTED_USER_PASS>>’;
CREATE SCHEMA nstest;
GRANT ALL PRIVILEGES ON nstest.* TO ‘airbyte’@‘%’;
SET GLOBAL local_infile = true;
Destination Settings:
Host: test-mysql
Port: 3306
DB NAme: nstest
User: airbyte
Optional Fields - Password: <<REDACTED_USER_PASS>>
My example was only proof of concept that communication between containers should work, but I agree that from security perspective it’s better to use container’s with non-root users. My approach is “make it work, then improve” 
You wrote the default port was what was needed , so I can assume that you solved your issue, right?
The end to end test worked but the MySQL connector is not, even though the destination tests successfully.
Where do I find the log files for abctl?
Hmm found this:
2024-08-16 15:42:36 destination > INFO main i.a.c.i.d.j.t.JdbcDestinationHandler(execute):171 Executing sql f8eedbe8-a8b8-4099-94bf-6bf9fb950a6b-c5da7c51-b6ca-45af-9ce7-8c9f169cc0ac: create database if not exists airbyte_internal;
161
2024-08-16 15:42:36 destination > ERROR main i.a.c.i.d.j.t.JdbcDestinationHandler(execute):182 Sql f8eedbe8-a8b8-4099-94bf-6bf9fb950a6b-c5da7c51-b6ca-45af-9ce7-8c9f169cc0ac failed java.sql.SQLSyntaxErrorException: Access denied for user ‘airbyte’@‘%’ to database ‘airbyte_internal’
The documentation doesn’t say anything about needing to create it’s own db…
GRANT ALL PRIVILEGES ON airbyte\_%.* TO ‘airbyte’@‘%’;
This allowed the airbyte user to create its own DB. Sync is successful but zero records being read by source NetSuite
got it! Forgot to push the Start Date back on the source.