Summary
Error message ‘[Errno 13] Permission denied: ‘/config/connectionConfiguration.json’’ when trying to use Mailchimp connector in local Airbyte Docker version
Question
i am trying a local version of Airbyte (in Docker) as an experiment… and trying to use the Mailchimp connector I keep getting an error:
[m i.a.w.i.VersionedAirbyteStreamFactory(internalLog):304 - [Errno 13] Permission denied: '/config/connectionConfiguration.json'
I’m struggling to find some way of solving it as well …
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["error", "mailchimp-connector", "local-docker", "permission-denied"]
One-one log is not enough to help here.
- Make sure you have the latest airbyte.
- Run with abctl. it will spin up local K8s cluster, don’t use docker compose.
Hi <@U069EMNRPA4>
It was spun up using the latest version of abctl available yesterday using the minimal resource option… for some reason I thought that was Docker, but I could easily be wrong…
Failure type: system_error```
File "/airbyte/integration_code/main.py", line 8, in <module>
run()
File "/airbyte/integration_code/source_mailchimp/run.py", line 15, in run
MigrateDataCenter.migrate(sys.argv[1:], source)
File "/airbyte/integration_code/source_mailchimp/config_migrations.py", line 96, in migrate
cls.emit_control_message(cls.modify_and_save(config_path, source, config))
File "/airbyte/integration_code/source_mailchimp/config_migrations.py", line 67, in modify_and_save
source.write_config(migrated_config, config_path)
File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/connector.py", line 60, in write_config
with open(config_path, "w") as fh:
PermissionError: [Errno 13] Permission denied: '/config/connectionConfiguration.json'```
2024-08-21 17:11:55 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):304 - [Errno 13] Permission denied: '/config/connectionConfiguration.json'
Traceback (most recent call last):
File "/airbyte/integration_code/main.py", line 8, in <module>
run()
File "/airbyte/integration_code/source_mailchimp/run.py", line 15, in run
MigrateDataCenter.migrate(sys.argv[1:], source)
File "/airbyte/integration_code/source_mailchimp/config_migrations.py", line 96, in migrate
cls.emit_control_message(cls.modify_and_save(config_path, source, config))
File "/airbyte/integration_code/source_mailchimp/config_migrations.py", line 67, in modify_and_save
source.write_config(migrated_config, config_path)
File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/connector.py", line 60, in write_config
with open(config_path, "w") as fh:
PermissionError: [Errno 13] Permission denied: '/config/connectionConfiguration.json'
2024-08-21 17:11:55 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):231 - Checking for optional control message...
2024-08-21 17:11:55 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):253 - Optional control message not found. Skipping...
2024-08-21 17:11:55 INFO i.a.c.ConnectorWatcher(run):134 - Writing output of b03a9f3e-22a5-11eb-adc1-0242ac120002_c06d511b-bd91-4715-a3b9-2a8a2713d0d6_0_check to the doc store
2024-08-21 17:11:55 INFO i.a.c.ConnectorWatcher(run):136 - Marking workload as successful
2024-08-21 17:11:56 INFO i.a.c.ConnectorWatcher(exitProperly):189 - Deliberately exiting process with code 0.
2024-08-21 17:11:56 INFO i.a.c.i.LineGobbler(voidCall):166 -
2024-08-21 17:11:56 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- END CHECK -----
2024-08-21 17:11:56 INFO i.a.c.i.LineGobbler(voidCall):166 - ```
Hmmm. I give up. Let me ask the deployments team
I also get this issue with Facebook Marketing connector. Fresh source. Using abctl on Ubuntu (all latest). Here are my logs:
2024-08-26 21:51:20 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):304 - [Errno 13] Permission denied: '/config/connectionConfiguration.json'
Traceback (most recent call last):
File "/airbyte/integration_code/main.py", line 9, in <module>
run()
File "/airbyte/integration_code/source_facebook_marketing/run.py", line 18, in run
MigrateSecretsPathInConnector.migrate(sys.argv[1:], source)
File "/airbyte/integration_code/source_facebook_marketing/config_migrations.py", line 160, in migrate
cls._modify_and_save(config_path, source, config),
File "/airbyte/integration_code/source_facebook_marketing/config_migrations.py", line 186, in _modify_and_save
source.write_config(migrated_config, config_path)
File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/connector.py", line 60, in write_config
with open(config_path, "w") as fh:
PermissionError: [Errno 13] Permission denied: '/config/connectionConfiguration.json'
2024-08-26 21:51:20 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):231 - Checking for optional control message...
2024-08-26 21:51:20 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):253 - Optional control message not found. Skipping...
2024-08-26 21:51:20 INFO i.a.c.ConnectorWatcher(run):134 - Writing output of e7778cfc-e97c-4458-9ecb-b4f2bba8946c_8e83c8fa-797c-4046-bd06-48517207460a_0_check to the doc store
2024-08-26 21:51:21 INFO i.a.c.ConnectorWatcher(run):136 - Marking workload as successful
2024-08-26 21:51:21 INFO i.a.c.ConnectorWatcher(exitProperly):189 - Deliberately exiting process with code 0.
2024-08-26 21:51:21 INFO i.a.c.i.LineGobbler(voidCall):166 -
2024-08-26 21:51:21 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- END CHECK -----
2024-08-26 21:51:21 INFO i.a.c.i.LineGobbler(voidCall):166 - ```
We’re looking into this internally. The path connectors are trying to write their config doesn’t quite look right, exploring where this comes from.
I am seeing this error as well when trying to set-up a GA4 Source.
<@U069EMNRPA4> Did you have any luck?
I’m fairly sure I’ve read the setup steps correctly but it still fails, and it would appear I am not necessarily the only one.
To clarify, both, are you setting up a new connection, or are you updating a connector on an existing one? Asking because of the failure in config_migrations specifically.
And, what version or Airbyte are you running, and how did you install it?
Assuming latest with abctl, but please confirm.
It looks like the connector code does not have the write permissions to perform it’s configuration migration, but I’m not sure why. Looking.
For me, I started receiving the error on my existing GA4 source after upgrading to latest version of Airbyte using abctl. I thought it might be because something didn’t migrate correctly, so I deleted and attempted to recreate the source, but continued to receive the same error.
Another user suggested downgrading the GA4 source in particular to v3.0.2 (I believe) and this resolved my issue.
can you share how the log that shows how the source connector is being executed?
I’d expect to see something like
2024-08-20 13:28:40 replication-orchestrator > kubectl cp /tmp/<...>/source_catalog.json jobs/source-<connector>-read-16538761-0-vlbpv:/config/source_catalog.json -c init --retries=3
For me, it’s a brand new install of the latest version using abctl on a ubutnu box…
The only log lines I know how to get are the ones I’ve placed above which come from the UI when running the ‘test’ for the connector… if there are other ways of getting the logs, then please do let me know…