- Is this your first time deploying Airbyte?: No
- Instance: GCP/Debian GNU/Linux 10 (buster)/e2-standard-2
- Memory/Disk: 16GB/200GB
- Deployment: Docker
- Airbyte Version: 0.40.4
- Source name/version: MySql 0.6.9
- Destination name/version: BigQuery 1.1.16
- Step: Sync
- Description: Sync fails as Airbyte can’t reach Mysql DB. This used to work well until a couple days ago. Log is attached.
If I test the connector I get:
Verification error(s) occurred for CHECK. Errors: [$.replication_method: should be valid to one and only one of the schemas , $.replication_method: string found, object expected]
If I try to create a new connector with the same properties I get:
Could not connect with provided configuration. Error: HikariPool-1 - Connection is not available, request timed out after 60002ms.
2022-09-07 15:48:22 e[32mINFOe[m i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed.
errors: $.method: must be a constant value Standard
2022-09-07 15:48:22 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-07 15:48:22 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-07 15:48:22 e[32mINFOe[m i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed.
errors: $.credential.hmac_key_access_id: object found, string expected, $.credential.hmac_key_secret: object found, string expected
2022-09-07 15:48:22 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):106 - Docker volume job log path: /tmp/workspace/1592/0/logs.log
2022-09-07 15:48:22 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):111 - Executing worker wrapper. Airbyte version: 0.40.4
2022-09-07 15:48:23 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-mysql:0.6.9 exists...
2022-09-07 15:48:23 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-mysql:0.6.9 was found locally.
2022-09-07 15:48:23 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = source-mysql-check-1592-0-cpkwq with resources io.airbyte.config.ResourceRequirements@753e1c3[cpuRequest=<null>,cpuLimit=<null>,memoryRequest=<null>,memoryLimit=<null>]
2022-09-07 15:48:23 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/1592/0 --log-driver none --name source-mysql-check-1592-0-cpkwq --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_JOB_ATTEMPT=0 -e WORKER_CONNECTOR_IMAGE=airbyte/source-mysql:0.6.9 -e AIRBYTE_VERSION=0.40.4 -e WORKER_JOB_ID=1592 airbyte/source-mysql:0.6.9 check --config source_config.json
2022-09-07 15:48:24 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:24 e[32mINFOe[m i.a.i.s.m.MySqlSource(main):242 - starting source: class io.airbyte.integrations.source.mysql.MySqlSource
2022-09-07 15:48:24 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:24 e[32mINFOe[m i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {check=null, config=source_config.json}
2022-09-07 15:48:24 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:24 e[32mINFOe[m i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-07 15:48:24 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:24 e[32mINFOe[m i.a.i.b.IntegrationRunner(runInternal):105 - Command: CHECK
2022-09-07 15:48:24 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:24 e[32mINFOe[m i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2022-09-07 15:48:25 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:25 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-07 15:48:25 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:25 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-07 15:48:25 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:25 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-07 15:48:25 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:25 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-07 15:48:25 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:25 e[32mINFOe[m i.a.i.b.s.SshTunnel(getInstance):172 - Starting connection with method: NO_TUNNEL
2022-09-07 15:48:25 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:25 e[32mINFOe[m c.z.h.HikariDataSource(<init>):80 - HikariPool-1 - Starting...
2022-09-07 15:48:25 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:25 e[32mINFOe[m c.z.h.HikariDataSource(<init>):82 - HikariPool-1 - Start completed.
2022-09-07 15:48:26 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:26 e[32mINFOe[m i.a.i.s.j.AbstractJdbcSource(lambda$getCheckOperations$1):140 - Attempting to get metadata from the database to see if we can connect.
2022-09-07 15:48:26 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:26 e[32mINFOe[m c.z.h.HikariDataSource(close):350 - HikariPool-1 - Shutdown initiated...
2022-09-07 15:48:26 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:26 e[32mINFOe[m c.z.h.HikariDataSource(close):352 - HikariPool-1 - Shutdown completed.
2022-09-07 15:48:26 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:26 e[32mINFOe[m i.a.i.b.IntegrationRunner(runInternal):152 - Completed integration: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-07 15:48:26 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-07 15:48:26 e[32mINFOe[m i.a.i.s.m.MySqlSource(main):244 - completed source: class io.airbyte.integrations.source.mysql.MySqlSource
2022-09-07 15:48:26 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling...
2022-09-07 15:48:26 e[1;31mERRORe[m i.a.s.p.JobNotifier(notifyJob):133 - Unable to read configuration:
java.lang.NullPointerException: null value in entry: max_seconds_before_source_state_message_emitted=null
at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33) ~[guava-31.0.1-jre.jar:?]
at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:337) ~[guava-31.0.1-jre.jar:?]
at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:447) ~[guava-31.0.1-jre.jar:?]
at io.airbyte.scheduler.persistence.job_tracker.TrackingMetadata.generateJobAttemptMetadata(TrackingMetadata.java:114) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
at io.airbyte.scheduler.persistence.JobNotifier.notifyJob(JobNotifier.java:82) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
at io.airbyte.scheduler.persistence.JobNotifier.notifyJob(JobNotifier.java:61) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
at io.airbyte.scheduler.persistence.JobNotifier.failJob(JobNotifier.java:167) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.jobFailure(JobCreationAndStatusUpdateActivityImpl.java:206) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
at java.lang.Thread.run(Thread.java:1589) [?:?]
2022-09-07 15:48:26 e[1;31mERRORe[m i.a.c.l.Exceptions(swallow):66 - Swallowed error.
java.lang.NullPointerException: null value in entry: max_seconds_before_source_state_message_emitted=null
at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33) ~[guava-31.0.1-jre.jar:?]
at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:337) ~[guava-31.0.1-jre.jar:?]
at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:447) ~[guava-31.0.1-jre.jar:?]
at io.airbyte.scheduler.persistence.job_tracker.TrackingMetadata.generateJobAttemptMetadata(TrackingMetadata.java:114) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
at io.airbyte.scheduler.persistence.job_tracker.JobTracker.generateJobAttemptMetadata(JobTracker.java:352) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
at io.airbyte.scheduler.persistence.job_tracker.JobTracker.lambda$trackSync$3(JobTracker.java:135) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
at io.airbyte.commons.lang.Exceptions.swallow(Exceptions.java:64) ~[io.airbyte-airbyte-commons-0.40.4.jar:?]
at io.airbyte.scheduler.persistence.job_tracker.JobTracker.trackSync(JobTracker.java:123) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.trackCompletion(JobCreationAndStatusUpdateActivityImpl.java:345) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.jobFailure(JobCreationAndStatusUpdateActivityImpl.java:208) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
at java.lang.Thread.run(Thread.java:1589) [?:?]
2022-09-07 15:48:26 e[32mINFOe[m i.a.c.f.EnvVariableFeatureFlags(autoDisablesFailingConnections):18 - Auto Disable Failing Connections: false ```