Error running Sheets Sync

Brian_Donald · September 9, 2022, 10:00pm

Is this your first time deploying Airbyte?: Yes
OS Version / Instance: Ubuntu Server
Memory / Disk: 16gb / 64gb
Deployment: Docker (Swarm)
Airbyte Version: 0.40.4
Source name/version: Postgres / 1.0.7
Destination name/version: Google Sheets / 0.1.1
Step: During manual sync, ~24h after connection / destination setup
Description: When attempting to re-run a sync that has worked previously; I get these errors. Unsure if related to auth token refresh or this error I’m seeing in the logs.

2022-09-09 20:01:17 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-google-sheets:0.1.1 exists...
2022-09-09 20:01:17 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-google-sheets:0.1.1 was found locally.
2022-09-09 20:01:17 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-google-sheets-check-5-0-lkitt with resources io.airbyte.config.ResourceRequirements@34b1b880[cpuRequest=<null>,cpuLimit=<null>,memoryRequest=<null>,memoryLimit=<null>]
2022-09-09 20:01:17 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/5/0 --log-driver none --name destination-google-sheets-check-5-0-lkitt --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_JOB_ATTEMPT=0 -e WORKER_CONNECTOR_IMAGE=airbyte/destination-google-sheets:0.1.1 -e AIRBYTE_VERSION=0.40.4 -e WORKER_JOB_ID=5 airbyte/destination-google-sheets:0.1.1 check --config source_config.json
2022-09-09 20:01:19 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Auth session is expired. Refreshing...
2022-09-09 20:01:19 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling...

2022-09-09 20:01:20 e[1;31mERRORe[m i.a.s.p.JobNotifier(notifyJob):133 - Unable to read configuration:
java.lang.NullPointerException: null value in entry: max_seconds_before_source_state_message_emitted=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:337) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:447) ~[guava-31.0.1-jre.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.TrackingMetadata.generateJobAttemptMetadata(TrackingMetadata.java:114) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.notifyJob(JobNotifier.java:82) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.notifyJob(JobNotifier.java:61) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.failJob(JobNotifier.java:167) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.jobFailure(JobCreationAndStatusUpdateActivityImpl.java:206) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
	at java.lang.Thread.run(Thread.java:1589) [?:?]

marcosmarxm · September 12, 2022, 7:02pm

Hello there! You are receiving this message because none of your fellow community members has stepped in to respond to your topic post. (If you are a community member and you are reading this response, feel free to jump in if you have the answer!) As a result, the Community Assistance Team has been made aware of this topic and will be investigating and responding as quickly as possible.
Some important considerations that will help your to get your issue solved faster:

It is best to use our topic creation template; if you haven’t yet, we recommend posting a followup with the requested information. With that information the team will be able to more quickly search for similar issues with connectors and the platform and troubleshoot more quickly your specific question or problem.
Make sure to upload the complete log file; a common investigation roadblock is that sometimes the error for the issue happens well before the problem is surfaced to the user, and so having the tail of the log is less useful than having the whole log to scan through.
Be as descriptive and specific as possible; when investigating it is extremely valuable to know what steps were taken to encounter the issue, what version of connector / platform / Java / Python / docker / k8s was used, etc. The more context supplied, the quicker the investigation can start on your topic and the faster we can drive towards an answer.
We in the Community Assistance Team are glad you’ve made yourself part of our community, and we’ll do our best to answer your questions and resolve the problems as quickly as possible. Expect to hear from a specific team member as soon as possible.

Thank you for your time and attention.
Best,
The Community Assistance Team

harshith · September 13, 2022, 6:10am

Hey could you share the complete log file for this sync

Brian_Donald · September 14, 2022, 3:35pm

2022-09-09 20:01:11 INFO i.a.a.LoggingTrackingClient(track):43 - track. version: 0.40.4, userId: 5961c28a-0b1b-408a-b7ee-21cb4a69c0c3, action: Connector Jobs, metadata: {job_type=sync, namespace_definition=source, config.source.schemas=set, config.source.ssl_mode.mode=disable, config.source.port=set, frequency=manual, connector_source_definition_id=decd338e-5647-4c0b-adf4-da0e75f5a750, workspace_id=3f7976a8-94e5-4ec0-adfb-0694dae91db3, attempt_stage=STARTED, config.destination.credentials.client_secret=set, attempt_id=1, connector_destination=Google Sheets, connector_destination_docker_repository=airbyte/destination-google-sheets, table_prefix=false, workspace_name=3f7976a8-94e5-4ec0-adfb-0694dae91db3, catalog.destination_sync_mode.overwrite=set, number_of_streams=2, config.destination.credentials.client_id=set, config.destination.spreadsheet_id=set, connector_source=Postgres, config.source.host=set, config.source.tunnel_method.tunnel_method=NO_TUNNEL, config.source.username=set, connector_source_docker_repository=airbyte/source-postgres, config.source.database=set, config.source.password=set, config.source.ssl=false, config.source.replication_method.method=Standard, config.destination.credentials.refresh_token=set, catalog.sync_mode.full_refresh=set, connection_id=f5c256b2-31e1-4414-b84f-ad7b2b85f05c, job_id=5, connector_source_version=1.0.7, connector_destination_version=0.1.1, operation_count=0, connector_destination_definition_id=a4cbd2d1-8dbe-4818-b8bc-b90ad782d12a}
2022-09-09 20:01:11 INFO i.a.w.t.TemporalAttemptExecution(get):106 - Docker volume job log path: /tmp/workspace/5/0/logs.log
2022-09-09 20:01:11 INFO i.a.w.t.TemporalAttemptExecution(get):111 - Executing worker wrapper. Airbyte version: 0.40.4
2022-09-09 20:01:11 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-postgres:1.0.7 exists...
2022-09-09 20:01:12 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-postgres:1.0.7 was found locally.
2022-09-09 20:01:12 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = source-postgres-check-5-0-kwkvb with resources io.airbyte.config.ResourceRequirements@34b1b880[cpuRequest=<null>,cpuLimit=<null>,memoryRequest=<null>,memoryLimit=<null>]
2022-09-09 20:01:12 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/5/0 --log-driver none --name source-postgres-check-5-0-kwkvb --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_JOB_ATTEMPT=0 -e WORKER_CONNECTOR_IMAGE=airbyte/source-postgres:1.0.7 -e AIRBYTE_VERSION=0.40.4 -e WORKER_JOB_ID=5 airbyte/source-postgres:1.0.7 check --config source_config.json
2022-09-09 20:01:14 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:14 INFO i.a.i.b.a.AdaptiveSourceRunner$Runner(getSource):73 - Running source under deployment mode: OSS
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 INFO i.a.i.b.a.AdaptiveSourceRunner$Runner(run):85 - Starting source: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {check=null, config=source_config.json}
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: CHECK
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword min - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword max - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 INFO i.a.i.b.s.SshTunnel(getInstance):172 - Starting connection with method: NO_TUNNEL
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 INFO c.z.h.HikariDataSource(<init>):80 - HikariPool-1 - Starting...
2022-09-09 20:01:15 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:15 INFO c.z.h.HikariDataSource(<init>):82 - HikariPool-1 - Start completed.
2022-09-09 20:01:16 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:16 INFO i.a.i.s.p.PostgresUtils(isCdc):54 - using CDC: false
2022-09-09 20:01:16 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:16 INFO i.a.i.s.j.AbstractJdbcSource(lambda$getCheckOperations$1):140 - Attempting to get metadata from the database to see if we can connect.
2022-09-09 20:01:16 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:16 INFO c.z.h.HikariDataSource(close):350 - HikariPool-1 - Shutdown initiated...
2022-09-09 20:01:16 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:16 INFO c.z.h.HikariDataSource(close):352 - HikariPool-1 - Shutdown completed.
2022-09-09 20:01:16 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:16 INFO i.a.i.b.IntegrationRunner(runInternal):152 - Completed integration: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-09 20:01:16 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-09 20:01:16 INFO i.a.i.b.a.AdaptiveSourceRunner$Runner(run):87 - Completed source: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-09 20:01:17 INFO i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling...
2022-09-09 20:01:17 INFO i.a.w.t.TemporalAttemptExecution(get):106 - Docker volume job log path: /tmp/workspace/5/0/logs.log
2022-09-09 20:01:17 INFO i.a.w.t.TemporalAttemptExecution(get):111 - Executing worker wrapper. Airbyte version: 0.40.4
2022-09-09 20:01:17 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-google-sheets:0.1.1 exists...
2022-09-09 20:01:17 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-google-sheets:0.1.1 was found locally.
2022-09-09 20:01:17 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-google-sheets-check-5-0-lkitt with resources io.airbyte.config.ResourceRequirements@34b1b880[cpuRequest=<null>,cpuLimit=<null>,memoryRequest=<null>,memoryLimit=<null>]
2022-09-09 20:01:17 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/5/0 --log-driver none --name destination-google-sheets-check-5-0-lkitt --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_JOB_ATTEMPT=0 -e WORKER_CONNECTOR_IMAGE=airbyte/destination-google-sheets:0.1.1 -e AIRBYTE_VERSION=0.40.4 -e WORKER_JOB_ID=5 airbyte/destination-google-sheets:0.1.1 check --config source_config.json
2022-09-09 20:01:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Auth session is expired. Refreshing...
2022-09-09 20:01:19 INFO i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling...
2022-09-09 20:01:20 ERROR i.a.s.p.JobNotifier(notifyJob):133 - Unable to read configuration:
java.lang.NullPointerException: null value in entry: max_seconds_before_source_state_message_emitted=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:337) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:447) ~[guava-31.0.1-jre.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.TrackingMetadata.generateJobAttemptMetadata(TrackingMetadata.java:114) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.notifyJob(JobNotifier.java:82) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.notifyJob(JobNotifier.java:61) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.failJob(JobNotifier.java:167) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.jobFailure(JobCreationAndStatusUpdateActivityImpl.java:206) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
	at java.lang.Thread.run(Thread.java:1589) [?:?]
2022-09-09 20:01:20 ERROR i.a.c.l.Exceptions(swallow):66 - Swallowed error.
java.lang.NullPointerException: null value in entry: max_seconds_before_source_state_message_emitted=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:337) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:447) ~[guava-31.0.1-jre.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.TrackingMetadata.generateJobAttemptMetadata(TrackingMetadata.java:114) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.generateJobAttemptMetadata(JobTracker.java:352) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.lambda$trackSync$3(JobTracker.java:135) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.commons.lang.Exceptions.swallow(Exceptions.java:64) ~[io.airbyte-airbyte-commons-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.trackSync(JobTracker.java:123) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.trackCompletion(JobCreationAndStatusUpdateActivityImpl.java:345) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
	at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.jobFailure(JobCreationAndStatusUpdateActivityImpl.java:208) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
	at java.lang.Thread.run(Thread.java:1589) [?:?]
2022-09-09 20:01:20 INFO i.a.c.f.EnvVariableFeatureFlags(autoDisablesFailingConnections):18 - Auto Disable Failing Connections: false

harshith · September 15, 2022, 9:34am

Hey looks like the postgres is not returning data. Can you try with different combination of source and destination?

Brian_Donald · September 15, 2022, 4:22pm

2022-09-15 16:22:05 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-15 16:22:05 e[32mINFOe[m i.a.a.LoggingTrackingClient(track):43 - track. version: 0.40.4, userId: 5961c28a-0b1b-408a-b7ee-21cb4a69c0c3, action: Connector Jobs, metadata: {job_type=sync, namespace_definition=source, config.source.schemas=set, config.source.ssl_mode.mode=disable, config.source.port=set, frequency=manual, connector_source_definition_id=decd338e-5647-4c0b-adf4-da0e75f5a750, workspace_id=3f7976a8-94e5-4ec0-adfb-0694dae91db3, attempt_stage=STARTED, config.destination.credentials.client_secret=set, attempt_id=1, connector_destination=Google Sheets, connector_destination_docker_repository=airbyte/destination-google-sheets, table_prefix=false, workspace_name=3f7976a8-94e5-4ec0-adfb-0694dae91db3, catalog.destination_sync_mode.overwrite=set, number_of_streams=1, config.destination.credentials.client_id=set, config.destination.spreadsheet_id=set, connector_source=Postgres, config.source.host=set, config.source.tunnel_method.tunnel_method=NO_TUNNEL, config.source.username=set, connector_source_docker_repository=airbyte/source-postgres, config.source.database=set, config.source.password=set, config.source.ssl=false, config.source.replication_method.method=Standard, config.destination.credentials.refresh_token=set, catalog.sync_mode.full_refresh=set, connection_id=7c358dc6-cbb0-4bdc-b808-361c6d3b635a, job_id=6, connector_source_version=1.0.7, connector_destination_version=0.1.1, operation_count=0, connector_destination_definition_id=a4cbd2d1-8dbe-4818-b8bc-b90ad782d12a}
2022-09-15 16:22:05 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):106 - Docker volume job log path: /tmp/workspace/6/0/logs.log
2022-09-15 16:22:05 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):111 - Executing worker wrapper. Airbyte version: 0.40.4
2022-09-15 16:22:05 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-postgres:1.0.7 exists...
2022-09-15 16:22:05 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-postgres:1.0.7 was found locally.
2022-09-15 16:22:05 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = source-postgres-check-6-0-wrfhq with resources io.airbyte.config.ResourceRequirements@2127afb7[cpuRequest=<null>,cpuLimit=<null>,memoryRequest=<null>,memoryLimit=<null>]
2022-09-15 16:22:05 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/6/0 --log-driver none --name source-postgres-check-6-0-wrfhq --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_JOB_ATTEMPT=0 -e WORKER_CONNECTOR_IMAGE=airbyte/source-postgres:1.0.7 -e AIRBYTE_VERSION=0.40.4 -e WORKER_JOB_ID=6 airbyte/source-postgres:1.0.7 check --config source_config.json
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[32mINFOe[m i.a.i.b.a.AdaptiveSourceRunner$Runner(getSource):73 - Running source under deployment mode: OSS
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[32mINFOe[m i.a.i.b.a.AdaptiveSourceRunner$Runner(run):85 - Starting source: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[32mINFOe[m i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {check=null, config=source_config.json}
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[32mINFOe[m i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[32mINFOe[m i.a.i.b.IntegrationRunner(runInternal):105 - Command: CHECK
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[32mINFOe[m i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword multiline - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword min - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-15 16:22:07 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:07 e[33mWARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword max - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2022-09-15 16:22:08 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:08 e[32mINFOe[m i.a.i.b.s.SshTunnel(getInstance):172 - Starting connection with method: NO_TUNNEL
2022-09-15 16:22:08 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:08 e[32mINFOe[m c.z.h.HikariDataSource(<init>):80 - HikariPool-1 - Starting...
2022-09-15 16:22:08 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:08 e[32mINFOe[m c.z.h.HikariDataSource(<init>):82 - HikariPool-1 - Start completed.
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:09 e[32mINFOe[m i.a.i.s.p.PostgresUtils(isCdc):54 - using CDC: false
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:09 e[32mINFOe[m i.a.i.s.j.AbstractJdbcSource(lambda$getCheckOperations$1):140 - Attempting to get metadata from the database to see if we can connect.
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:09 e[32mINFOe[m c.z.h.HikariDataSource(close):350 - HikariPool-1 - Shutdown initiated...
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:09 e[32mINFOe[m c.z.h.HikariDataSource(close):352 - HikariPool-1 - Shutdown completed.
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:09 e[32mINFOe[m i.a.i.b.IntegrationRunner(runInternal):152 - Completed integration: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-09-15 16:22:09 e[32mINFOe[m i.a.i.b.a.AdaptiveSourceRunner$Runner(run):87 - Completed source: io.airbyte.integrations.base.ssh.SshWrappedSource
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling...
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):106 - Docker volume job log path: /tmp/workspace/6/0/logs.log
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):111 - Executing worker wrapper. Airbyte version: 0.40.4
2022-09-15 16:22:09 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-google-sheets:0.1.1 exists...
2022-09-15 16:22:09 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-google-sheets:0.1.1 was found locally.
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-google-sheets-check-6-0-dgccf with resources io.airbyte.config.ResourceRequirements@2127afb7[cpuRequest=<null>,cpuLimit=<null>,memoryRequest=<null>,memoryLimit=<null>]
2022-09-15 16:22:09 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/6/0 --log-driver none --name destination-google-sheets-check-6-0-dgccf --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_JOB_ATTEMPT=0 -e WORKER_CONNECTOR_IMAGE=airbyte/destination-google-sheets:0.1.1 -e AIRBYTE_VERSION=0.40.4 -e WORKER_JOB_ID=6 airbyte/destination-google-sheets:0.1.1 check --config source_config.json
2022-09-15 16:22:11 e[32mINFOe[m i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Auth session is expired. Refreshing...
2022-09-15 16:22:11 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling...
2022-09-15 16:22:12 e[1;31mERRORe[m i.a.s.p.JobNotifier(notifyJob):133 - Unable to read configuration:
java.lang.NullPointerException: null value in entry: max_seconds_before_source_state_message_emitted=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:337) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:447) ~[guava-31.0.1-jre.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.TrackingMetadata.generateJobAttemptMetadata(TrackingMetadata.java:114) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.notifyJob(JobNotifier.java:82) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.notifyJob(JobNotifier.java:61) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.JobNotifier.failJob(JobNotifier.java:167) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.jobFailure(JobCreationAndStatusUpdateActivityImpl.java:206) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
	at java.lang.Thread.run(Thread.java:1589) [?:?]
2022-09-15 16:22:12 e[1;31mERRORe[m i.a.c.l.Exceptions(swallow):66 - Swallowed error.
java.lang.NullPointerException: null value in entry: max_seconds_before_source_state_message_emitted=null
	at com.google.common.collect.CollectPreconditions.checkEntryNotNull(CollectPreconditions.java:33) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap.entryOf(ImmutableMap.java:337) ~[guava-31.0.1-jre.jar:?]
	at com.google.common.collect.ImmutableMap$Builder.put(ImmutableMap.java:447) ~[guava-31.0.1-jre.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.TrackingMetadata.generateJobAttemptMetadata(TrackingMetadata.java:114) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.generateJobAttemptMetadata(JobTracker.java:352) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.lambda$trackSync$3(JobTracker.java:135) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.commons.lang.Exceptions.swallow(Exceptions.java:64) ~[io.airbyte-airbyte-commons-0.40.4.jar:?]
	at io.airbyte.scheduler.persistence.job_tracker.JobTracker.trackSync(JobTracker.java:123) ~[io.airbyte.airbyte-scheduler-scheduler-persistence-0.40.4.jar:?]
	at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.trackCompletion(JobCreationAndStatusUpdateActivityImpl.java:345) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
	at io.airbyte.workers.temporal.scheduling.activities.JobCreationAndStatusUpdateActivityImpl.jobFailure(JobCreationAndStatusUpdateActivityImpl.java:208) ~[io.airbyte-airbyte-workers-0.40.4.jar:?]
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
	at java.lang.Thread.run(Thread.java:1589) [?:?]
2022-09-15 16:22:12 e[32mINFOe[m i.a.c.f.EnvVariableFeatureFlags(autoDisablesFailingConnections):18 - Auto Disable Failing Connections: false

harshith · September 19, 2022, 5:42am

Hey is the above sync for a different destination?

Brian_Donald · September 19, 2022, 3:44pm

Currently, I only have one destination configured, so it should be the same Google Sheets destination

harshith · September 20, 2022, 3:31pm

Hey I was checking if you have tried this Hey looks like the postgres is not returning data. Can you try with different combination of source and destination?

Topic		Replies	Views
Source Google Sheets connection tests failed Connector Questions & Issues source-google-sheets	7	839	November 21, 2022
Google sheets 0.2.30 missing rows Connector Questions & Issues source-google-sheets , data-loading	3	358	December 16, 2022
Unknown error occurred while setting up connection 0.40.27 Connector Questions & Issues docker , connectors	2	413	January 12, 2023
Source Google Search Console: Check connection failed Connector Questions & Issues source-google-search-console , connectors	5	621	September 13, 2022
ERROR: for airbyte-scheduler Connector Questions & Issues data-loading , connectors	6	600	July 14, 2022

Error running Sheets Sync

Related Topics