2022-10-07 08:52:06 INFO i.a.a.LoggingTrackingClient(track):43 - track. version: null, userId: null, action: Connector Jobs, metadata: {job_type=sync, config.destination.big_query_client_buffer_size_mb=set, namespace_definition=source, config.source.access_key=set, config.source.start_time=set, config.destination.loading_method.method=Standard, config.destination.dataset_location=EU, frequency=manual, connector_source_definition_id=72464cfe-c783-42bc-be28-35b2f02795bb, workspace_id=d27eeca0-5564-4ec7-939d-0fc34ac0691a, attempt_stage=STARTED, config.destination.project_id=set, attempt_id=1, connector_destination=BigQuery, catalog.sync_mode.incremental=set, connector_destination_docker_repository=airbyte/destination-bigquery, table_prefix=false, config.destination.credentials_json._secret=set, workspace_name=d27eeca0-5564-4ec7-939d-0fc34ac0691a, number_of_streams=1, config.destination.dataset_id=set, catalog.destination_sync_mode.append_dedup=set, connector_source=Plenigo Transactions, config.destination.transformation_priority=batch, connector_source_docker_repository=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector, connection_id=450d253d-9b99-4069-b2e9-d5baf00cd1c6, job_id=76, connector_source_version=latest, connector_destination_version=1.2.4, operation.normalization=1, operation_count=1, connector_destination_definition_id=22f6c74f-5699-40ff-833c-4a879ea40133} 2022-10-07 08:52:06 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/0/logs.log 2022-10-07 08:52:06 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:52:07 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:07 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2022-10-07 08:52:07 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:07 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest exists... 2022-10-07 08:52:07 INFO i.a.c.i.LineGobbler(voidCall):114 - europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest was found locally. 2022-10-07 08:52:07 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = airbyte-plenigo-connector-check-76-0-viimj with resources io.airbyte.config.ResourceRequirements@58e7381e[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:07 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/0 --log-driver none --name airbyte-plenigo-connector-check-76-0-viimj --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest -e WORKER_JOB_ATTEMPT=0 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest check --config source_config.json 2022-10-07 08:52:08 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Check succeeded 2022-10-07 08:52:08 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:52:08 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:08 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK ----- 2022-10-07 08:52:08 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:08 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/0/logs.log 2022-10-07 08:52:08 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:52:08 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:08 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2022-10-07 08:52:08 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:08 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-bigquery:1.2.4 exists... 2022-10-07 08:52:08 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-bigquery:1.2.4 was found locally. 2022-10-07 08:52:08 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-bigquery-check-76-0-cyxit with resources io.airbyte.config.ResourceRequirements@58e7381e[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:08 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/0 --log-driver none --name destination-bigquery-check-76-0-cyxit --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.2.4 -e WORKER_JOB_ATTEMPT=0 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 airbyte/destination-bigquery:1.2.4 check --config source_config.json 2022-10-07 08:52:11 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - integration args: {check=null, config=source_config.json} 2022-10-07 08:52:11 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:52:11 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Command: CHECK 2022-10-07 08:52:11 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'} 2022-10-07 08:52:11 WARN i.a.w.i.DefaultAirbyteStreamFactory(internalLog):100 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:52:11 WARN i.a.w.i.DefaultAirbyteStreamFactory(internalLog):100 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:52:11 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Selected loading method is set to: STANDARD 2022-10-07 08:52:14 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:52:14 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:52:14 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:14 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK ----- 2022-10-07 08:52:14 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:15 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/0/logs.log 2022-10-07 08:52:15 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:52:15 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable LOG_CONNECTOR_MESSAGES: 'false' 2022-10-07 08:52:15 INFO i.a.c.EnvConfigs(getEnvOrDefault):1072 - Using default value for environment variable METRIC_CLIENT: '' 2022-10-07 08:52:15 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 2022-10-07 08:52:15 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable LOG_CONNECTOR_MESSAGES: 'false' 2022-10-07 08:52:15 INFO i.a.w.g.DefaultReplicationWorker(run):122 - start sync worker. job id: 76 attempt id: 0 2022-10-07 08:52:15 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:15 INFO i.a.w.g.DefaultReplicationWorker(run):135 - configured sync modes: {null.incremental_transaction_stream=incremental - append_dedup} 2022-10-07 08:52:15 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START REPLICATION ----- 2022-10-07 08:52:15 INFO i.a.w.i.DefaultAirbyteDestination(start):69 - Running destination... 2022-10-07 08:52:15 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:15 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-bigquery:1.2.4 exists... 2022-10-07 08:52:15 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-bigquery:1.2.4 was found locally. 2022-10-07 08:52:15 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-bigquery-write-76-0-ifkkw with resources io.airbyte.config.ResourceRequirements@7626ee74[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:15 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/0 --log-driver none --name destination-bigquery-write-76-0-ifkkw --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.2.4 -e WORKER_JOB_ATTEMPT=0 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 airbyte/destination-bigquery:1.2.4 write --config destination_config.json --catalog destination_catalog.json 2022-10-07 08:52:15 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest exists... 2022-10-07 08:52:15 INFO i.a.c.i.LineGobbler(voidCall):114 - europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest was found locally. 2022-10-07 08:52:15 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = airbyte-plenigo-connector-read-76-0-wyghc with resources io.airbyte.config.ResourceRequirements@582d7c0a[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:15 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/0 --log-driver none --name airbyte-plenigo-connector-read-76-0-wyghc --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest -e WORKER_JOB_ATTEMPT=0 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest read --config source_config.json --catalog source_catalog.json 2022-10-07 08:52:15 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):435 - Destination output thread started. 2022-10-07 08:52:15 INFO i.a.w.g.DefaultReplicationWorker(run):177 - Waiting for source and destination threads to complete. 2022-10-07 08:52:15 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):322 - Replication thread started. 2022-10-07 08:52:16 source > Starting syncing SourcePlenigoTransactions 2022-10-07 08:52:16 source > Syncing stream: incremental_transaction_stream 2022-10-07 08:52:18 destination > integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-10-07 08:52:18 destination > Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:52:18 destination > Command: WRITE 2022-10-07 08:52:18 destination > Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-10-07 08:52:19 destination > Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:52:19 destination > Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:52:19 destination > Selected loading method is set to: STANDARD 2022-10-07 08:52:19 destination > The "standard" upload mode is not performant, and is not recommended for production. Please use the GCS upload mode if you are syncing a large amount of data. 2022-10-07 08:52:19 destination > Selected loading method is set to: STANDARD 2022-10-07 08:52:21 destination > Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_tmp_vzm_incremental_transaction_stream}} 2022-10-07 08:52:21 destination > Selected loading method is set to: STANDARD 2022-10-07 08:52:22 source > Read 843 records from incremental_transaction_stream stream 2022-10-07 08:52:22 source > Finished syncing incremental_transaction_stream 2022-10-07 08:52:22 source > SourcePlenigoTransactions runtimes: Syncing stream incremental_transaction_stream 0:00:00.960214 2022-10-07 08:52:22 source > Finished syncing SourcePlenigoTransactions 2022-10-07 08:52:22 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):363 - Total records read: 844 (409 KB) 2022-10-07 08:52:22 WARN i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$5):366 - Schema validation errors found for stream incremental_transaction_stream. Error messages: [$ is of an incorrect type. Expected it to be array] 2022-10-07 08:52:22 INFO i.a.w.g.DefaultReplicationWorker(run):182 - One of source or destination thread complete. Waiting on the other. 2022-10-07 08:52:23 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):445 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@3e311f3f[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@30f6f689[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@34c9744f[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@71cbfcb2[name=incremental_transaction_stream,namespace=,additionalProperties={}],streamState={"startTime":"2022-10-07T08:42:41.645Z"},additionalProperties={}],global=,data={"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}},additionalProperties={}],trace=,additionalProperties={}] 2022-10-07 08:52:23 destination > Airbyte message consumer: succeeded. 2022-10-07 08:52:23 destination > Started closing all connections 2022-10-07 08:52:23 destination > Closing connector: AbstractBigQueryUploader{table=_airbyte_raw_incremental_transaction_stream, tmpTable=_airbyte_tmp_vzm_incremental_transaction_stream, syncMode=WRITE_APPEND, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter} 2022-10-07 08:52:24 destination > Waiting for job finish Job{job=JobId{project=npg-airbyte-destination-dev, job=d85dcdd7-620f-4f4e-ae5e-13bfe3331d45, location=EU}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1665132743717, endTime=null, startTime=1665132743982, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=npg-aribyte-bigquery-sa@npg-airbyte-destination-dev.iam.gserviceaccount.com, etag=1f5dHJO+dPVxuhznBvUHEg==, generatedId=npg-airbyte-destination-dev:EU.d85dcdd7-620f-4f4e-ae5e-13bfe3331d45, selfLink=https://www.googleapis.com/bigquery/v2/projects/npg-airbyte-destination-dev/jobs/d85dcdd7-620f-4f4e-ae5e-13bfe3331d45?location=EU, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, projectId=npg-airbyte-destination-dev, tableId=_airbyte_tmp_vzm_incremental_transaction_stream}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=CREATE_IF_NEEDED, writeDisposition=null, formatOptions=FormatOptions{format=NEWLINE_DELIMITED_JSON}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=null, schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=null, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}}. Status: JobStatus{state=RUNNING, error=null, executionErrors=null} 2022-10-07 08:52:27 destination > Job finish Job{job=JobId{project=npg-airbyte-destination-dev, job=d85dcdd7-620f-4f4e-ae5e-13bfe3331d45, location=EU}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1665132743717, endTime=null, startTime=1665132743982, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=npg-aribyte-bigquery-sa@npg-airbyte-destination-dev.iam.gserviceaccount.com, etag=1f5dHJO+dPVxuhznBvUHEg==, generatedId=npg-airbyte-destination-dev:EU.d85dcdd7-620f-4f4e-ae5e-13bfe3331d45, selfLink=https://www.googleapis.com/bigquery/v2/projects/npg-airbyte-destination-dev/jobs/d85dcdd7-620f-4f4e-ae5e-13bfe3331d45?location=EU, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, projectId=npg-airbyte-destination-dev, tableId=_airbyte_tmp_vzm_incremental_transaction_stream}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=CREATE_IF_NEEDED, writeDisposition=null, formatOptions=FormatOptions{format=NEWLINE_DELIMITED_JSON}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=null, schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=null, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}} with status JobStatus{state=RUNNING, error=null, executionErrors=null} 2022-10-07 08:52:27 destination > Uploading data from the tmp table _airbyte_tmp_vzm_incremental_transaction_stream to the source table _airbyte_raw_incremental_transaction_stream. 2022-10-07 08:52:27 destination > Replication finished with no explicit errors. Copying data from tmp tables to permanent 2022-10-07 08:52:30 destination > successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_tmp_vzm_incremental_transaction_stream}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_raw_incremental_transaction_stream}} 2022-10-07 08:52:30 destination > Data is successfully loaded to the source table _airbyte_raw_incremental_transaction_stream! 2022-10-07 08:52:30 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):445 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@4735dd53[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@40f02982[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@3fe456df[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@11e403fd[name=incremental_transaction_stream,namespace=,additionalProperties={}],streamState={"startTime":"2022-10-07T08:42:41.645Z"},additionalProperties={}],global=,data={"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}},additionalProperties={}],trace=,additionalProperties={}] 2022-10-07 08:52:30 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):196 - The message tracker encountered an issue that prevents committed record counts from being reliably computed. 2022-10-07 08:52:30 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):197 - This only impacts metadata and does not indicate a problem with actual sync data. 2022-10-07 08:52:30 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):198 - State hash -2068298252 was already committed, likely indicating a state hash collision io.airbyte.workers.internal.StateDeltaTracker$StateDeltaTrackerException: State hash -2068298252 was already committed, likely indicating a state hash collision at io.airbyte.workers.internal.StateDeltaTracker.commitStateHash(StateDeltaTracker.java:114) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.handleDestinationEmittedState(AirbyteMessageTracker.java:193) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.java:119) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$7(DefaultReplicationWorker.java:446) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:52:30 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):207 - The state message tracker was unable to match the destination state message to a corresponding source state message. 2022-10-07 08:52:30 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):208 - This only impacts metrics and does not indicate a problem with actual sync data. 2022-10-07 08:52:30 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):209 - Destination state message cannot be matched to corresponding Source state message. io.airbyte.workers.internal.StateMetricsTracker$StateMetricsTrackerNoStateMatchException: Destination state message cannot be matched to corresponding Source state message. at io.airbyte.workers.internal.StateMetricsTracker.findStartingTimeStampAndRemoveOlderEntries(StateMetricsTracker.java:147) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.StateMetricsTracker.updateStates(StateMetricsTracker.java:80) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.handleDestinationEmittedState(AirbyteMessageTracker.java:204) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.java:119) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$7(DefaultReplicationWorker.java:446) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:52:30 destination > Final state message is accepted. 2022-10-07 08:52:30 destination > Removing tmp tables... 2022-10-07 08:52:30 destination > Finishing destination process...completed 2022-10-07 08:52:30 destination > Closed connector: AbstractBigQueryUploader{table=_airbyte_raw_incremental_transaction_stream, tmpTable=_airbyte_tmp_vzm_incremental_transaction_stream, syncMode=WRITE_APPEND, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter} 2022-10-07 08:52:30 destination > Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:52:30 INFO i.a.w.g.DefaultReplicationWorker(run):184 - Source and destination threads complete. 2022-10-07 08:52:30 INFO i.a.w.g.DefaultReplicationWorker(run):279 - Source output at least one state message 2022-10-07 08:52:30 INFO i.a.w.g.DefaultReplicationWorker(run):285 - State capture: Updated state to: Optional[io.airbyte.config.State@47b012e2[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"incremental_transaction_stream"},"stream_state":{"startTime":"2022-10-07T08:42:41.645Z"}},"data":{"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}}}]]] 2022-10-07 08:52:30 INFO i.a.w.g.DefaultReplicationWorker(run):300 - sync summary: { "status" : "completed", "recordsSynced" : 843, "bytesSynced" : 418864, "startTime" : 1665132735284, "endTime" : 1665132750985, "totalStats" : { "recordsEmitted" : 843, "bytesEmitted" : 418864, "sourceStateMessagesEmitted" : 1, "destinationStateMessagesEmitted" : 2, "recordsCommitted" : 843, "meanSecondsBeforeSourceStateMessageEmitted" : 5, "maxSecondsBeforeSourceStateMessageEmitted" : 5 }, "streamStats" : [ { "streamName" : "incremental_transaction_stream", "stats" : { "recordsEmitted" : 843, "bytesEmitted" : 418864, "recordsCommitted" : 843 } } ] } 2022-10-07 08:52:30 INFO i.a.w.g.DefaultReplicationWorker(run):301 - failures: [ ] 2022-10-07 08:52:30 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:52:30 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):159 - sync summary: io.airbyte.config.StandardSyncOutput@3604aa26[standardSyncSummary=io.airbyte.config.StandardSyncSummary@73062282[status=completed,recordsSynced=843,bytesSynced=418864,startTime=1665132735284,endTime=1665132750985,totalStats=io.airbyte.config.SyncStats@29aaabc5[recordsEmitted=843,bytesEmitted=418864,sourceStateMessagesEmitted=1,destinationStateMessagesEmitted=2,recordsCommitted=843,meanSecondsBeforeSourceStateMessageEmitted=5,maxSecondsBeforeSourceStateMessageEmitted=5,maxSecondsBetweenStateMessageEmittedandCommitted=,meanSecondsBetweenStateMessageEmittedandCommitted=,additionalProperties={}],streamStats=[io.airbyte.config.StreamSyncStats@423f30a9[streamName=incremental_transaction_stream,stats=io.airbyte.config.SyncStats@5179e752[recordsEmitted=843,bytesEmitted=418864,sourceStateMessagesEmitted=,destinationStateMessagesEmitted=,recordsCommitted=843,meanSecondsBeforeSourceStateMessageEmitted=,maxSecondsBeforeSourceStateMessageEmitted=,maxSecondsBetweenStateMessageEmittedandCommitted=,meanSecondsBetweenStateMessageEmittedandCommitted=,additionalProperties={}]]]],normalizationSummary=,state=io.airbyte.config.State@47b012e2[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"incremental_transaction_stream"},"stream_state":{"startTime":"2022-10-07T08:42:41.645Z"}},"data":{"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}}}]],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@1329ddcd[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@6fa1b8ee[stream=io.airbyte.protocol.models.AirbyteStream@542067cb[name=incremental_transaction_stream,jsonSchema={"type":"array","items":{"type":"object","required":["transactionId","paymentProvider","customerId"],"properties":{"amount":{"type":"number"},"currency":{"type":"string"},"errorCode":{"type":"string"},"customerId":{"type":"string"},"changedDate":{"type":"string"},"description":{"type":"string"},"errorMessage":{"type":"string"},"paymentAction":{"type":"string"},"paymentMethod":{"type":"string"},"paymentStatus":{"type":"string"},"transactionId":{"type":"string"},"fulfillmentDate":{"type":"string"},"paymentProvider":{"type":"string"},"transactionDate":{"type":"string"},"pspTransactionId":{"type":"string"},"plenigoTransactionId":{"type":"string"}}},"$schema":"http://json-schema.org/draft-04/schema#"},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[startTime],sourceDefinedPrimaryKey=[[transactionId]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[startTime],destinationSyncMode=append_dedup,primaryKey=[[transactionId]],additionalProperties={}]],additionalProperties={}],failures=[]] 2022-10-07 08:52:30 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):280 - Stopping temporal heartbeating... 2022-10-07 08:52:30 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:30 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END REPLICATION ----- 2022-10-07 08:52:30 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:31 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable NEED_STATE_VALIDATION: 'true' 2022-10-07 08:52:31 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/0/logs.log 2022-10-07 08:52:31 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:52:31 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:31 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START DEFAULT NORMALIZATION ----- 2022-10-07 08:52:31 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:31 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):138 - Running with normalization version: airbyte/normalization:0.2.22 2022-10-07 08:52:31 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/normalization:0.2.22 exists... 2022-10-07 08:52:31 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/normalization:0.2.22 was found locally. 2022-10-07 08:52:31 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = normalization-normalize-76-0-jufox with resources io.airbyte.config.ResourceRequirements@7af6dc2b[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:31 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/0/normalize --log-driver none --name normalization-normalize-76-0-jufox --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_VERSION=0.40.10 airbyte/normalization:0.2.22 run --integration-type bigquery --config destination_config.json --catalog destination_catalog.json 2022-10-07 08:52:31 normalization > Running: transform-config --config destination_config.json --integration-type bigquery --out /data/76/0/normalize 2022-10-07 08:52:32 normalization > Namespace(config='destination_config.json', integration_type=, out='/data/76/0/normalize') 2022-10-07 08:52:32 normalization > transform_bigquery 2022-10-07 08:52:32 normalization > Running: transform-catalog --integration-type bigquery --profile-config-dir /data/76/0/normalize --catalog destination_catalog.json --out /data/76/0/normalize/models/generated/ --json-column _airbyte_data 2022-10-07 08:52:33 normalization > Processing destination_catalog.json... 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 2022-10-07 08:52:33 normalization > Showing destination_catalog.json to diagnose/debug errors (1): 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 2022-10-07 08:52:33 normalization > { 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "streams" 2022-10-07 08:52:33 normalization > { 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "stream" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "name" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "json_schema" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "items" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "required" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentProvider" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "customerId" 2022-10-07 08:52:33 normalization > ], 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "properties" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "amount" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "currency" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "errorCode" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "customerId" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "changedDate" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "description" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "errorMessage" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentAction" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentMethod" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentStatus" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "fulfillmentDate" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentProvider" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionDate" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "pspTransactionId" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "plenigoTransactionId" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:52:33 normalization > } 2022-10-07 08:52:33 normalization > } 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "$schema" 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "supported_sync_modes" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "full_refresh" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "incremental" 2022-10-07 08:52:33 normalization > ], 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "source_defined_cursor" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "default_cursor_field" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "startTime" 2022-10-07 08:52:33 normalization > ], 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "source_defined_primary_key" 2022-10-07 08:52:33 normalization > [ 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:52:33 normalization > ] 2022-10-07 08:52:33 normalization > ] 2022-10-07 08:52:33 normalization > }, 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "sync_mode" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "cursor_field" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "startTime" 2022-10-07 08:52:33 normalization > ], 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "destination_sync_mode" 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "primary_key" 2022-10-07 08:52:33 normalization > [ 2022-10-07 08:52:33 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:52:33 normalization > ] 2022-10-07 08:52:33 normalization > ] 2022-10-07 08:52:33 normalization > } 2022-10-07 08:52:33 normalization > ] 2022-10-07 08:52:33 normalization > } 2022-10-07 08:52:33 INFO i.a.w.g.DefaultNormalizationWorker(run):82 - Normalization executed in 2 seconds. 2022-10-07 08:52:33 normalization > Traceback (most recent call last): 2022-10-07 08:52:33 normalization > File "/usr/local/bin/transform-catalog", line 8, in 2022-10-07 08:52:33 normalization > sys.exit(main()) 2022-10-07 08:52:33 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 104, in main 2022-10-07 08:52:33 normalization > TransformCatalog().run(args) 2022-10-07 08:52:33 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 36, in run 2022-10-07 08:52:33 normalization > self.process_catalog() 2022-10-07 08:52:33 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):164 - Completing future exceptionally... io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:92) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:27) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:161) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:52:33 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 64, in process_catalog 2022-10-07 08:52:33 normalization > processor.process(catalog_file=catalog_file, json_column_name=json_col, default_schema=schema) 2022-10-07 08:52:33 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 55, in process 2022-10-07 08:52:33 normalization > stream_processors = self.build_stream_processor( 2022-10-07 08:52:33 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 138, in build_stream_processor 2022-10-07 08:52:33 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:52:33 normalization > properties = get_field(get_field(stream_config, "json_schema", message), "properties", message) 2022-10-07 08:52:33 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):280 - Stopping temporal heartbeating... 2022-10-07 08:52:33 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 230, in get_field 2022-10-07 08:52:33 normalization > raise KeyError(message) 2022-10-07 08:52:33 normalization > KeyError: "'json_schema'.'properties' are not defined for stream incremental_transaction_stream" 2022-10-07 08:52:33 WARN i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=aa711210-c995-36cb-b48c-cdc77eb2a90b, activityType=Normalize, attempt=1 java.lang.RuntimeException: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:278) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.normalize(NormalizationActivityImpl.java:93) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at jdk.internal.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] Caused by: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.temporal.serviceclient.CheckedExceptionWrapper.wrap(CheckedExceptionWrapper.java:56) ~[temporal-serviceclient-1.8.1.jar:?] at io.temporal.internal.sync.WorkflowInternal.wrap(WorkflowInternal.java:448) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.activity.Activity.wrap(Activity.java:51) ~[temporal-sdk-1.8.1.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:138) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$3(NormalizationActivityImpl.java:121) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:273) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] ... 13 more Caused by: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$3(NormalizationActivityImpl.java:121) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:273) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] ... 13 more Caused by: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:92) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:27) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:161) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] ... 1 more 2022-10-07 08:52:34 INFO i.a.a.LoggingTrackingClient(track):43 - track. version: null, userId: null, action: Connector Jobs, metadata: {job_type=sync, config.destination.big_query_client_buffer_size_mb=set, namespace_definition=source, config.source.access_key=set, config.source.start_time=set, config.destination.loading_method.method=Standard, config.destination.dataset_location=EU, frequency=manual, connector_source_definition_id=72464cfe-c783-42bc-be28-35b2f02795bb, workspace_id=d27eeca0-5564-4ec7-939d-0fc34ac0691a, attempt_stage=STARTED, config.destination.project_id=set, attempt_id=2, connector_destination=BigQuery, catalog.sync_mode.incremental=set, connector_destination_docker_repository=airbyte/destination-bigquery, table_prefix=false, config.destination.credentials_json._secret=set, workspace_name=d27eeca0-5564-4ec7-939d-0fc34ac0691a, number_of_streams=1, config.destination.dataset_id=set, catalog.destination_sync_mode.append_dedup=set, connector_source=Plenigo Transactions, config.destination.transformation_priority=batch, connector_source_docker_repository=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector, connection_id=450d253d-9b99-4069-b2e9-d5baf00cd1c6, job_id=76, connector_source_version=latest, connector_destination_version=1.2.4, operation.normalization=1, operation_count=1, connector_destination_definition_id=22f6c74f-5699-40ff-833c-4a879ea40133} 2022-10-07 08:52:34 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/1/logs.log 2022-10-07 08:52:34 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:52:34 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:34 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2022-10-07 08:52:34 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:34 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest exists... 2022-10-07 08:52:34 INFO i.a.c.i.LineGobbler(voidCall):114 - europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest was found locally. 2022-10-07 08:52:34 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = airbyte-plenigo-connector-check-76-1-cadwa with resources io.airbyte.config.ResourceRequirements@58e7381e[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:34 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/1 --log-driver none --name airbyte-plenigo-connector-check-76-1-cadwa --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest -e WORKER_JOB_ATTEMPT=1 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest check --config source_config.json 2022-10-07 08:52:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Check succeeded 2022-10-07 08:52:35 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:52:35 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:35 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK ----- 2022-10-07 08:52:35 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:36 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/1/logs.log 2022-10-07 08:52:36 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:52:36 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:36 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2022-10-07 08:52:36 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:36 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-bigquery:1.2.4 exists... 2022-10-07 08:52:36 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-bigquery:1.2.4 was found locally. 2022-10-07 08:52:36 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-bigquery-check-76-1-nmjlp with resources io.airbyte.config.ResourceRequirements@58e7381e[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:36 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/1 --log-driver none --name destination-bigquery-check-76-1-nmjlp --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.2.4 -e WORKER_JOB_ATTEMPT=1 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 airbyte/destination-bigquery:1.2.4 check --config source_config.json 2022-10-07 08:52:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - integration args: {check=null, config=source_config.json} 2022-10-07 08:52:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:52:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Command: CHECK 2022-10-07 08:52:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'} 2022-10-07 08:52:38 WARN i.a.w.i.DefaultAirbyteStreamFactory(internalLog):100 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:52:38 WARN i.a.w.i.DefaultAirbyteStreamFactory(internalLog):100 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:52:39 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Selected loading method is set to: STANDARD 2022-10-07 08:52:42 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:52:42 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK ----- 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:42 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/1/logs.log 2022-10-07 08:52:42 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:52:42 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable LOG_CONNECTOR_MESSAGES: 'false' 2022-10-07 08:52:42 INFO i.a.c.EnvConfigs(getEnvOrDefault):1072 - Using default value for environment variable METRIC_CLIENT: '' 2022-10-07 08:52:42 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 2022-10-07 08:52:42 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable LOG_CONNECTOR_MESSAGES: 'false' 2022-10-07 08:52:42 INFO i.a.w.g.DefaultReplicationWorker(run):122 - start sync worker. job id: 76 attempt id: 1 2022-10-07 08:52:42 INFO i.a.w.g.DefaultReplicationWorker(run):135 - configured sync modes: {null.incremental_transaction_stream=incremental - append_dedup} 2022-10-07 08:52:42 INFO i.a.w.i.DefaultAirbyteDestination(start):69 - Running destination... 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START REPLICATION ----- 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-bigquery:1.2.4 exists... 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-bigquery:1.2.4 was found locally. 2022-10-07 08:52:42 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-bigquery-write-76-1-oisnn with resources io.airbyte.config.ResourceRequirements@2cc6f018[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:42 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/1 --log-driver none --name destination-bigquery-write-76-1-oisnn --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.2.4 -e WORKER_JOB_ATTEMPT=1 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 airbyte/destination-bigquery:1.2.4 write --config destination_config.json --catalog destination_catalog.json 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest exists... 2022-10-07 08:52:42 INFO i.a.c.i.LineGobbler(voidCall):114 - europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest was found locally. 2022-10-07 08:52:42 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = airbyte-plenigo-connector-read-76-1-fhftl with resources io.airbyte.config.ResourceRequirements@a2404ec[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:42 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/1 --log-driver none --name airbyte-plenigo-connector-read-76-1-fhftl --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest -e WORKER_JOB_ATTEMPT=1 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest read --config source_config.json --catalog source_catalog.json 2022-10-07 08:52:42 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):435 - Destination output thread started. 2022-10-07 08:52:42 INFO i.a.w.g.DefaultReplicationWorker(run):177 - Waiting for source and destination threads to complete. 2022-10-07 08:52:42 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):322 - Replication thread started. 2022-10-07 08:52:44 source > Starting syncing SourcePlenigoTransactions 2022-10-07 08:52:44 source > Syncing stream: incremental_transaction_stream 2022-10-07 08:52:46 destination > integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-10-07 08:52:46 destination > Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:52:46 destination > Command: WRITE 2022-10-07 08:52:46 destination > Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-10-07 08:52:46 destination > Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:52:46 destination > Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:52:46 destination > Selected loading method is set to: STANDARD 2022-10-07 08:52:46 destination > The "standard" upload mode is not performant, and is not recommended for production. Please use the GCS upload mode if you are syncing a large amount of data. 2022-10-07 08:52:46 destination > Selected loading method is set to: STANDARD 2022-10-07 08:52:49 destination > Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_tmp_jkz_incremental_transaction_stream}} 2022-10-07 08:52:49 destination > Selected loading method is set to: STANDARD 2022-10-07 08:52:49 source > Read 843 records from incremental_transaction_stream stream 2022-10-07 08:52:49 source > Finished syncing incremental_transaction_stream 2022-10-07 08:52:49 source > SourcePlenigoTransactions runtimes: Syncing stream incremental_transaction_stream 0:00:00.843134 2022-10-07 08:52:49 source > Finished syncing SourcePlenigoTransactions 2022-10-07 08:52:49 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):363 - Total records read: 844 (409 KB) 2022-10-07 08:52:49 WARN i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$5):366 - Schema validation errors found for stream incremental_transaction_stream. Error messages: [$ is of an incorrect type. Expected it to be array] 2022-10-07 08:52:49 INFO i.a.w.g.DefaultReplicationWorker(run):182 - One of source or destination thread complete. Waiting on the other. 2022-10-07 08:52:50 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):445 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@38495f09[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@368eb670[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@32b13479[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@77b3bc4b[name=incremental_transaction_stream,namespace=,additionalProperties={}],streamState={"startTime":"2022-10-07T08:42:41.645Z"},additionalProperties={}],global=,data={"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}},additionalProperties={}],trace=,additionalProperties={}] 2022-10-07 08:52:50 destination > Airbyte message consumer: succeeded. 2022-10-07 08:52:50 destination > Started closing all connections 2022-10-07 08:52:50 destination > Closing connector: AbstractBigQueryUploader{table=_airbyte_raw_incremental_transaction_stream, tmpTable=_airbyte_tmp_jkz_incremental_transaction_stream, syncMode=WRITE_APPEND, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter} 2022-10-07 08:52:51 destination > Waiting for job finish Job{job=JobId{project=npg-airbyte-destination-dev, job=8bc7a50d-feff-44eb-9f85-1f971ff8dda8, location=EU}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1665132770785, endTime=null, startTime=1665132771061, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=npg-aribyte-bigquery-sa@npg-airbyte-destination-dev.iam.gserviceaccount.com, etag=dQjr5HqW/F4Tmrjy77Z0/Q==, generatedId=npg-airbyte-destination-dev:EU.8bc7a50d-feff-44eb-9f85-1f971ff8dda8, selfLink=https://www.googleapis.com/bigquery/v2/projects/npg-airbyte-destination-dev/jobs/8bc7a50d-feff-44eb-9f85-1f971ff8dda8?location=EU, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, projectId=npg-airbyte-destination-dev, tableId=_airbyte_tmp_jkz_incremental_transaction_stream}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=CREATE_IF_NEEDED, writeDisposition=null, formatOptions=FormatOptions{format=NEWLINE_DELIMITED_JSON}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=null, schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=null, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}}. Status: JobStatus{state=RUNNING, error=null, executionErrors=null} 2022-10-07 08:52:55 destination > Job finish Job{job=JobId{project=npg-airbyte-destination-dev, job=8bc7a50d-feff-44eb-9f85-1f971ff8dda8, location=EU}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1665132770785, endTime=null, startTime=1665132771061, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=npg-aribyte-bigquery-sa@npg-airbyte-destination-dev.iam.gserviceaccount.com, etag=dQjr5HqW/F4Tmrjy77Z0/Q==, generatedId=npg-airbyte-destination-dev:EU.8bc7a50d-feff-44eb-9f85-1f971ff8dda8, selfLink=https://www.googleapis.com/bigquery/v2/projects/npg-airbyte-destination-dev/jobs/8bc7a50d-feff-44eb-9f85-1f971ff8dda8?location=EU, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, projectId=npg-airbyte-destination-dev, tableId=_airbyte_tmp_jkz_incremental_transaction_stream}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=CREATE_IF_NEEDED, writeDisposition=null, formatOptions=FormatOptions{format=NEWLINE_DELIMITED_JSON}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=null, schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=null, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}} with status JobStatus{state=RUNNING, error=null, executionErrors=null} 2022-10-07 08:52:55 destination > Uploading data from the tmp table _airbyte_tmp_jkz_incremental_transaction_stream to the source table _airbyte_raw_incremental_transaction_stream. 2022-10-07 08:52:55 destination > Replication finished with no explicit errors. Copying data from tmp tables to permanent 2022-10-07 08:52:58 destination > successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_tmp_jkz_incremental_transaction_stream}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_raw_incremental_transaction_stream}} 2022-10-07 08:52:58 destination > Data is successfully loaded to the source table _airbyte_raw_incremental_transaction_stream! 2022-10-07 08:52:58 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):445 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@5f014a3[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@468e0f5b[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@7d27f90f[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@179c4304[name=incremental_transaction_stream,namespace=,additionalProperties={}],streamState={"startTime":"2022-10-07T08:42:41.645Z"},additionalProperties={}],global=,data={"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}},additionalProperties={}],trace=,additionalProperties={}] 2022-10-07 08:52:58 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):196 - The message tracker encountered an issue that prevents committed record counts from being reliably computed. 2022-10-07 08:52:58 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):197 - This only impacts metadata and does not indicate a problem with actual sync data. 2022-10-07 08:52:58 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):198 - State hash -2068298252 was already committed, likely indicating a state hash collision io.airbyte.workers.internal.StateDeltaTracker$StateDeltaTrackerException: State hash -2068298252 was already committed, likely indicating a state hash collision at io.airbyte.workers.internal.StateDeltaTracker.commitStateHash(StateDeltaTracker.java:114) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.handleDestinationEmittedState(AirbyteMessageTracker.java:193) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.java:119) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$7(DefaultReplicationWorker.java:446) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:52:58 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):207 - The state message tracker was unable to match the destination state message to a corresponding source state message. 2022-10-07 08:52:58 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):208 - This only impacts metrics and does not indicate a problem with actual sync data. 2022-10-07 08:52:58 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):209 - Destination state message cannot be matched to corresponding Source state message. io.airbyte.workers.internal.StateMetricsTracker$StateMetricsTrackerNoStateMatchException: Destination state message cannot be matched to corresponding Source state message. at io.airbyte.workers.internal.StateMetricsTracker.findStartingTimeStampAndRemoveOlderEntries(StateMetricsTracker.java:147) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.StateMetricsTracker.updateStates(StateMetricsTracker.java:80) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.handleDestinationEmittedState(AirbyteMessageTracker.java:204) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.java:119) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$7(DefaultReplicationWorker.java:446) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:52:58 destination > Final state message is accepted. 2022-10-07 08:52:58 destination > Removing tmp tables... 2022-10-07 08:52:58 destination > Finishing destination process...completed 2022-10-07 08:52:58 destination > Closed connector: AbstractBigQueryUploader{table=_airbyte_raw_incremental_transaction_stream, tmpTable=_airbyte_tmp_jkz_incremental_transaction_stream, syncMode=WRITE_APPEND, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter} 2022-10-07 08:52:58 destination > Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:52:58 INFO i.a.w.g.DefaultReplicationWorker(run):184 - Source and destination threads complete. 2022-10-07 08:52:58 INFO i.a.w.g.DefaultReplicationWorker(run):279 - Source output at least one state message 2022-10-07 08:52:58 INFO i.a.w.g.DefaultReplicationWorker(run):285 - State capture: Updated state to: Optional[io.airbyte.config.State@3f9ef895[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"incremental_transaction_stream"},"stream_state":{"startTime":"2022-10-07T08:42:41.645Z"}},"data":{"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}}}]]] 2022-10-07 08:52:58 INFO i.a.w.g.DefaultReplicationWorker(run):300 - sync summary: { "status" : "completed", "recordsSynced" : 843, "bytesSynced" : 418864, "startTime" : 1665132762650, "endTime" : 1665132778450, "totalStats" : { "recordsEmitted" : 843, "bytesEmitted" : 418864, "sourceStateMessagesEmitted" : 1, "destinationStateMessagesEmitted" : 2, "recordsCommitted" : 843, "meanSecondsBeforeSourceStateMessageEmitted" : 5, "maxSecondsBeforeSourceStateMessageEmitted" : 5 }, "streamStats" : [ { "streamName" : "incremental_transaction_stream", "stats" : { "recordsEmitted" : 843, "bytesEmitted" : 418864, "recordsCommitted" : 843 } } ] } 2022-10-07 08:52:58 INFO i.a.w.g.DefaultReplicationWorker(run):301 - failures: [ ] 2022-10-07 08:52:58 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:52:58 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:58 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END REPLICATION ----- 2022-10-07 08:52:58 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:58 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):159 - sync summary: io.airbyte.config.StandardSyncOutput@531002ca[standardSyncSummary=io.airbyte.config.StandardSyncSummary@161296cc[status=completed,recordsSynced=843,bytesSynced=418864,startTime=1665132762650,endTime=1665132778450,totalStats=io.airbyte.config.SyncStats@cad980d[recordsEmitted=843,bytesEmitted=418864,sourceStateMessagesEmitted=1,destinationStateMessagesEmitted=2,recordsCommitted=843,meanSecondsBeforeSourceStateMessageEmitted=5,maxSecondsBeforeSourceStateMessageEmitted=5,maxSecondsBetweenStateMessageEmittedandCommitted=,meanSecondsBetweenStateMessageEmittedandCommitted=,additionalProperties={}],streamStats=[io.airbyte.config.StreamSyncStats@27ce654b[streamName=incremental_transaction_stream,stats=io.airbyte.config.SyncStats@749b352a[recordsEmitted=843,bytesEmitted=418864,sourceStateMessagesEmitted=,destinationStateMessagesEmitted=,recordsCommitted=843,meanSecondsBeforeSourceStateMessageEmitted=,maxSecondsBeforeSourceStateMessageEmitted=,maxSecondsBetweenStateMessageEmittedandCommitted=,meanSecondsBetweenStateMessageEmittedandCommitted=,additionalProperties={}]]]],normalizationSummary=,state=io.airbyte.config.State@3f9ef895[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"incremental_transaction_stream"},"stream_state":{"startTime":"2022-10-07T08:42:41.645Z"}},"data":{"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}}}]],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@763d6983[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@16b0fab9[stream=io.airbyte.protocol.models.AirbyteStream@42003119[name=incremental_transaction_stream,jsonSchema={"type":"array","items":{"type":"object","required":["transactionId","paymentProvider","customerId"],"properties":{"amount":{"type":"number"},"currency":{"type":"string"},"errorCode":{"type":"string"},"customerId":{"type":"string"},"changedDate":{"type":"string"},"description":{"type":"string"},"errorMessage":{"type":"string"},"paymentAction":{"type":"string"},"paymentMethod":{"type":"string"},"paymentStatus":{"type":"string"},"transactionId":{"type":"string"},"fulfillmentDate":{"type":"string"},"paymentProvider":{"type":"string"},"transactionDate":{"type":"string"},"pspTransactionId":{"type":"string"},"plenigoTransactionId":{"type":"string"}}},"$schema":"http://json-schema.org/draft-04/schema#"},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[startTime],sourceDefinedPrimaryKey=[[transactionId]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[startTime],destinationSyncMode=append_dedup,primaryKey=[[transactionId]],additionalProperties={}]],additionalProperties={}],failures=[]] 2022-10-07 08:52:58 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):280 - Stopping temporal heartbeating... 2022-10-07 08:52:58 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable NEED_STATE_VALIDATION: 'true' 2022-10-07 08:52:58 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/1/logs.log 2022-10-07 08:52:58 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:52:58 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:58 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):138 - Running with normalization version: airbyte/normalization:0.2.22 2022-10-07 08:52:58 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START DEFAULT NORMALIZATION ----- 2022-10-07 08:52:58 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:52:58 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/normalization:0.2.22 exists... 2022-10-07 08:52:58 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = normalization-normalize-76-1-bljvt with resources io.airbyte.config.ResourceRequirements@1746131c[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:52:58 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/1/normalize --log-driver none --name normalization-normalize-76-1-bljvt --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_VERSION=0.40.10 airbyte/normalization:0.2.22 run --integration-type bigquery --config destination_config.json --catalog destination_catalog.json 2022-10-07 08:52:58 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/normalization:0.2.22 was found locally. 2022-10-07 08:52:59 normalization > Running: transform-config --config destination_config.json --integration-type bigquery --out /data/76/1/normalize 2022-10-07 08:52:59 normalization > Namespace(config='destination_config.json', integration_type=, out='/data/76/1/normalize') 2022-10-07 08:52:59 normalization > transform_bigquery 2022-10-07 08:53:00 normalization > Running: transform-catalog --integration-type bigquery --profile-config-dir /data/76/1/normalize --catalog destination_catalog.json --out /data/76/1/normalize/models/generated/ --json-column _airbyte_data 2022-10-07 08:53:00 normalization > Processing destination_catalog.json... 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 2022-10-07 08:53:00 normalization > Showing destination_catalog.json to diagnose/debug errors (1): 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 2022-10-07 08:53:00 normalization > { 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "streams" 2022-10-07 08:53:00 normalization > { 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "stream" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "name" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "json_schema" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "items" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "required" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentProvider" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "customerId" 2022-10-07 08:53:00 normalization > ], 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "properties" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "amount" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "currency" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "errorCode" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "customerId" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "changedDate" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "description" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "errorMessage" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentAction" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentMethod" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentStatus" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "fulfillmentDate" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentProvider" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionDate" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "pspTransactionId" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "plenigoTransactionId" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:00 normalization > } 2022-10-07 08:53:00 normalization > } 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "$schema" 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "supported_sync_modes" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "full_refresh" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "incremental" 2022-10-07 08:53:00 normalization > ], 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "source_defined_cursor" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "default_cursor_field" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "startTime" 2022-10-07 08:53:00 normalization > ], 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "source_defined_primary_key" 2022-10-07 08:53:00 normalization > [ 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:53:00 normalization > ] 2022-10-07 08:53:00 normalization > ] 2022-10-07 08:53:00 normalization > }, 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "sync_mode" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "cursor_field" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "startTime" 2022-10-07 08:53:00 normalization > ], 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "destination_sync_mode" 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "primary_key" 2022-10-07 08:53:00 normalization > [ 2022-10-07 08:53:00 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:53:00 normalization > ] 2022-10-07 08:53:00 normalization > ] 2022-10-07 08:53:00 normalization > } 2022-10-07 08:53:00 normalization > ] 2022-10-07 08:53:00 normalization > } 2022-10-07 08:53:01 INFO i.a.w.g.DefaultNormalizationWorker(run):82 - Normalization executed in 2 seconds. 2022-10-07 08:53:01 normalization > Traceback (most recent call last): 2022-10-07 08:53:01 normalization > File "/usr/local/bin/transform-catalog", line 8, in 2022-10-07 08:53:01 normalization > sys.exit(main()) 2022-10-07 08:53:01 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 104, in main 2022-10-07 08:53:01 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):164 - Completing future exceptionally... io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:92) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:27) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:161) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:53:01 normalization > TransformCatalog().run(args) 2022-10-07 08:53:01 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 36, in run 2022-10-07 08:53:01 normalization > self.process_catalog() 2022-10-07 08:53:01 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:53:01 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 64, in process_catalog 2022-10-07 08:53:01 normalization > processor.process(catalog_file=catalog_file, json_column_name=json_col, default_schema=schema) 2022-10-07 08:53:01 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 55, in process 2022-10-07 08:53:01 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):280 - Stopping temporal heartbeating... 2022-10-07 08:53:01 normalization > stream_processors = self.build_stream_processor( 2022-10-07 08:53:01 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 138, in build_stream_processor 2022-10-07 08:53:01 normalization > properties = get_field(get_field(stream_config, "json_schema", message), "properties", message) 2022-10-07 08:53:01 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 230, in get_field 2022-10-07 08:53:01 normalization > raise KeyError(message) 2022-10-07 08:53:01 normalization > KeyError: "'json_schema'.'properties' are not defined for stream incremental_transaction_stream" 2022-10-07 08:53:01 WARN i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=3944d440-9227-39ff-a1b6-9601c1ef1dbe, activityType=Normalize, attempt=1 java.lang.RuntimeException: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:278) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.normalize(NormalizationActivityImpl.java:93) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at jdk.internal.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] Caused by: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.temporal.serviceclient.CheckedExceptionWrapper.wrap(CheckedExceptionWrapper.java:56) ~[temporal-serviceclient-1.8.1.jar:?] at io.temporal.internal.sync.WorkflowInternal.wrap(WorkflowInternal.java:448) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.activity.Activity.wrap(Activity.java:51) ~[temporal-sdk-1.8.1.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:138) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$3(NormalizationActivityImpl.java:121) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:273) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] ... 13 more Caused by: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$3(NormalizationActivityImpl.java:121) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:273) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] ... 13 more Caused by: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:92) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:27) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:161) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] ... 1 more 2022-10-07 08:53:01 INFO i.a.a.LoggingTrackingClient(track):43 - track. version: null, userId: null, action: Connector Jobs, metadata: {job_type=sync, config.destination.big_query_client_buffer_size_mb=set, namespace_definition=source, config.source.access_key=set, config.source.start_time=set, config.destination.loading_method.method=Standard, config.destination.dataset_location=EU, frequency=manual, connector_source_definition_id=72464cfe-c783-42bc-be28-35b2f02795bb, workspace_id=d27eeca0-5564-4ec7-939d-0fc34ac0691a, attempt_stage=STARTED, config.destination.project_id=set, attempt_id=3, connector_destination=BigQuery, catalog.sync_mode.incremental=set, connector_destination_docker_repository=airbyte/destination-bigquery, table_prefix=false, config.destination.credentials_json._secret=set, workspace_name=d27eeca0-5564-4ec7-939d-0fc34ac0691a, number_of_streams=1, config.destination.dataset_id=set, catalog.destination_sync_mode.append_dedup=set, connector_source=Plenigo Transactions, config.destination.transformation_priority=batch, connector_source_docker_repository=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector, connection_id=450d253d-9b99-4069-b2e9-d5baf00cd1c6, job_id=76, connector_source_version=latest, connector_destination_version=1.2.4, operation.normalization=1, operation_count=1, connector_destination_definition_id=22f6c74f-5699-40ff-833c-4a879ea40133} 2022-10-07 08:53:02 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/2/logs.log 2022-10-07 08:53:02 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:53:02 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:02 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2022-10-07 08:53:02 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:02 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest exists... 2022-10-07 08:53:02 INFO i.a.c.i.LineGobbler(voidCall):114 - europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest was found locally. 2022-10-07 08:53:02 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = airbyte-plenigo-connector-check-76-2-bcryx with resources io.airbyte.config.ResourceRequirements@58e7381e[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:53:02 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/2 --log-driver none --name airbyte-plenigo-connector-check-76-2-bcryx --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest -e WORKER_JOB_ATTEMPT=2 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest check --config source_config.json 2022-10-07 08:53:03 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Check succeeded 2022-10-07 08:53:03 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:53:03 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:03 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK ----- 2022-10-07 08:53:03 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:03 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/2/logs.log 2022-10-07 08:53:03 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:53:03 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:03 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2022-10-07 08:53:03 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:03 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-bigquery:1.2.4 exists... 2022-10-07 08:53:03 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-bigquery:1.2.4 was found locally. 2022-10-07 08:53:03 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-bigquery-check-76-2-uhqap with resources io.airbyte.config.ResourceRequirements@58e7381e[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:53:03 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/2 --log-driver none --name destination-bigquery-check-76-2-uhqap --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.2.4 -e WORKER_JOB_ATTEMPT=2 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 airbyte/destination-bigquery:1.2.4 check --config source_config.json 2022-10-07 08:53:05 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - integration args: {check=null, config=source_config.json} 2022-10-07 08:53:05 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:53:05 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Command: CHECK 2022-10-07 08:53:05 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'} 2022-10-07 08:53:06 WARN i.a.w.i.DefaultAirbyteStreamFactory(internalLog):100 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:53:06 WARN i.a.w.i.DefaultAirbyteStreamFactory(internalLog):100 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:53:06 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Selected loading method is set to: STANDARD 2022-10-07 08:53:09 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:53:09 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK ----- 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:09 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/2/logs.log 2022-10-07 08:53:09 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:53:09 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable LOG_CONNECTOR_MESSAGES: 'false' 2022-10-07 08:53:09 INFO i.a.c.EnvConfigs(getEnvOrDefault):1072 - Using default value for environment variable METRIC_CLIENT: '' 2022-10-07 08:53:09 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 2022-10-07 08:53:09 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable LOG_CONNECTOR_MESSAGES: 'false' 2022-10-07 08:53:09 INFO i.a.w.g.DefaultReplicationWorker(run):122 - start sync worker. job id: 76 attempt id: 2 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:09 INFO i.a.w.g.DefaultReplicationWorker(run):135 - configured sync modes: {null.incremental_transaction_stream=incremental - append_dedup} 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START REPLICATION ----- 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:09 INFO i.a.w.i.DefaultAirbyteDestination(start):69 - Running destination... 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/destination-bigquery:1.2.4 exists... 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/destination-bigquery:1.2.4 was found locally. 2022-10-07 08:53:09 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = destination-bigquery-write-76-2-aaveq with resources io.airbyte.config.ResourceRequirements@3d9ae173[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:53:09 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/2 --log-driver none --name destination-bigquery-write-76-2-aaveq --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.2.4 -e WORKER_JOB_ATTEMPT=2 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 airbyte/destination-bigquery:1.2.4 write --config destination_config.json --catalog destination_catalog.json 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest exists... 2022-10-07 08:53:09 INFO i.a.c.i.LineGobbler(voidCall):114 - europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest was found locally. 2022-10-07 08:53:09 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = airbyte-plenigo-connector-read-76-2-cmjia with resources io.airbyte.config.ResourceRequirements@2d40b9a7[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:53:09 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/2 --log-driver none --name airbyte-plenigo-connector-read-76-2-cmjia --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_CONNECTOR_IMAGE=europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest -e WORKER_JOB_ATTEMPT=2 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=76 europe-west3-docker.pkg.dev/npg-aw-dev3/airbyte-docker-repo/airbyte-plenigo-connector:latest read --config source_config.json --catalog source_catalog.json 2022-10-07 08:53:09 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):435 - Destination output thread started. 2022-10-07 08:53:09 INFO i.a.w.g.DefaultReplicationWorker(run):177 - Waiting for source and destination threads to complete. 2022-10-07 08:53:09 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):322 - Replication thread started. 2022-10-07 08:53:11 source > Starting syncing SourcePlenigoTransactions 2022-10-07 08:53:11 source > Syncing stream: incremental_transaction_stream 2022-10-07 08:53:13 destination > integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-10-07 08:53:13 destination > Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:53:13 destination > Command: WRITE 2022-10-07 08:53:13 destination > Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-10-07 08:53:13 destination > Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:53:13 destination > Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-07 08:53:13 destination > Selected loading method is set to: STANDARD 2022-10-07 08:53:13 destination > The "standard" upload mode is not performant, and is not recommended for production. Please use the GCS upload mode if you are syncing a large amount of data. 2022-10-07 08:53:14 destination > Selected loading method is set to: STANDARD 2022-10-07 08:53:16 destination > Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_tmp_owf_incremental_transaction_stream}} 2022-10-07 08:53:16 destination > Selected loading method is set to: STANDARD 2022-10-07 08:53:16 source > Read 843 records from incremental_transaction_stream stream 2022-10-07 08:53:16 source > Finished syncing incremental_transaction_stream 2022-10-07 08:53:16 source > SourcePlenigoTransactions runtimes: Syncing stream incremental_transaction_stream 0:00:00.947070 2022-10-07 08:53:16 source > Finished syncing SourcePlenigoTransactions 2022-10-07 08:53:16 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):363 - Total records read: 844 (409 KB) 2022-10-07 08:53:16 WARN i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$5):366 - Schema validation errors found for stream incremental_transaction_stream. Error messages: [$ is of an incorrect type. Expected it to be array] 2022-10-07 08:53:16 INFO i.a.w.g.DefaultReplicationWorker(run):182 - One of source or destination thread complete. Waiting on the other. 2022-10-07 08:53:17 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):445 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@489764da[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@465083b8[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@426f0faa[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@76d8983[name=incremental_transaction_stream,namespace=,additionalProperties={}],streamState={"startTime":"2022-10-07T08:42:41.645Z"},additionalProperties={}],global=,data={"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}},additionalProperties={}],trace=,additionalProperties={}] 2022-10-07 08:53:17 destination > Airbyte message consumer: succeeded. 2022-10-07 08:53:17 destination > Started closing all connections 2022-10-07 08:53:17 destination > Closing connector: AbstractBigQueryUploader{table=_airbyte_raw_incremental_transaction_stream, tmpTable=_airbyte_tmp_owf_incremental_transaction_stream, syncMode=WRITE_APPEND, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter} 2022-10-07 08:53:18 destination > Waiting for job finish Job{job=JobId{project=npg-airbyte-destination-dev, job=ce7c7d72-5505-4674-b2ea-a963361727b0, location=EU}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1665132797859, endTime=null, startTime=1665132798140, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=npg-aribyte-bigquery-sa@npg-airbyte-destination-dev.iam.gserviceaccount.com, etag=HxTGDDG7SrNdE8+iUL10Eg==, generatedId=npg-airbyte-destination-dev:EU.ce7c7d72-5505-4674-b2ea-a963361727b0, selfLink=https://www.googleapis.com/bigquery/v2/projects/npg-airbyte-destination-dev/jobs/ce7c7d72-5505-4674-b2ea-a963361727b0?location=EU, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, projectId=npg-airbyte-destination-dev, tableId=_airbyte_tmp_owf_incremental_transaction_stream}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=CREATE_IF_NEEDED, writeDisposition=null, formatOptions=FormatOptions{format=NEWLINE_DELIMITED_JSON}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=null, schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=null, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}}. Status: JobStatus{state=RUNNING, error=null, executionErrors=null} 2022-10-07 08:53:21 destination > Job finish Job{job=JobId{project=npg-airbyte-destination-dev, job=ce7c7d72-5505-4674-b2ea-a963361727b0, location=EU}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1665132797859, endTime=null, startTime=1665132798140, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=npg-aribyte-bigquery-sa@npg-airbyte-destination-dev.iam.gserviceaccount.com, etag=HxTGDDG7SrNdE8+iUL10Eg==, generatedId=npg-airbyte-destination-dev:EU.ce7c7d72-5505-4674-b2ea-a963361727b0, selfLink=https://www.googleapis.com/bigquery/v2/projects/npg-airbyte-destination-dev/jobs/ce7c7d72-5505-4674-b2ea-a963361727b0?location=EU, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, projectId=npg-airbyte-destination-dev, tableId=_airbyte_tmp_owf_incremental_transaction_stream}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=CREATE_IF_NEEDED, writeDisposition=null, formatOptions=FormatOptions{format=NEWLINE_DELIMITED_JSON}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=null, schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=null, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}} with status JobStatus{state=RUNNING, error=null, executionErrors=null} 2022-10-07 08:53:21 destination > Uploading data from the tmp table _airbyte_tmp_owf_incremental_transaction_stream to the source table _airbyte_raw_incremental_transaction_stream. 2022-10-07 08:53:21 destination > Replication finished with no explicit errors. Copying data from tmp tables to permanent 2022-10-07 08:53:24 destination > successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_tmp_owf_incremental_transaction_stream}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=api_responses, tableId=_airbyte_raw_incremental_transaction_stream}} 2022-10-07 08:53:24 destination > Data is successfully loaded to the source table _airbyte_raw_incremental_transaction_stream! 2022-10-07 08:53:24 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):445 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@74af8c00[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@36208c6e[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@c08d75f[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@674276cc[name=incremental_transaction_stream,namespace=,additionalProperties={}],streamState={"startTime":"2022-10-07T08:42:41.645Z"},additionalProperties={}],global=,data={"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}},additionalProperties={}],trace=,additionalProperties={}] 2022-10-07 08:53:24 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):196 - The message tracker encountered an issue that prevents committed record counts from being reliably computed. 2022-10-07 08:53:24 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):197 - This only impacts metadata and does not indicate a problem with actual sync data. 2022-10-07 08:53:24 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):198 - State hash -2068298252 was already committed, likely indicating a state hash collision io.airbyte.workers.internal.StateDeltaTracker$StateDeltaTrackerException: State hash -2068298252 was already committed, likely indicating a state hash collision at io.airbyte.workers.internal.StateDeltaTracker.commitStateHash(StateDeltaTracker.java:114) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.handleDestinationEmittedState(AirbyteMessageTracker.java:193) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.java:119) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$7(DefaultReplicationWorker.java:446) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:53:24 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):207 - The state message tracker was unable to match the destination state message to a corresponding source state message. 2022-10-07 08:53:24 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):208 - This only impacts metrics and does not indicate a problem with actual sync data. 2022-10-07 08:53:24 WARN i.a.w.i.AirbyteMessageTracker(handleDestinationEmittedState):209 - Destination state message cannot be matched to corresponding Source state message. io.airbyte.workers.internal.StateMetricsTracker$StateMetricsTrackerNoStateMatchException: Destination state message cannot be matched to corresponding Source state message. at io.airbyte.workers.internal.StateMetricsTracker.findStartingTimeStampAndRemoveOlderEntries(StateMetricsTracker.java:147) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.StateMetricsTracker.updateStates(StateMetricsTracker.java:80) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.handleDestinationEmittedState(AirbyteMessageTracker.java:204) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.internal.AirbyteMessageTracker.acceptFromDestination(AirbyteMessageTracker.java:119) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$7(DefaultReplicationWorker.java:446) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:53:24 destination > Final state message is accepted. 2022-10-07 08:53:24 destination > Removing tmp tables... 2022-10-07 08:53:24 destination > Finishing destination process...completed 2022-10-07 08:53:24 destination > Closed connector: AbstractBigQueryUploader{table=_airbyte_raw_incremental_transaction_stream, tmpTable=_airbyte_tmp_owf_incremental_transaction_stream, syncMode=WRITE_APPEND, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter} 2022-10-07 08:53:24 destination > Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-10-07 08:53:24 INFO i.a.w.g.DefaultReplicationWorker(run):184 - Source and destination threads complete. 2022-10-07 08:53:24 INFO i.a.w.g.DefaultReplicationWorker(run):279 - Source output at least one state message 2022-10-07 08:53:24 INFO i.a.w.g.DefaultReplicationWorker(run):285 - State capture: Updated state to: Optional[io.airbyte.config.State@6bd0f600[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"incremental_transaction_stream"},"stream_state":{"startTime":"2022-10-07T08:42:41.645Z"}},"data":{"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}}}]]] 2022-10-07 08:53:24 INFO i.a.w.g.DefaultReplicationWorker(run):300 - sync summary: { "status" : "completed", "recordsSynced" : 843, "bytesSynced" : 418864, "startTime" : 1665132789733, "endTime" : 1665132804616, "totalStats" : { "recordsEmitted" : 843, "bytesEmitted" : 418864, "sourceStateMessagesEmitted" : 1, "destinationStateMessagesEmitted" : 2, "recordsCommitted" : 843, "meanSecondsBeforeSourceStateMessageEmitted" : 5, "maxSecondsBeforeSourceStateMessageEmitted" : 5 }, "streamStats" : [ { "streamName" : "incremental_transaction_stream", "stats" : { "recordsEmitted" : 843, "bytesEmitted" : 418864, "recordsCommitted" : 843 } } ] } 2022-10-07 08:53:24 INFO i.a.w.g.DefaultReplicationWorker(run):301 - failures: [ ] 2022-10-07 08:53:24 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:53:24 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):159 - sync summary: io.airbyte.config.StandardSyncOutput@3bd88efc[standardSyncSummary=io.airbyte.config.StandardSyncSummary@7eda17e3[status=completed,recordsSynced=843,bytesSynced=418864,startTime=1665132789733,endTime=1665132804616,totalStats=io.airbyte.config.SyncStats@48e08131[recordsEmitted=843,bytesEmitted=418864,sourceStateMessagesEmitted=1,destinationStateMessagesEmitted=2,recordsCommitted=843,meanSecondsBeforeSourceStateMessageEmitted=5,maxSecondsBeforeSourceStateMessageEmitted=5,maxSecondsBetweenStateMessageEmittedandCommitted=,meanSecondsBetweenStateMessageEmittedandCommitted=,additionalProperties={}],streamStats=[io.airbyte.config.StreamSyncStats@4f6e4c28[streamName=incremental_transaction_stream,stats=io.airbyte.config.SyncStats@30be8457[recordsEmitted=843,bytesEmitted=418864,sourceStateMessagesEmitted=,destinationStateMessagesEmitted=,recordsCommitted=843,meanSecondsBeforeSourceStateMessageEmitted=,maxSecondsBeforeSourceStateMessageEmitted=,maxSecondsBetweenStateMessageEmittedandCommitted=,meanSecondsBetweenStateMessageEmittedandCommitted=,additionalProperties={}]]]],normalizationSummary=,state=io.airbyte.config.State@6bd0f600[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"incremental_transaction_stream"},"stream_state":{"startTime":"2022-10-07T08:42:41.645Z"}},"data":{"incremental_transaction_stream":{"startTime":"2022-10-07T08:42:41.645Z"}}}]],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@1ef5e8ac[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@52d00b4a[stream=io.airbyte.protocol.models.AirbyteStream@6fe21234[name=incremental_transaction_stream,jsonSchema={"type":"array","items":{"type":"object","required":["transactionId","paymentProvider","customerId"],"properties":{"amount":{"type":"number"},"currency":{"type":"string"},"errorCode":{"type":"string"},"customerId":{"type":"string"},"changedDate":{"type":"string"},"description":{"type":"string"},"errorMessage":{"type":"string"},"paymentAction":{"type":"string"},"paymentMethod":{"type":"string"},"paymentStatus":{"type":"string"},"transactionId":{"type":"string"},"fulfillmentDate":{"type":"string"},"paymentProvider":{"type":"string"},"transactionDate":{"type":"string"},"pspTransactionId":{"type":"string"},"plenigoTransactionId":{"type":"string"}}},"$schema":"http://json-schema.org/draft-04/schema#"},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[startTime],sourceDefinedPrimaryKey=[[transactionId]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[startTime],destinationSyncMode=append_dedup,primaryKey=[[transactionId]],additionalProperties={}]],additionalProperties={}],failures=[]] 2022-10-07 08:53:24 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):280 - Stopping temporal heartbeating... 2022-10-07 08:53:24 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:24 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END REPLICATION ----- 2022-10-07 08:53:24 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:24 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):55 - Using default value for environment variable NEED_STATE_VALIDATION: 'true' 2022-10-07 08:53:24 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/76/2/logs.log 2022-10-07 08:53:24 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10 2022-10-07 08:53:25 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:25 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START DEFAULT NORMALIZATION ----- 2022-10-07 08:53:25 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-07 08:53:25 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):138 - Running with normalization version: airbyte/normalization:0.2.22 2022-10-07 08:53:25 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/normalization:0.2.22 exists... 2022-10-07 08:53:25 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/normalization:0.2.22 was found locally. 2022-10-07 08:53:25 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = normalization-normalize-76-2-tlzxa with resources io.airbyte.config.ResourceRequirements@1c7f47df[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-07 08:53:25 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/76/2/normalize --log-driver none --name normalization-normalize-76-2-tlzxa --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_VERSION=0.40.10 airbyte/normalization:0.2.22 run --integration-type bigquery --config destination_config.json --catalog destination_catalog.json 2022-10-07 08:53:25 normalization > Running: transform-config --config destination_config.json --integration-type bigquery --out /data/76/2/normalize 2022-10-07 08:53:26 normalization > Namespace(config='destination_config.json', integration_type=, out='/data/76/2/normalize') 2022-10-07 08:53:26 normalization > transform_bigquery 2022-10-07 08:53:26 normalization > Running: transform-catalog --integration-type bigquery --profile-config-dir /data/76/2/normalize --catalog destination_catalog.json --out /data/76/2/normalize/models/generated/ --json-column _airbyte_data 2022-10-07 08:53:26 normalization > Processing destination_catalog.json... 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 2022-10-07 08:53:27 normalization > Showing destination_catalog.json to diagnose/debug errors (1): 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 2022-10-07 08:53:27 normalization > { 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "streams" 2022-10-07 08:53:27 normalization > { 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "stream" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "name" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "json_schema" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "items" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "required" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentProvider" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "customerId" 2022-10-07 08:53:27 normalization > ], 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "properties" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "amount" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "currency" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "errorCode" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "customerId" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "changedDate" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "description" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "errorMessage" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentAction" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentMethod" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentStatus" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "fulfillmentDate" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "paymentProvider" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionDate" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "pspTransactionId" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "plenigoTransactionId" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "type" 2022-10-07 08:53:27 normalization > } 2022-10-07 08:53:27 normalization > } 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "$schema" 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "supported_sync_modes" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "full_refresh" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "incremental" 2022-10-07 08:53:27 normalization > ], 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "source_defined_cursor" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "default_cursor_field" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "startTime" 2022-10-07 08:53:27 normalization > ], 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "source_defined_primary_key" 2022-10-07 08:53:27 normalization > [ 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:53:27 normalization > ] 2022-10-07 08:53:27 normalization > ] 2022-10-07 08:53:27 normalization > }, 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "sync_mode" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "cursor_field" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "startTime" 2022-10-07 08:53:27 normalization > ], 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "destination_sync_mode" 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "primary_key" 2022-10-07 08:53:27 normalization > [ 2022-10-07 08:53:27 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - "transactionId" 2022-10-07 08:53:27 normalization > ] 2022-10-07 08:53:27 normalization > ] 2022-10-07 08:53:27 normalization > } 2022-10-07 08:53:27 normalization > ] 2022-10-07 08:53:27 normalization > } 2022-10-07 08:53:27 INFO i.a.w.g.DefaultNormalizationWorker(run):82 - Normalization executed in 2 seconds. 2022-10-07 08:53:27 normalization > Traceback (most recent call last): 2022-10-07 08:53:27 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):164 - Completing future exceptionally... io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:92) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:27) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:161) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] 2022-10-07 08:53:27 normalization > File "/usr/local/bin/transform-catalog", line 8, in 2022-10-07 08:53:27 normalization > sys.exit(main()) 2022-10-07 08:53:27 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 104, in main 2022-10-07 08:53:27 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-10-07 08:53:27 normalization > TransformCatalog().run(args) 2022-10-07 08:53:27 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):280 - Stopping temporal heartbeating... 2022-10-07 08:53:27 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 36, in run 2022-10-07 08:53:27 normalization > self.process_catalog() 2022-10-07 08:53:27 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 64, in process_catalog 2022-10-07 08:53:27 normalization > processor.process(catalog_file=catalog_file, json_column_name=json_col, default_schema=schema) 2022-10-07 08:53:27 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 55, in process 2022-10-07 08:53:27 normalization > stream_processors = self.build_stream_processor( 2022-10-07 08:53:27 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 138, in build_stream_processor 2022-10-07 08:53:27 normalization > properties = get_field(get_field(stream_config, "json_schema", message), "properties", message) 2022-10-07 08:53:27 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 230, in get_field 2022-10-07 08:53:27 normalization > raise KeyError(message) 2022-10-07 08:53:27 normalization > KeyError: "'json_schema'.'properties' are not defined for stream incremental_transaction_stream" 2022-10-07 08:53:27 WARN i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=e00e2eca-778d-3b10-a312-e732c14b9ff1, activityType=Normalize, attempt=1 java.lang.RuntimeException: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:278) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.normalize(NormalizationActivityImpl.java:93) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at jdk.internal.reflect.GeneratedMethodAccessor561.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] at java.lang.Thread.run(Thread.java:833) ~[?:?] Caused by: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.temporal.serviceclient.CheckedExceptionWrapper.wrap(CheckedExceptionWrapper.java:56) ~[temporal-serviceclient-1.8.1.jar:?] at io.temporal.internal.sync.WorkflowInternal.wrap(WorkflowInternal.java:448) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.activity.Activity.wrap(Activity.java:51) ~[temporal-sdk-1.8.1.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:138) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$3(NormalizationActivityImpl.java:121) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:273) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] ... 13 more Caused by: java.util.concurrent.ExecutionException: io.airbyte.workers.exception.WorkerException: Normalization Failed. at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:132) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$3(NormalizationActivityImpl.java:121) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.commons.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:273) ~[io.airbyte-airbyte-commons-temporal-0.40.10.jar:?] ... 13 more Caused by: io.airbyte.workers.exception.WorkerException: Normalization Failed. at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:92) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.general.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:27) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:161) ~[io.airbyte-airbyte-workers-0.40.10.jar:?] ... 1 more