2023-05-05 15:38:26 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to get state 2023-05-05 15:38:26 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to set attempt sync config 2023-05-05 15:38:26 INFO i.a.c.t.s.DefaultTaskQueueMapper(getTaskQueue):31 - Called DefaultTaskQueueMapper getTaskQueue for geography auto 2023-05-05 15:38:26 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/276/0/logs.log 2023-05-05 15:38:26 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.44.2 2023-05-05 15:38:26 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to save workflow id for cancellation 2023-05-05 15:38:26 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to get the source for heartbeat 2023-05-05 15:38:26 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to get the source definition 2023-05-05 15:38:26 INFO i.a.w.g.ReplicationWorkerFactory(create):95 - Setting up source... 2023-05-05 15:38:26 INFO i.a.w.g.ReplicationWorkerFactory(create):99 - Setting up destination... 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable METRIC_CLIENT: '' 2023-05-05 15:38:26 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 2023-05-05 15:38:26 INFO i.a.w.g.ReplicationWorkerFactory(create):108 - Setting up replication worker... 2023-05-05 15:38:26 INFO i.a.w.g.DefaultReplicationWorker(run):161 - start sync worker. job id: 276 attempt id: 0 2023-05-05 15:38:26 INFO i.a.w.g.DefaultReplicationWorker(run):163 - Committing states from replication activity 2023-05-05 15:38:26 INFO i.a.w.g.DefaultReplicationWorker(run):166 - Committing stats from replication activity 2023-05-05 15:38:26 INFO i.a.c.i.LineGobbler(voidCall):149 - 2023-05-05 15:38:26 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START REPLICATION ----- 2023-05-05 15:38:26 INFO i.a.w.g.DefaultReplicationWorker(run):182 - configured sync modes: {null.Case=incremental - append_dedup} 2023-05-05 15:38:26 INFO i.a.c.i.LineGobbler(voidCall):149 - 2023-05-05 15:38:26 INFO i.a.w.i.DefaultAirbyteDestination(start):92 - Running destination... 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable LAUNCHDARKLY_KEY: '' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable FEATURE_FLAG_CLIENT: '' 2023-05-05 15:38:26 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/destination-bigquery:1.2.20 exists... 2023-05-05 15:38:26 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/destination-bigquery:1.2.20 was found locally. 2023-05-05 15:38:26 INFO i.a.w.p.DockerProcessFactory(create):133 - Creating docker container = destination-bigquery-write-276-0-qeuql with resources io.airbyte.config.ResourceRequirements@35462351[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null 2023-05-05 15:38:26 INFO i.a.w.p.DockerProcessFactory(create):184 - Preparing command: docker run --rm --init -i -w /data/276/0 --log-driver none --name destination-bigquery-write-276-0-qeuql --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.2.20 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT= -e AIRBYTE_VERSION=0.44.2 -e WORKER_JOB_ID=276 airbyte/destination-bigquery:1.2.20 write --config destination_config.json --catalog destination_catalog.json 2023-05-05 15:38:26 INFO i.a.w.i.VersionedAirbyteMessageBufferedWriterFactory(createWriter):41 - Writing messages to protocol version 0.2.0 2023-05-05 15:38:26 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):181 - Reading messages from protocol version 0.2.0 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable LAUNCHDARKLY_KEY: '' 2023-05-05 15:38:26 INFO i.a.c.EnvConfigs(getEnvOrDefault):1222 - Using default value for environment variable FEATURE_FLAG_CLIENT: '' 2023-05-05 15:38:26 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/source-salesforce:2.0.12 exists... 2023-05-05 15:38:26 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/source-salesforce:2.0.12 was found locally. 2023-05-05 15:38:26 INFO i.a.w.p.DockerProcessFactory(create):133 - Creating docker container = source-salesforce-read-276-0-uzjwp with resources io.airbyte.config.ResourceRequirements@3fc4dde6[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null 2023-05-05 15:38:26 INFO i.a.w.p.DockerProcessFactory(create):184 - Preparing command: docker run --rm --init -i -w /data/276/0 --log-driver none --name source-salesforce-read-276-0-uzjwp --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_CONNECTOR_IMAGE=airbyte/source-salesforce:2.0.12 -e AUTO_DETECT_SCHEMA=true -e LAUNCHDARKLY_KEY= -e SOCAT_KUBE_CPU_REQUEST=0.1 -e SOCAT_KUBE_CPU_LIMIT=2.0 -e USE_STREAM_CAPABLE_STATE=true -e FIELD_SELECTION_WORKSPACES= -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_JOB_ATTEMPT=0 -e OTEL_COLLECTOR_ENDPOINT=http://host.docker.internal:4317 -e FEATURE_FLAG_CLIENT= -e AIRBYTE_VERSION=0.44.2 -e WORKER_JOB_ID=276 airbyte/source-salesforce:2.0.12 read --config source_config.json --catalog source_catalog.json 2023-05-05 15:38:26 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):181 - Reading messages from protocol version 0.2.0 2023-05-05 15:38:26 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromDstRunnable$4):310 - Destination output thread started. 2023-05-05 15:38:26 INFO i.a.w.i.HeartbeatTimeoutChaperone(runWithHeartbeatThread):94 - Starting source heartbeat check. Will check every 1 minutes. 2023-05-05 15:38:26 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):376 - Replication thread started. 2023-05-05 15:38:28 source > Starting syncing SourceSalesforce 2023-05-05 15:38:28 source > Stream AIPredictionEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AIUpdateRecordEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AccountChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AccountContactRoleChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ActivityHistory is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AggregateResult is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AssetChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AssetTokenEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AsyncOperationEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AsyncOperationStatus is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AttachedContentDocument is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AttachedContentNote is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream AuthorizationFormConsentChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream BatchApexErrorEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream BriefcaseAssignmentChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream BriefcaseDefinitionChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CampaignChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CampaignMemberChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CampaignMemberStatusChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CaseChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CaseContacts__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CaseLotPool__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CaseRoutingHelper__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Case_Quality__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CombinedAttachment is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream CommSubscriptionConsentChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContactChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContactPointAddressChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContactPointConsentChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContactPointEmailChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContactPointPhoneChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContactPointTypeConsentChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContentBody is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContentDocumentChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContentDocumentLinkChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContentVersionChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContractChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ContractLineItemChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream DataObjectDataChgEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream EmailMessageChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream EmailStatus is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream EmailTemplateChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream EntitlementChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream EventChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream EventRelationChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream EventRelayConfigChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream FeedLike is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream FeedSignal is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream FeedTrackedChange is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream FlowExecutionErrorEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream FlowOrchestrationEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream FlowPersonalConfiguration__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream FlowTableViewDefinition__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream FolderedContentDocument is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream IndividualChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream LeadChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ListEmailChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream LocationChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream LogoutEventStream is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream LookedUpFromActivity is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Lot_Measurement__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Lot_Partner__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream MacroChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream MacroInstructionChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Name is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream NoteAndAttachment is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream OmnichannelSyncWebhook__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream OpenActivity is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream OpportunityChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream OpportunityContactRoleChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream OrderChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream OrderItemChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream OrgLifecycleNotification is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream OwnedContentDocument is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream PL_Count_ID__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Parking_Lot_Rule__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Parking_Lot__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream PartyConsentChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream PlatformStatusAlertEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Pricebook2ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream PricebookEntryChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ProcessExceptionEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ProcessInstanceHistory is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Product2ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream QuickTextChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream QuoteChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream QuoteLineItemChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream QuoteTemplateRichTextData is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream RecommendationChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream Sales_Handover__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ServiceContractChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ServiceResourceChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ServiceResourceSkillChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream SkillChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream SkillRequirementChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream SocialPostChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream TaskChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream TaskRelationChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream UserChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream dlrs__DeclarativeLookupRollupSummaries__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream dlrs__LookupChildAReallyReallyReallyBigBigName__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream dlrs__LookupChild__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream dlrs__LookupParent__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream dlrs__LookupRollupCalculateJob__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream dlrs__LookupRollupSummaryLog__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream dlrs__LookupRollupSummaryScheduleItems__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream dlrs__LookupRollupSummary__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream ers_datatableConfig__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Custom_Relationship__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Customer_Buying_Center__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Default_Org_Chart_Configurations__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__MilestoneTemplate__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Milestone_Change_Event__e is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Milestone_Header__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Milestone__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Org_Chart_Configuration__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Org_Chart_Global_Settings__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Plan_Summary__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Product_Offering__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Relationship__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Squivr_Trigger_Settings__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Tracked_Milestone__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Whitespace_Summary__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Stream squivr__Whitespace_Template__ChangeEvent is not queryable and will be ignored. 2023-05-05 15:38:28 source > Marking stream Case as STARTED 2023-05-05 15:38:28 source > {"type": "TRACE", "trace": {"type": "STREAM_STATUS", "emitted_at": 1683301108641.716, "stream_status": {"stream_descriptor": {"name": "Case", "namespace": null}, "status": "STARTED"}}} 2023-05-05 15:38:28 source > Syncing stream: Case 2023-05-05 15:38:28 destination > INFO i.a.i.b.IntegrationCliParser(parseOptions):126 integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2023-05-05 15:38:28 destination > INFO i.a.i.b.IntegrationRunner(runInternal):108 Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2023-05-05 15:38:28 destination > INFO i.a.i.b.IntegrationRunner(runInternal):109 Command: WRITE 2023-05-05 15:38:28 destination > INFO i.a.i.b.IntegrationRunner(runInternal):110 Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2023-05-05 15:38:28 destination > WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2023-05-05 15:38:28 destination > WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2023-05-05 15:38:28 destination > WARN c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword always_show - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2023-05-05 15:38:28 destination > INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):419 Selected loading method is set to: GCS 2023-05-05 15:38:29 destination > INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 S3 format config: {"format_type":"AVRO","flattening":"No flattening"} 2023-05-05 15:38:29 destination > INFO i.a.i.d.b.BigQueryUtils(isKeepFilesInGcs):435 All tmp files will be removed from GCS when replication is finished 2023-05-05 15:38:29 destination > INFO i.a.i.d.b.BigQueryDestination(getGcsRecordConsumer):315 Creating BigQuery staging message consumer with staging ID 91f5083e-8434-4014-9083-4f401aed3852 at 2023-05-05T15:38:29.223Z 2023-05-05 15:38:29 destination > INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):93 BigQuery write config: BigQueryWriteConfig[streamName=Case, namespace=test_patricia, datasetId=test_patricia, datasetLocation=europe-west3, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, tableId=_airbyte_tmp_qxq_Case}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, tableId=_airbyte_raw_Case}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append_dedup, stagedFiles=[]] 2023-05-05 15:38:29 destination > INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):146 class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started. 2023-05-05 15:38:29 destination > INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):116 Preparing airbyte_raw tables in destination started for 1 streams 2023-05-05 15:38:29 destination > INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):118 Preparing staging are in destination for schema: Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, stream: Case, target table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, tableId=_airbyte_raw_Case}}, stage: Case 2023-05-05 15:38:29 destination > INFO i.a.i.d.b.BigQueryGcsOperations(createSchemaIfNotExists):86 Creating dataset test_patricia 2023-05-05 15:38:30 destination > INFO i.a.i.d.b.BigQueryGcsOperations(createTableIfNotExists):102 Creating target table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, tableId=_airbyte_raw_Case}} 2023-05-05 15:38:30 destination > INFO i.a.i.d.b.BigQueryUtils(createPartitionedTableIfNotExists):224 Partitioned table ALREADY EXISTS: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, tableId=_airbyte_raw_Case}} 2023-05-05 15:38:30 destination > INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):109 Creating staging path for stream Case (dataset test_patricia): data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/ 2023-05-05 15:38:30 destination > INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):131 Preparing airbyte_raw tables in destination completed. 2023-05-05 15:38:32 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4mJAAT to complete. Current state: InProgress 2023-05-05 15:38:34 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4gqAAD to complete. Current state: UploadComplete 2023-05-05 15:38:35 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7500900000VN4gqAAD to complete. Current state: UploadComplete 2023-05-05 15:38:36 source > Sleeping 0.5073890560989307 seconds while waiting for Job: Case/7500900000VN4gqAAD to complete. Current state: UploadComplete 2023-05-05 15:38:37 source > Sleeping 0.5200855369231877 seconds while waiting for Job: Case/7500900000VN4gqAAD to complete. Current state: UploadComplete 2023-05-05 15:38:37 source > Sleeping 0.5545981500331443 seconds while waiting for Job: Case/7500900000VN4gqAAD to complete. Current state: UploadComplete 2023-05-05 15:38:38 source > Sleeping 0.6484131591025766 seconds while waiting for Job: Case/7500900000VN4gqAAD to complete. Current state: UploadComplete 2023-05-05 15:38:40 source > Sleeping 0.9034287934927351 seconds while waiting for Job: Case/7500900000VN4gqAAD to complete. Current state: UploadComplete 2023-05-05 15:38:41 source > Sleeping 1.5966331584284585 seconds while waiting for Job: Case/7500900000VN4gqAAD to complete. Current state: UploadComplete 2023-05-05 15:38:43 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4mOAAT to complete. Current state: UploadComplete 2023-05-05 15:38:45 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4mTAAT to complete. Current state: InProgress 2023-05-05 15:38:47 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4mYAAT to complete. Current state: UploadComplete 2023-05-05 15:38:51 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN47qAAD to complete. Current state: InProgress 2023-05-05 15:38:53 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4mnAAD to complete. Current state: UploadComplete 2023-05-05 15:38:54 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7500900000VN4mnAAD to complete. Current state: UploadComplete 2023-05-05 15:38:55 source > Sleeping 0.5073890560989307 seconds while waiting for Job: Case/7500900000VN4mnAAD to complete. Current state: UploadComplete 2023-05-05 15:38:55 source > Sleeping 0.5200855369231877 seconds while waiting for Job: Case/7500900000VN4mnAAD to complete. Current state: UploadComplete 2023-05-05 15:38:56 source > Sleeping 0.5545981500331443 seconds while waiting for Job: Case/7500900000VN4mnAAD to complete. Current state: UploadComplete 2023-05-05 15:38:59 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN3v7AAD to complete. Current state: UploadComplete 2023-05-05 15:39:00 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7500900000VN3v7AAD to complete. Current state: InProgress 2023-05-05 15:39:02 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4GZAA1 to complete. Current state: UploadComplete 2023-05-05 15:39:04 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4aTAAT to complete. Current state: UploadComplete 2023-05-05 15:39:04 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7500900000VN4aTAAT to complete. Current state: InProgress 2023-05-05 15:39:09 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4nCAAT to complete. Current state: UploadComplete 2023-05-05 15:39:09 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7500900000VN4nCAAT to complete. Current state: InProgress 2023-05-05 15:39:11 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4MgAAL to complete. Current state: UploadComplete 2023-05-05 15:39:12 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7500900000VN4MgAAL to complete. Current state: UploadComplete 2023-05-05 15:39:13 source > Sleeping 0.5073890560989307 seconds while waiting for Job: Case/7500900000VN4MgAAL to complete. Current state: UploadComplete 2023-05-05 15:39:13 source > Sleeping 0.5200855369231877 seconds while waiting for Job: Case/7500900000VN4MgAAL to complete. Current state: UploadComplete 2023-05-05 15:39:18 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4NZAA1 to complete. Current state: InProgress 2023-05-05 15:39:19 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:20 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:21 source > Sleeping 0.5073890560989307 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:22 source > Sleeping 0.5200855369231877 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:22 source > Sleeping 0.5545981500331443 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:23 source > Sleeping 0.6484131591025766 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:24 source > Sleeping 0.9034287934927351 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:26 source > Sleeping 1.5966331584284585 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:30 source > Sleeping 3.4809579870417284 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:33 source > Sleeping 3.4809579870417284 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:37 source > Sleeping 3.4809579870417284 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:41 source > Sleeping 3.4809579870417284 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:45 source > Sleeping 3.4809579870417284 seconds while waiting for Job: Case/7500900000VN4nMAAT to complete. Current state: InProgress 2023-05-05 15:39:50 source > Marking stream Case as RUNNING 2023-05-05 15:39:50 source > {"type": "TRACE", "trace": {"type": "STREAM_STATUS", "emitted_at": 1683301190702.558, "stream_status": {"stream_descriptor": {"name": "Case", "namespace": null}, "status": "RUNNING"}}} 2023-05-05 15:39:50 destination > INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$addRecord$0):76 Starting a new buffer for stream Case (current state: 0 bytes in 0 buffers) 2023-05-05 15:39:50 destination > INFO i.a.i.d.g.u.GcsUtils(getDefaultAvroSchema):25 Default schema. 2023-05-05 15:40:02 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):434 - Records read: 5000 (18 MB) 2023-05-05 15:40:13 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):434 - Records read: 10000 (36 MB) 2023-05-05 15:40:25 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):434 - Records read: 15000 (54 MB) 2023-05-05 15:40:27 source > Sleeping 0.501 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: UploadComplete 2023-05-05 15:40:28 source > Sleeping 0.502718281828459 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:28 source > Sleeping 0.5073890560989307 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:29 source > Sleeping 0.5200855369231877 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:30 source > Sleeping 0.5545981500331443 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:31 source > Sleeping 0.6484131591025766 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:32 source > Sleeping 0.9034287934927351 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:34 source > Sleeping 1.5966331584284585 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:37 source > Sleeping 3.4809579870417284 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:41 source > Sleeping 3.4809579870417284 seconds while waiting for Job: Case/7500900000VN4ngAAD to complete. Current state: InProgress 2023-05-05 15:40:58 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):434 - Records read: 20000 (72 MB) 2023-05-05 15:41:10 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):434 - Records read: 25000 (89 MB) 2023-05-05 15:41:21 source > Read 29627 records from Case stream 2023-05-05 15:41:21 source > Marking stream Case as STOPPED 2023-05-05 15:41:21 source > {"type": "TRACE", "trace": {"type": "STREAM_STATUS", "emitted_at": 1683301281061.0, "stream_status": {"stream_descriptor": {"name": "Case", "namespace": null}, "status": "COMPLETE"}}} 2023-05-05 15:41:21 source > Finished syncing Case 2023-05-05 15:41:21 source > SourceSalesforce runtimes: Syncing stream Case 0:02:52.420008 2023-05-05 15:41:21 source > Finished syncing SourceSalesforce 2023-05-05 15:41:22 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):438 - Source has no more messages, closing connection. 2023-05-05 15:41:22 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):447 - Total records read: 29709 (106 MB) 2023-05-05 15:41:22 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromSrcAndWriteToDstRunnable$8):456 - Schema validation was performed to a max of 10 records with errors per stream. 2023-05-05 15:41:22 INFO i.a.w.i.HeartbeatTimeoutChaperone(runWithHeartbeatThread):111 - thread status... heartbeat thread: false , replication thread: true 2023-05-05 15:41:22 INFO i.a.w.g.DefaultReplicationWorker(replicate):266 - Waiting for source and destination threads to complete. 2023-05-05 15:41:22 INFO i.a.w.g.DefaultReplicationWorker(replicate):271 - One of source or destination thread complete. Waiting on the other. 2023-05-05 15:41:22 destination > INFO i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 Airbyte message consumer: succeeded. 2023-05-05 15:41:22 destination > INFO i.a.i.d.b.BufferedStreamConsumer(close):257 executing on success close procedure. 2023-05-05 15:41:22 destination > INFO i.a.i.d.r.SerializedBufferingStrategy(flushAll):130 Flushing all 1 current buffers (107 MB in total) 2023-05-05 15:41:22 destination > INFO i.a.i.d.r.SerializedBufferingStrategy(flushAll):132 Flushing buffer of stream Case (107 MB) 2023-05-05 15:41:22 destination > INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$flushBufferFunction$4):147 Flushing buffer for stream Case (107 MB) to staging 2023-05-05 15:41:22 destination > INFO i.a.i.d.r.BaseSerializedBuffer(flush):138 Finished writing data to a663aa33-d82e-4515-8194-89d2a1e695d6667612567936005418.avro (107 MB) 2023-05-05 15:41:22 destination > INFO i.a.i.d.b.BigQueryGcsOperations(uploadRecordsToStage):116 Uploading records to staging for stream Case (dataset test_patricia): data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/ 2023-05-05 15:41:22 destination > INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 Initiated multipart upload to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with full ID ABPnzm4E97moMpP1rziHpe7PYFDgqxvONEyOS3E5vW4RxUBSxVmffc2xio4NVlv3EDq0JFpj 2023-05-05 15:41:22 destination > INFO a.m.s.MultiPartOutputStream(close):158 Called close() on [MultipartOutputStream for parts 1 - 10000] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 3 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 5 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 10 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 2 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 6 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 8 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 7 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 4 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 9 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 1 containing 10.01 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(uploadStreamPart):558 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Finished uploading [Part number 11 containing 7.81 MB] 2023-05-05 15:41:23 destination > INFO a.m.s.StreamTransferManager(complete):395 [Manager uploading to gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro with id ABPnzm4E9...3EDq0JFpj]: Completed 2023-05-05 15:41:23 destination > INFO i.a.i.d.s.S3StorageOperations(loadDataIntoBucket):214 Uploaded buffer file to storage: a663aa33-d82e-4515-8194-89d2a1e695d6667612567936005418.avro -> data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro (filename: 0.avro) 2023-05-05 15:41:23 destination > INFO i.a.i.d.s.S3StorageOperations(uploadRecordsToBucket):131 Successfully loaded records to stage data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/ with 0 re-attempt(s) 2023-05-05 15:41:23 destination > INFO i.a.i.d.b.BigQueryWriteConfig(addStagedFile):61 Added staged file: 0.avro 2023-05-05 15:41:23 destination > INFO i.a.i.d.b.BigQueryGcsOperations(copyIntoTableFromStage):133 Uploading records from staging files to target table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, tableId=_airbyte_raw_Case}} (dataset test_patricia): [0.avro] 2023-05-05 15:41:23 destination > INFO i.a.i.d.b.BigQueryGcsOperations(lambda$copyIntoTableFromStage$0):138 Uploading staged file: gs://gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro 2023-05-05 15:41:23 destination > INFO i.a.i.d.b.BigQueryGcsOperations(lambda$copyIntoTableFromStage$0):147 [JobId{project=parkdepot-prod, job=285dbd23-4882-4e2c-a0f6-e543bdc215ea, location=europe-west3}] Created a new job to upload record(s) to target table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, tableId=_airbyte_raw_Case}} (dataset test_patricia): Job{job=JobId{project=parkdepot-prod, job=285dbd23-4882-4e2c-a0f6-e543bdc215ea, location=europe-west3}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1683301283745, endTime=null, startTime=1683301283818, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=warehouse-raw-layer@parkdepot-prod.iam.gserviceaccount.com, etag=ZxRAoVTS0Ue/6t+U7j+cJg==, generatedId=parkdepot-prod:europe-west3.285dbd23-4882-4e2c-a0f6-e543bdc215ea, selfLink=https://www.googleapis.com/bigquery/v2/projects/parkdepot-prod/jobs/285dbd23-4882-4e2c-a0f6-e543bdc215ea?location=europe-west3, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, projectId=parkdepot-prod, tableId=_airbyte_raw_Case}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=null, writeDisposition=WRITE_APPEND, formatOptions=FormatOptions{format=AVRO}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=[gs://gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro], schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=true, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}} 2023-05-05 15:41:23 destination > INFO i.a.i.d.b.BigQueryUtils(waitForJobFinish):443 Waiting for job finish Job{job=JobId{project=parkdepot-prod, job=285dbd23-4882-4e2c-a0f6-e543bdc215ea, location=europe-west3}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1683301283745, endTime=null, startTime=1683301283818, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=warehouse-raw-layer@parkdepot-prod.iam.gserviceaccount.com, etag=ZxRAoVTS0Ue/6t+U7j+cJg==, generatedId=parkdepot-prod:europe-west3.285dbd23-4882-4e2c-a0f6-e543bdc215ea, selfLink=https://www.googleapis.com/bigquery/v2/projects/parkdepot-prod/jobs/285dbd23-4882-4e2c-a0f6-e543bdc215ea?location=europe-west3, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, projectId=parkdepot-prod, tableId=_airbyte_raw_Case}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=null, writeDisposition=WRITE_APPEND, formatOptions=FormatOptions{format=AVRO}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=[gs://gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro], schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=true, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}}. Status: JobStatus{state=RUNNING, error=null, executionErrors=null} 2023-05-05 15:41:28 destination > INFO i.a.i.d.b.BigQueryUtils(waitForJobFinish):445 Job finish Job{job=JobId{project=parkdepot-prod, job=285dbd23-4882-4e2c-a0f6-e543bdc215ea, location=europe-west3}, status=JobStatus{state=RUNNING, error=null, executionErrors=null}, statistics=LoadStatistics{creationTime=1683301283745, endTime=null, startTime=1683301283818, numChildJobs=null, parentJobId=null, scriptStatistics=null, reservationUsage=null, inputBytes=null, inputFiles=null, outputBytes=null, outputRows=null, badRecords=null}, userEmail=warehouse-raw-layer@parkdepot-prod.iam.gserviceaccount.com, etag=ZxRAoVTS0Ue/6t+U7j+cJg==, generatedId=parkdepot-prod:europe-west3.285dbd23-4882-4e2c-a0f6-e543bdc215ea, selfLink=https://www.googleapis.com/bigquery/v2/projects/parkdepot-prod/jobs/285dbd23-4882-4e2c-a0f6-e543bdc215ea?location=europe-west3, configuration=LoadJobConfiguration{type=LOAD, destinationTable=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, projectId=parkdepot-prod, tableId=_airbyte_raw_Case}}, decimalTargetTypes=null, destinationEncryptionConfiguration=null, createDisposition=null, writeDisposition=WRITE_APPEND, formatOptions=FormatOptions{format=AVRO}, nullMarker=null, maxBadRecords=null, schema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, ignoreUnknownValue=null, sourceUris=[gs://gcs_airbyte_dev_parkdepot/data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro], schemaUpdateOptions=null, autodetect=null, timePartitioning=null, clustering=null, useAvroLogicalTypes=true, labels=null, jobTimeoutMs=null, rangePartitioning=null, hivePartitioningOptions=null}} with status JobStatus{state=RUNNING, error=null, executionErrors=null} 2023-05-05 15:41:28 destination > INFO i.a.i.d.b.BigQueryGcsOperations(lambda$copyIntoTableFromStage$0):152 [JobId{project=parkdepot-prod, job=285dbd23-4882-4e2c-a0f6-e543bdc215ea, location=europe-west3}] Target table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=test_patricia, tableId=_airbyte_raw_Case}} (dataset test_patricia) is successfully appended with staging files 2023-05-05 15:41:28 destination > INFO i.a.i.d.r.FileBuffer(deleteFile):109 Deleting tempFile data a663aa33-d82e-4515-8194-89d2a1e695d6667612567936005418.avro 2023-05-05 15:41:28 destination > INFO i.a.i.d.r.SerializedBufferingStrategy(flushAll):134 Flushing completed for Case 2023-05-05 15:41:28 destination > INFO i.a.i.d.r.SerializedBufferingStrategy(close):152 Closing buffer for stream Case 2023-05-05 15:41:28 INFO i.a.w.g.DefaultReplicationWorker(lambda$readFromDstRunnable$4):321 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@69123779[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@67d5c42b[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@63ea5c09[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@26e8ce10[name=Case,namespace=,additionalProperties={}],streamState={"SystemModstamp":"2023-05-05T15:37:04.000Z"},additionalProperties={}],global=,data={"Case":{"SystemModstamp":"2023-05-05T15:37:04.000Z"}},additionalProperties={}],trace=,control=,additionalProperties={}] 2023-05-05 15:41:28 INFO i.a.w.i.s.SyncPersistenceImpl(startBackgroundFlushStateTask):168 - starting state flush thread for connectionId 396eddce-dd24-4d41-b7af-1a21f9b7639e 2023-05-05 15:41:28 destination > INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onCloseFunction$5):190 Cleaning up destination started for 1 streams 2023-05-05 15:41:28 destination > INFO i.a.i.d.b.BigQueryGcsOperations(dropStageIfExists):186 Cleaning up staging path for stream Case (dataset test_patricia): data_sync/test_patricia_Case 2023-05-05 15:41:29 destination > INFO i.a.i.d.g.GcsStorageOperations(cleanUpObjects):43 Deleting object data_sync/test_patricia_Case/2023/05/05/15/91f5083e-8434-4014-9083-4f401aed3852/0.avro 2023-05-05 15:41:29 destination > INFO i.a.i.d.s.S3StorageOperations(cleanUpBucketObject):306 Storage bucket data_sync/test_patricia_Case has been cleaned-up (1 objects were deleted)... 2023-05-05 15:41:29 destination > INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onCloseFunction$5):194 Cleaning up destination completed. 2023-05-05 15:41:29 destination > INFO i.a.i.b.IntegrationRunner(runInternal):186 Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2023-05-05 15:41:29 INFO i.a.w.g.DefaultReplicationWorker(replicate):273 - Source and destination threads complete. 2023-05-05 15:41:29 INFO i.a.w.g.DefaultReplicationWorker(getReplicationOutput):561 - sync summary: { "status" : "completed", "recordsSynced" : 29627, "bytesSynced" : 111499134, "startTime" : 1683301106508, "endTime" : 1683301289297, "totalStats" : { "bytesCommitted" : 111499134, "bytesEmitted" : 111499134, "destinationStateMessagesEmitted" : 1, "destinationWriteEndTime" : 1683301289296, "destinationWriteStartTime" : 1683301106574, "meanSecondsBeforeSourceStateMessageEmitted" : 0, "maxSecondsBeforeSourceStateMessageEmitted" : 32, "maxSecondsBetweenStateMessageEmittedandCommitted" : 176, "meanSecondsBetweenStateMessageEmittedandCommitted" : 176, "recordsEmitted" : 29627, "recordsCommitted" : 29627, "replicationEndTime" : 1683301289297, "replicationStartTime" : 1683301106508, "sourceReadEndTime" : 1683301282101, "sourceReadStartTime" : 1683301106541, "sourceStateMessagesEmitted" : 82 }, "streamStats" : [ { "streamName" : "Case", "streamNamespace" : "test_patricia", "stats" : { "bytesCommitted" : 111499134, "bytesEmitted" : 111499134, "recordsEmitted" : 29627, "recordsCommitted" : 29627 } } ] } 2023-05-05 15:41:29 INFO i.a.w.g.DefaultReplicationWorker(getReplicationOutput):562 - failures: [ ] 2023-05-05 15:41:29 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling... 2023-05-05 15:41:29 INFO i.a.c.i.LineGobbler(voidCall):149 - 2023-05-05 15:41:29 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- END REPLICATION ----- 2023-05-05 15:41:29 INFO i.a.c.i.LineGobbler(voidCall):149 - 2023-05-05 15:41:29 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):157 - sync summary: io.airbyte.config.StandardSyncOutput@5d29f98f[standardSyncSummary=io.airbyte.config.StandardSyncSummary@196a45e2[status=completed,recordsSynced=29627,bytesSynced=111499134,startTime=1683301106508,endTime=1683301289297,totalStats=io.airbyte.config.SyncStats@27a26184[bytesCommitted=111499134,bytesEmitted=111499134,destinationStateMessagesEmitted=1,destinationWriteEndTime=1683301289296,destinationWriteStartTime=1683301106574,estimatedBytes=,estimatedRecords=,meanSecondsBeforeSourceStateMessageEmitted=0,maxSecondsBeforeSourceStateMessageEmitted=32,maxSecondsBetweenStateMessageEmittedandCommitted=176,meanSecondsBetweenStateMessageEmittedandCommitted=176,recordsEmitted=29627,recordsCommitted=29627,replicationEndTime=1683301289297,replicationStartTime=1683301106508,sourceReadEndTime=1683301282101,sourceReadStartTime=1683301106541,sourceStateMessagesEmitted=82,additionalProperties={}],streamStats=[io.airbyte.config.StreamSyncStats@72d0384d[streamName=Case,streamNamespace=test_patricia,stats=io.airbyte.config.SyncStats@60275219[bytesCommitted=111499134,bytesEmitted=111499134,destinationStateMessagesEmitted=,destinationWriteEndTime=,destinationWriteStartTime=,estimatedBytes=,estimatedRecords=,meanSecondsBeforeSourceStateMessageEmitted=,maxSecondsBeforeSourceStateMessageEmitted=,maxSecondsBetweenStateMessageEmittedandCommitted=,meanSecondsBetweenStateMessageEmittedandCommitted=,recordsEmitted=29627,recordsCommitted=29627,replicationEndTime=,replicationStartTime=,sourceReadEndTime=,sourceReadStartTime=,sourceStateMessagesEmitted=,additionalProperties={}],additionalProperties={}]],additionalProperties={}],normalizationSummary=,webhookOperationSummary=,state=,outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@2b294eaa[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@1f33a40a[stream=io.airbyte.protocol.models.AirbyteStream@6cd2f5f1[name=Case,jsonSchema={"$schema":"http://json-schema.org/draft-07/schema#","additionalProperties":true,"type":"object","properties":{"Origin":{"type":["string","null"]},"Parking_Lot__c":{"type":["string","null"]},"LastModifiedDate":{"format":"date-time","type":["string","null"]},"Case_Resolution_Time__c":{"type":["number","null"]},"ServiceContractId":{"type":["string","null"]},"Hour_of_the_day__c":{"type":["number","null"]},"EntitlementId":{"type":["string","null"]},"Hubspot_Contact_Id__c":{"type":["string","null"]},"Prediction_Timestamp__c":{"format":"date-time","type":["string","null"]},"Number_of_Messages_Received__c":{"type":["number","null"]},"BusinessHoursId":{"type":["string","null"]},"SuppliedName":{"type":["string","null"]},"SuppliedPhone":{"type":["string","null"]},"IsClosedOnCreate":{"type":["boolean","null"]},"Hubspot_Created_Date__c":{"format":"date-time","type":["string","null"]},"Closed_Date_Formula__c":{"format":"date-time","type":["string","null"]},"First_Inbound_Email__c":{"format":"date-time","type":["string","null"]},"Status":{"type":["string","null"]},"ContactMobile":{"type":["string","null"]},"IsDeleted":{"type":["boolean","null"]},"Last_Outbound_Email__c":{"format":"date-time","type":["string","null"]},"InboxAT__c":{"type":["string","null"]},"Email_Inboxes__c":{"type":["string","null"]},"IsEscalated":{"type":["boolean","null"]},"Id":{"type":["string","null"]},"AssetId":{"type":["string","null"]},"Number_of_Messages_Sent__c":{"type":["number","null"]},"Description":{"type":["string","null"]},"Parking_Lot_Id__c":{"type":["string","null"]},"SuppliedCompany":{"type":["string","null"]},"SourceId":{"type":["string","null"]},"Parking_Lot_Account__c":{"type":["string","null"]},"SlaExitDate":{"format":"date-time","type":["string","null"]},"IsClosed":{"type":["boolean","null"]},"ContactPhone":{"type":["string","null"]},"Hubspot_Ticket_Id__c":{"type":["string","null"]},"OwnerId":{"type":["string","null"]},"Closed_by__c":{"type":["string","null"]},"First_Closed_Date__c":{"format":"date-time","type":["string","null"]},"RecordTypeId":{"type":["string","null"]},"CaseNumber":{"type":["string","null"]},"Rating__c":{"type":["number","null"]},"Customer_Complaint__c":{"type":["boolean","null"]},"ParentId":{"type":["string","null"]},"SlaStartDate":{"format":"date-time","type":["string","null"]},"Country__c":{"type":["string","null"]},"ConvertedToLead__c":{"type":["boolean","null"]},"Licence_Plate__c":{"type":["string","null"]},"First_Outbound_or_Closed_Date__c":{"format":"date-time","type":["string","null"]},"First_Outbound_Email__c":{"format":"date-time","type":["string","null"]},"Support_Level__c":{"type":["string","null"]},"SystemModstamp":{"format":"date-time","type":["string","null"]},"Escalated_to__c":{"type":["string","null"]},"Type":{"type":["string","null"]},"ClosedDate":{"format":"date-time","type":["string","null"]},"Case_Language__c":{"type":["string","null"]},"State_Language__c":{"type":["string","null"]},"AccountId":{"type":["string","null"]},"SignatureAddressFormula__c":{"type":["string","null"]},"Customer_Email__c":{"type":["string","null"]},"ProductId":{"type":["string","null"]},"Hubspot_Case_Owner__c":{"type":["string","null"]},"File_number__c":{"type":["string","null"]},"CreatedById":{"type":["string","null"]},"Due_Date__c":{"format":"date","type":["string","null"]},"CaseOriginWeb__c":{"type":["number","null"]},"Case_Quality__c":{"type":["string","null"]},"SuppliedEmailDomain__c":{"type":["string","null"]},"Import__c":{"type":["boolean","null"]},"Closed_hour_of_the_day__c":{"type":["number","null"]},"MasterRecordId":{"type":["string","null"]},"Rating_Reason__c":{"type":["string","null"]},"Priority":{"type":["string","null"]},"LastViewedDate":{"format":"date-time","type":["string","null"]},"First_Inbound_Email_Address__c":{"type":["string","null"]},"MilestoneStatus":{"type":["string","null"]},"IsStopped":{"type":["boolean","null"]},"Salutation__c":{"type":["string","null"]},"Hubspot_Closed_Date__c":{"format":"date-time","type":["string","null"]},"CreatedDate":{"format":"date-time","type":["string","null"]},"Hubspot_PD_Kategorie__c":{"type":["string","null"]},"Lead__c":{"type":["string","null"]},"PO1__c":{"type":["number","null"]},"Language_Predicted__c":{"type":["boolean","null"]},"LastReferencedDate":{"format":"date-time","type":["string","null"]},"ContactId":{"type":["string","null"]},"Reason":{"type":["string","null"]},"Enforcement__c":{"type":["boolean","null"]},"SuppliedEmail":{"type":["string","null"]},"Language":{"type":["string","null"]},"Confidence_Language_Prediction__c":{"type":["number","null"]},"SupportTier__c":{"type":["number","null"]},"Last_Inbound_Email__c":{"format":"date-time","type":["string","null"]},"WebFormPhone__c":{"type":["string","null"]},"Case_Evaluation__c":{"type":["boolean","null"]},"ContactFax":{"type":["string","null"]},"InboxDEen__c":{"type":["string","null"]},"CreatedDate_Formula__c":{"format":"date-time","type":["string","null"]},"InboxCHde__c":{"type":["string","null"]},"Comments":{"type":["string","null"]},"First_Response_Time_h__c":{"type":["number","null"]},"StopStartDate":{"format":"date-time","type":["string","null"]},"Subject":{"type":["string","null"]},"personAccTypeFormula__c":{"type":["string","null"]},"InboxCHfr__c":{"type":["string","null"]},"ContactEmail":{"type":["string","null"]},"Case_Web_Source__c":{"type":["string","null"]},"LastModifiedById":{"type":["string","null"]}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[SystemModstamp],sourceDefinedPrimaryKey=[[Id]],namespace=test_patricia,additionalProperties={}],syncMode=incremental,cursorField=[SystemModstamp],destinationSyncMode=append_dedup,primaryKey=[[Id]],additionalProperties={}]],additionalProperties={}],failures=[],commitStateAsap=true,additionalProperties={}] 2023-05-05 15:41:29 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):162 - Sync summary length: 7831 2023-05-05 15:41:29 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):307 - Stopping temporal heartbeating... 2023-05-05 15:41:29 INFO i.a.w.t.s.NormalizationActivityImpl(lambda$normalize$3):125 - Using protocol v0 2023-05-05 15:41:29 INFO i.a.w.t.s.NormalizationActivityImpl(lambda$normalize$3):145 - Using normalization: airbyte/normalization:0.4.0 2023-05-05 15:41:29 INFO i.a.w.t.TemporalAttemptExecution(get):136 - Docker volume job log path: /tmp/workspace/276/0/logs.log 2023-05-05 15:41:29 INFO i.a.w.t.TemporalAttemptExecution(get):141 - Executing worker wrapper. Airbyte version: 0.44.2 2023-05-05 15:41:29 INFO i.a.a.c.AirbyteApiClient(retryWithJitter):179 - Attempt 0 to save workflow id for cancellation 2023-05-05 15:41:29 INFO i.a.c.i.LineGobbler(voidCall):149 - 2023-05-05 15:41:29 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- START DEFAULT NORMALIZATION ----- 2023-05-05 15:41:29 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):129 - Running with normalization version: airbyte/normalization:0.4.0 2023-05-05 15:41:29 INFO i.a.c.i.LineGobbler(voidCall):149 - 2023-05-05 15:41:29 INFO i.a.c.i.LineGobbler(voidCall):149 - Checking if airbyte/normalization:0.4.0 exists... 2023-05-05 15:41:29 INFO i.a.c.i.LineGobbler(voidCall):149 - airbyte/normalization:0.4.0 was found locally. 2023-05-05 15:41:29 INFO i.a.w.p.DockerProcessFactory(create):133 - Creating docker container = normalization-normalize-276-0-lrwkh with resources io.airbyte.config.ResourceRequirements@2bbfca3c[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}] and allowedHosts null 2023-05-05 15:41:29 INFO i.a.w.p.DockerProcessFactory(create):184 - Preparing command: docker run --rm --init -i -w /data/276/0/normalize --log-driver none --name normalization-normalize-276-0-lrwkh --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_VERSION=0.44.2 airbyte/normalization:0.4.0 run --integration-type bigquery --config destination_config.json --catalog destination_catalog.json 2023-05-05 15:41:30 normalization > Running: transform-config --config destination_config.json --integration-type bigquery --out /data/276/0/normalize 2023-05-05 15:41:30 normalization > Namespace(config='destination_config.json', integration_type=, out='/data/276/0/normalize') 2023-05-05 15:41:30 normalization > transform_bigquery 2023-05-05 15:41:30 normalization > Running: transform-catalog --integration-type bigquery --profile-config-dir /data/276/0/normalize --catalog destination_catalog.json --out /data/276/0/normalize/models/generated/ --json-column _airbyte_data 2023-05-05 15:41:31 normalization > Processing destination_catalog.json... 2023-05-05 15:41:31 normalization > Generating airbyte_ctes/test_patricia/Case_ab1.sql from Case 2023-05-05 15:41:31 normalization > Generating airbyte_ctes/test_patricia/Case_ab2.sql from Case 2023-05-05 15:41:31 normalization > Generating airbyte_views/test_patricia/Case_stg.sql from Case 2023-05-05 15:41:31 normalization > Generating airbyte_incremental/scd/test_patricia/Case_scd.sql from Case 2023-05-05 15:41:31 normalization > Generating airbyte_incremental/test_patricia/Case.sql from Case 2023-05-05 15:41:31 normalization > detected no config file for ssh, assuming ssh is off. 2023-05-05 15:41:34 normalization > [--event-buffer-size EVENT_BUFFER_SIZE] 2023-05-05 15:41:34 normalization > --event-buffer-size EVENT_BUFFER_SIZE 2023-05-05 15:41:34 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 2023-05-05 15:41:34 normalization > DBT >=1.0.0 detected; using 10K event buffer size 2023-05-05 15:41:34 INFO i.a.w.n.NormalizationAirbyteStreamFactory(filterOutAndHandleNonAirbyteMessageLines):104 - 2023-05-05 15:41:37 normalization > Running with dbt=1.0.0 2023-05-05 15:41:37 normalization > Partial parse save file not found. Starting full parse. 2023-05-05 15:41:39 normalization > [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources. There are 1 unused configuration paths: - models.airbyte_utils.generated.airbyte_tables 2023-05-05 15:41:39 normalization > Found 5 models, 0 tests, 0 snapshots, 0 analyses, 624 macros, 0 operations, 0 seed files, 1 source, 0 exposures, 0 metrics 2023-05-05 15:41:40 normalization > Concurrency: 8 threads (target='prod') 2023-05-05 15:41:41 normalization > 1 of 3 START view model _airbyte_test_patricia.Case_stg................................................................. [RUN] 2023-05-05 15:41:42 normalization > 1 of 3 OK created view model _airbyte_test_patricia.Case_stg............................................................ [OK in 1.04s] 2023-05-05 15:41:42 normalization > 2 of 3 START incremental model test_patricia.Case_scd................................................................... [RUN] 2023-05-05 15:41:42 normalization > 15:41:42 + `parkdepot-prod`.test_patricia.`Case_scd`._airbyte_ab_id does not exist yet. The table will be created or rebuilt with dbt.full_refresh 2023-05-05 15:43:16 normalization > 2 of 3 OK created incremental model test_patricia.Case_scd.............................................................. [CREATE TABLE (29.6k rows, 107.9 MB processed) in 93.90s] 2023-05-05 15:43:16 normalization > 3 of 3 START incremental model test_patricia.Case....................................................................... [RUN] 2023-05-05 15:43:28 normalization > 3 of 3 OK created incremental model test_patricia.Case.................................................................. [MERGE (28.1k rows, 38.7 MB processed) in 11.93s] 2023-05-05 15:43:28 normalization > Finished running 1 view model, 2 incremental models in 108.32s. 2023-05-05 15:43:28 normalization > Completed successfully 2023-05-05 15:43:28 normalization > Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3 2023-05-05 15:43:28 INFO i.a.w.n.DefaultNormalizationRunner(close):194 - Terminating normalization process... 2023-05-05 15:43:28 INFO i.a.w.n.DefaultNormalizationRunner(close):207 - Normalization process successfully terminated. 2023-05-05 15:43:28 INFO i.a.w.g.DefaultNormalizationWorker(run):97 - Normalization executed in 1 minute 59 seconds for job 276. 2023-05-05 15:43:28 INFO i.a.w.g.DefaultNormalizationWorker(run):109 - Normalization summary: io.airbyte.config.NormalizationSummary@62114303[startTime=1683301289449,endTime=1683301408639,failures=[],additionalProperties={}] 2023-05-05 15:43:28 INFO i.a.c.i.LineGobbler(voidCall):149 - 2023-05-05 15:43:28 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling... 2023-05-05 15:43:28 INFO i.a.c.i.LineGobbler(voidCall):149 - ----- END DEFAULT NORMALIZATION ----- 2023-05-05 15:43:28 INFO i.a.c.i.LineGobbler(voidCall):149 - 2023-05-05 15:43:28 INFO i.a.c.t.TemporalUtils(withBackgroundHeartbeat):307 - Stopping temporal heartbeating...