2022-08-18 19:42:32 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.access_token: is missing but it is required, $.auth_type: must be a constant value oauth2.0 2022-08-18 19:42:32 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.apikey: object found, string expected 2022-08-18 19:42:32 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.method: must be a constant value Standard 2022-08-18 19:42:32 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.credential.hmac_key_access_id: object found, string expected, $.credential.hmac_key_secret: object found, string expected 2022-08-18 19:42:32 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/0/logs.log 2022-08-18 19:42:32 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-18 19:42:32 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-mailchimp:0.2.14 exists... 2022-08-18 19:42:32 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-mailchimp:0.2.14 was found locally. 2022-08-18 19:42:32 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-18 19:42:32 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/0 --log-driver none --name source-mailchimp-check-167-0-mcpvu --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/source-mailchimp:0.2.14 -e WORKER_JOB_ATTEMPT=0 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/source-mailchimp:0.2.14 check --config source_config.json 2022-08-18 19:42:33 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Check succeeded 2022-08-18 19:42:33 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-18 19:42:33 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/0/logs.log 2022-08-18 19:42:33 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-18 19:42:33 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-bigquery:1.1.14 exists... 2022-08-18 19:42:33 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-bigquery:1.1.14 was found locally. 2022-08-18 19:42:33 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-18 19:42:33 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/0 --log-driver none --name destination-bigquery-check-167-0-exohd --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.1.14 -e WORKER_JOB_ATTEMPT=0 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/destination-bigquery:1.1.14 check --config source_config.json 2022-08-18 19:42:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:35 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {check=null, config=source_config.json} 2022-08-18 19:42:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:35 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-18 19:42:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:35 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: CHECK 2022-08-18 19:42:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:35 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'} 2022-08-18 19:42:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:35 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 19:42:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:35 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 19:42:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:35 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 19:42:35 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:35 INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):301 - Selected loading method is set to: GCS 2022-08-18 19:42:36 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:36 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"CSV","flattening":"No flattening"} 2022-08-18 19:42:36 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:36 INFO i.a.i.d.s.S3Destination(testSingleUpload):81 - Started testing if all required credentials assigned to user for single file uploading 2022-08-18 19:42:37 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:37 INFO i.a.i.d.s.S3Destination(testSingleUpload):91 - Finished checking for normal upload mode 2022-08-18 19:42:37 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:37 INFO i.a.i.d.s.S3Destination(testMultipartUpload):95 - Started testing if all required credentials assigned to user for multipart upload 2022-08-18 19:42:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:38 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to na_airbyte_sync/data_sync/do/test_1660851757789 with full ID ABPnzm59V-jOiqAqZQXVPR1uQ-1t1O-DmPzRcQg7kqhxOO8-OCVBCq4a3EMCS2v3RQ9Hkx0 2022-08-18 19:42:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:38 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-18 19:42:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:38 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-18 19:42:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:38 WARN a.m.s.MultiPartOutputStream(close):160 - [MultipartOutputStream for parts 1 - 10000] is already closed 2022-08-18 19:42:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:38 INFO a.m.s.StreamTransferManager(complete):367 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660851757789 with id ABPnzm59V...v3RQ9Hkx0]: Uploading leftover stream [Part number 1 containing 3.34 MB] 2022-08-18 19:42:38 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:38 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660851757789 with id ABPnzm59V...v3RQ9Hkx0]: Finished uploading [Part number 1 containing 3.34 MB] 2022-08-18 19:42:39 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:39 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660851757789 with id ABPnzm59V...v3RQ9Hkx0]: Completed 2022-08-18 19:42:39 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:39 INFO i.a.i.d.s.S3Destination(testMultipartUpload):119 - Finished verification for multipart upload mode 2022-08-18 19:42:40 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 19:42:40 INFO i.a.i.b.IntegrationRunner(runInternal):152 - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-18 19:42:40 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-18 19:42:41 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/0/logs.log 2022-08-18 19:42:41 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-18 19:42:41 INFO i.a.c.EnvConfigs(getEnvOrDefault):977 - Using default value for environment variable METRIC_CLIENT: '' 2022-08-18 19:42:41 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 2022-08-18 19:42:41 INFO i.a.w.g.DefaultReplicationWorker(run):119 - start sync worker. job id: 167 attempt id: 0 2022-08-18 19:42:41 INFO i.a.w.g.DefaultReplicationWorker(run):131 - configured sync modes: {null.campaigns=incremental - append_dedup, null.lists=incremental - append_dedup, null.email_activity=incremental - append} 2022-08-18 19:42:41 INFO i.a.w.i.DefaultAirbyteDestination(start):69 - Running destination... 2022-08-18 19:42:41 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-bigquery:1.1.14 exists... 2022-08-18 19:42:41 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-bigquery:1.1.14 was found locally. 2022-08-18 19:42:41 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-18 19:42:41 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/0 --log-driver none --name destination-bigquery-write-167-0-uvubh --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.1.14 -e WORKER_JOB_ATTEMPT=0 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/destination-bigquery:1.1.14 write --config destination_config.json --catalog destination_catalog.json 2022-08-18 19:42:41 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-mailchimp:0.2.14 exists... 2022-08-18 19:42:41 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-mailchimp:0.2.14 was found locally. 2022-08-18 19:42:41 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-18 19:42:41 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/0 --log-driver none --name source-mailchimp-read-167-0-hcuyf --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/source-mailchimp:0.2.14 -e WORKER_JOB_ATTEMPT=0 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/source-mailchimp:0.2.14 read --config source_config.json --catalog source_catalog.json 2022-08-18 19:42:41 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):303 - Replication thread started. 2022-08-18 19:42:41 INFO i.a.w.g.DefaultReplicationWorker(run):173 - Waiting for source and destination threads to complete. 2022-08-18 19:42:41 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):411 - Destination output thread started. 2022-08-18 19:42:41 source > Starting syncing SourceMailchimp 2022-08-18 19:42:41 source > Syncing stream: campaigns 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: WRITE 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):301 - Selected loading method is set to: GCS 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"AVRO","flattening":"No flattening"} 2022-08-18 19:42:42 destination > 2022-08-18 19:42:42 INFO i.a.i.d.b.BigQueryUtils(isKeepFilesInGcs):317 - All tmp files will be removed from GCS when replication is finished 2022-08-18 19:42:43 destination > 2022-08-18 19:42:43 INFO i.a.i.d.b.BigQueryDestination(getGcsRecordConsumer):288 - Creating BigQuery staging message consumer with staging ID d70c3e3a-6135-4186-84d7-74a9c6a620a6 at 2022-08-18T19:42:42.910Z 2022-08-18 19:42:43 destination > 2022-08-18 19:42:43 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_campaigns, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_aor_mailchimp_ctf_campaigns}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_campaigns}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append_dedup, stagedFiles=[]] 2022-08-18 19:42:43 destination > 2022-08-18 19:42:43 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_email_activity, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_bgp_mailchimp_ctf_email_activity}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_email_activity}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append, stagedFiles=[]] 2022-08-18 19:42:43 destination > 2022-08-18 19:42:43 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_lists, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_cbl_mailchimp_ctf_lists}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_lists}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append_dedup, stagedFiles=[]] 2022-08-18 19:42:43 destination > 2022-08-18 19:42:43 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):116 - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started. 2022-08-18 19:42:43 destination > 2022-08-18 19:42:43 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):98 - Preparing tmp tables in destination started for 3 streams 2022-08-18 19:42:43 destination > 2022-08-18 19:42:43 INFO i.a.i.d.b.BigQueryGcsOperations(createSchemaIfNotExists):86 - Creating dataset do 2022-08-18 19:42:44 destination > 2022-08-18 19:42:44 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_cbl_mailchimp_ctf_lists}} 2022-08-18 19:42:44 destination > 2022-08-18 19:42:44 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_cbl_mailchimp_ctf_lists}} 2022-08-18 19:42:44 destination > 2022-08-18 19:42:44 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_lists (dataset do): data_sync/do/do_mailchimp_ctf_lists/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ 2022-08-18 19:42:44 destination > 2022-08-18 19:42:44 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_lists/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ does not exist in bucket; creating... 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_lists/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ has been created in bucket. 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_aor_mailchimp_ctf_campaigns}} 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_aor_mailchimp_ctf_campaigns}} 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_campaigns (dataset do): data_sync/do/do_mailchimp_ctf_campaigns/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_campaigns/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ does not exist in bucket; creating... 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_campaigns/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ has been created in bucket. 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_bgp_mailchimp_ctf_email_activity}} 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_bgp_mailchimp_ctf_email_activity}} 2022-08-18 19:42:45 destination > 2022-08-18 19:42:45 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_email_activity (dataset do): data_sync/do/do_mailchimp_ctf_email_activity/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ 2022-08-18 19:42:46 destination > 2022-08-18 19:42:46 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_email_activity/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ does not exist in bucket; creating... 2022-08-18 19:42:46 destination > 2022-08-18 19:42:46 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_email_activity/2022/08/18/19/d70c3e3a-6135-4186-84d7-74a9c6a620a6/ has been created in bucket. 2022-08-18 19:42:46 destination > 2022-08-18 19:42:46 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):107 - Preparing tmp tables in destination completed. 2022-08-18 19:42:46 destination > 2022-08-18 19:42:46 INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$addRecord$0):47 - Starting a new buffer for stream mailchimp_ctf_campaigns (current state: 0 bytes in 0 buffers) 2022-08-18 19:42:46 destination > 2022-08-18 19:42:46 INFO i.a.i.d.g.u.GcsUtils(getDefaultAvroSchema):25 - Default schema. 2022-08-18 19:43:08 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 1000 (3 MB) 2022-08-18 19:43:40 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 2000 (8 MB) 2022-08-18 19:44:13 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 3000 (12 MB) 2022-08-18 19:44:24 source > Read 3423 records from campaigns stream 2022-08-18 19:44:24 source > Finished syncing campaigns 2022-08-18 19:44:24 source > SourceMailchimp runtimes: Syncing stream campaigns 0:01:42.906727 2022-08-18 19:44:24 source > Syncing stream: email_activity 2022-08-18 19:44:28 destination > 2022-08-18 19:44:28 INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$addRecord$0):47 - Starting a new buffer for stream mailchimp_ctf_email_activity (current state: 14 MB in 1 buffers) 2022-08-18 19:44:28 destination > 2022-08-18 19:44:28 INFO i.a.i.d.g.u.GcsUtils(getDefaultAvroSchema):25 - Default schema. 2022-08-18 19:44:38 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 4000 (14 MB) 2022-08-18 19:45:05 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 5000 (15 MB) 2022-08-18 19:45:39 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 6000 (16 MB) 2022-08-18 19:46:05 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 7000 (17 MB) 2022-08-18 19:46:31 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 8000 (18 MB) 2022-08-18 19:47:11 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 9000 (19 MB) 2022-08-18 19:47:37 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 10000 (20 MB) 2022-08-18 19:47:51 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 11000 (20 MB) 2022-08-18 19:48:01 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 12000 (21 MB) 2022-08-18 19:48:20 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 13000 (22 MB) 2022-08-18 19:48:37 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 14000 (23 MB) 2022-08-18 19:58:16 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 15000 (24 MB) 2022-08-18 20:09:04 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 16000 (25 MB) 2022-08-18 20:20:35 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 17000 (26 MB) 2022-08-18 20:28:34 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 18000 (27 MB) 2022-08-18 20:43:28 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 19000 (28 MB) 2022-08-18 20:47:00 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 20000 (29 MB) 2022-08-18 20:53:51 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 21000 (30 MB) 2022-08-18 20:53:55 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 22000 (31 MB) 2022-08-18 20:56:12 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 23000 (32 MB) 2022-08-18 20:58:59 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 24000 (33 MB) 2022-08-18 21:01:35 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 25000 (34 MB) 2022-08-18 21:06:16 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 26000 (34 MB) 2022-08-18 21:10:16 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 27000 (35 MB) 2022-08-18 21:13:46 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 28000 (36 MB) 2022-08-18 21:16:21 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 29000 (37 MB) 2022-08-18 21:19:22 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 30000 (38 MB) 2022-08-18 21:19:58 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 31000 (39 MB) 2022-08-18 21:20:34 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 32000 (40 MB) 2022-08-18 21:22:41 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 33000 (41 MB) 2022-08-18 21:24:23 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 34000 (42 MB) 2022-08-18 21:37:40 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 35000 (43 MB) 2022-08-18 21:46:48 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 36000 (44 MB) 2022-08-18 21:48:47 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 37000 (45 MB) 2022-08-18 21:56:34 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 38000 (46 MB) 2022-08-18 21:56:42 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 39000 (47 MB) 2022-08-18 22:00:47 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 40000 (47 MB) 2022-08-18 22:06:32 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 41000 (48 MB) 2022-08-18 22:10:07 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 42000 (49 MB) 2022-08-18 22:10:12 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 43000 (50 MB) 2022-08-18 22:15:22 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 44000 (51 MB) 2022-08-18 22:20:46 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 45000 (52 MB) 2022-08-18 22:24:29 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 46000 (53 MB) 2022-08-18 22:25:27 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 47000 (54 MB) 2022-08-18 22:30:49 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 48000 (55 MB) 2022-08-18 22:35:07 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 49000 (56 MB) 2022-08-18 22:39:48 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 50000 (57 MB) 2022-08-18 22:46:41 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):191 - Running sync worker cancellation... 2022-08-18 22:46:41 INFO i.a.w.g.DefaultReplicationWorker(cancel):449 - Cancelling replication worker... 2022-08-18 22:46:51 INFO i.a.w.g.DefaultReplicationWorker(cancel):457 - Cancelling destination... 2022-08-18 22:46:51 INFO i.a.w.i.DefaultAirbyteDestination(cancel):125 - Attempting to cancel destination process... 2022-08-18 22:46:51 INFO i.a.w.i.DefaultAirbyteDestination(cancel):130 - Destination process exists, cancelling... 2022-08-18 22:46:51 INFO i.a.w.g.DefaultReplicationWorker(run):178 - One of source or destination thread complete. Waiting on the other. 2022-08-18 22:46:51 WARN i.a.c.i.LineGobbler(voidCall):88 - airbyte-destination gobbler IOException: Stream closed. Typically happens when cancelling a job. 2022-08-18 22:46:51 INFO i.a.w.i.DefaultAirbyteDestination(cancel):132 - Cancelled destination process! 2022-08-18 22:46:51 INFO i.a.w.g.DefaultReplicationWorker(cancel):464 - Cancelling source... 2022-08-18 22:46:51 INFO i.a.w.i.DefaultAirbyteSource(cancel):142 - Attempting to cancel source process... 2022-08-18 22:46:51 INFO i.a.w.i.DefaultAirbyteSource(cancel):147 - Source process exists, cancelling... 2022-08-18 22:46:51 INFO i.a.w.i.DefaultAirbyteSource(cancel):149 - Cancelled source process! 2022-08-18 22:46:51 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):195 - Interrupting worker thread... 2022-08-18 22:46:51 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):198 - Cancelling completable future... 2022-08-18 22:46:51 WARN i.a.w.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled. 2022-08-18 22:46:51 WARN i.a.w.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled. 2022-08-18 22:46:51 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-18 22:46:51 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):312 - Stopping temporal heartbeating... 2022-08-18 22:46:51 WARN i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=ad7b0d46-7c03-3f46-9f76-5403808c94d1, activityType=Replicate, attempt=1 java.lang.RuntimeException: java.util.concurrent.CancellationException at io.airbyte.workers.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:310) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.sync.ReplicationActivityImpl.replicate(ReplicationActivityImpl.java:119) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?] at java.lang.Thread.run(Thread.java:1589) [?:?] Caused by: java.util.concurrent.CancellationException at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2510) ~[?:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getCancellationChecker$3(TemporalAttemptExecution.java:201) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.CancellationHandler$TemporalCancellationHandler.checkAndHandleCancellation(CancellationHandler.java:52) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getCancellationChecker$4(TemporalAttemptExecution.java:204) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] ... 3 more 2022-08-18 22:46:51 INFO i.a.w.g.DefaultReplicationWorker(run):180 - Source and destination threads complete. 2022-08-18 22:46:51 ERROR i.a.w.g.DefaultReplicationWorker(run):184 - Sync worker failed. io.airbyte.workers.exception.WorkerException: Source process exit with code 143. This warning is normal if the job was cancelled. at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:182) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:65) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:1589) [?:?] Suppressed: java.io.IOException: Stream closed at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:445) ~[?:?] at java.io.OutputStream.write(OutputStream.java:164) ~[?:?] at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:125) ~[?:?] at java.io.BufferedOutputStream.implFlush(BufferedOutputStream.java:251) ~[?:?] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:245) ~[?:?] at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:416) ~[?:?] at sun.nio.cs.StreamEncoder.lockedFlush(StreamEncoder.java:218) ~[?:?] at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:205) ~[?:?] at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:263) ~[?:?] at java.io.BufferedWriter.implFlush(BufferedWriter.java:372) ~[?:?] at java.io.BufferedWriter.flush(BufferedWriter.java:359) ~[?:?] at io.airbyte.workers.internal.DefaultAirbyteDestination.notifyEndOfInput(DefaultAirbyteDestination.java:98) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.internal.DefaultAirbyteDestination.close(DefaultAirbyteDestination.java:111) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:141) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:65) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:1589) [?:?] 2022-08-18 22:46:51 INFO i.a.w.g.DefaultReplicationWorker(run):243 - sync summary: io.airbyte.config.ReplicationAttemptSummary@70152c8[status=cancelled,recordsSynced=50104,bytesSynced=60489415,startTime=1660851761098,endTime=1660862811303,totalStats=io.airbyte.config.SyncStats@400233ba[recordsEmitted=50104,bytesEmitted=60489415,stateMessagesEmitted=392,recordsCommitted=0],streamStats=[io.airbyte.config.StreamSyncStats@3107c787[streamName=mailchimp_ctf_campaigns,stats=io.airbyte.config.SyncStats@50793eaa[recordsEmitted=3423,bytesEmitted=14822308,stateMessagesEmitted=,recordsCommitted=]], io.airbyte.config.StreamSyncStats@5d63a17f[streamName=mailchimp_ctf_email_activity,stats=io.airbyte.config.SyncStats@cadd68a[recordsEmitted=46681,bytesEmitted=45667107,stateMessagesEmitted=,recordsCommitted=]]]] 2022-08-18 22:46:51 INFO i.a.w.g.DefaultReplicationWorker(run):270 - Source output at least one state message 2022-08-18 22:46:51 WARN i.a.w.g.DefaultReplicationWorker(run):283 - State capture: No state retained. 2022-08-18 22:46:40 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.access_token: is missing but it is required, $.auth_type: must be a constant value oauth2.0 2022-08-18 22:46:40 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.apikey: object found, string expected 2022-08-18 22:46:40 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.method: must be a constant value Standard 2022-08-18 22:46:40 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.credential.hmac_key_access_id: object found, string expected, $.credential.hmac_key_secret: object found, string expected 2022-08-18 22:46:40 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/1/logs.log 2022-08-18 22:46:40 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-18 22:46:40 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-mailchimp:0.2.14 exists... 2022-08-18 22:46:40 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-mailchimp:0.2.14 was found locally. 2022-08-18 22:46:40 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-18 22:46:40 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/1 --log-driver none --name source-mailchimp-check-167-1-avkzx --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/source-mailchimp:0.2.14 -e WORKER_JOB_ATTEMPT=1 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/source-mailchimp:0.2.14 check --config source_config.json 2022-08-18 22:46:41 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Check succeeded 2022-08-18 22:46:41 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-18 22:46:41 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/1/logs.log 2022-08-18 22:46:41 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-18 22:46:41 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-bigquery:1.1.14 exists... 2022-08-18 22:46:41 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-bigquery:1.1.14 was found locally. 2022-08-18 22:46:41 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-18 22:46:41 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/1 --log-driver none --name destination-bigquery-check-167-1-hcrii --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.1.14 -e WORKER_JOB_ATTEMPT=1 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/destination-bigquery:1.1.14 check --config source_config.json 2022-08-18 22:46:43 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:43 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {check=null, config=source_config.json} 2022-08-18 22:46:43 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:43 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-18 22:46:43 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:43 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: CHECK 2022-08-18 22:46:43 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:43 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'} 2022-08-18 22:46:43 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:43 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 22:46:43 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:43 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 22:46:43 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:43 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 22:46:43 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:43 INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):301 - Selected loading method is set to: GCS 2022-08-18 22:46:44 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:44 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"CSV","flattening":"No flattening"} 2022-08-18 22:46:45 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:45 INFO i.a.i.d.s.S3Destination(testSingleUpload):81 - Started testing if all required credentials assigned to user for single file uploading 2022-08-18 22:46:45 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:45 INFO i.a.i.d.s.S3Destination(testSingleUpload):91 - Finished checking for normal upload mode 2022-08-18 22:46:45 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:45 INFO i.a.i.d.s.S3Destination(testMultipartUpload):95 - Started testing if all required credentials assigned to user for multipart upload 2022-08-18 22:46:46 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:46 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to na_airbyte_sync/data_sync/do/test_1660862805938 with full ID ABPnzm4B2XEKbt6kh_CGNaDi2Otsr2L9Vi1cEe_0RxjqBqIccEmA4V_eN9F3ELZhrDV_Ijo 2022-08-18 22:46:46 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:46 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-18 22:46:46 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:46 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-18 22:46:46 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:46 WARN a.m.s.MultiPartOutputStream(close):160 - [MultipartOutputStream for parts 1 - 10000] is already closed 2022-08-18 22:46:46 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:46 INFO a.m.s.StreamTransferManager(complete):367 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660862805938 with id ABPnzm4B2...ZhrDV_Ijo]: Uploading leftover stream [Part number 1 containing 3.34 MB] 2022-08-18 22:46:46 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:46 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660862805938 with id ABPnzm4B2...ZhrDV_Ijo]: Finished uploading [Part number 1 containing 3.34 MB] 2022-08-18 22:46:47 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:47 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660862805938 with id ABPnzm4B2...ZhrDV_Ijo]: Completed 2022-08-18 22:46:47 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:47 INFO i.a.i.d.s.S3Destination(testMultipartUpload):119 - Finished verification for multipart upload mode 2022-08-18 22:46:48 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-18 22:46:48 INFO i.a.i.b.IntegrationRunner(runInternal):152 - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-18 22:46:48 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-18 22:46:48 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/1/logs.log 2022-08-18 22:46:48 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-18 22:46:48 INFO i.a.c.EnvConfigs(getEnvOrDefault):977 - Using default value for environment variable METRIC_CLIENT: '' 2022-08-18 22:46:48 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 2022-08-18 22:46:48 INFO i.a.w.g.DefaultReplicationWorker(run):119 - start sync worker. job id: 167 attempt id: 1 2022-08-18 22:46:48 INFO i.a.w.g.DefaultReplicationWorker(run):131 - configured sync modes: {null.campaigns=incremental - append_dedup, null.lists=incremental - append_dedup, null.email_activity=incremental - append} 2022-08-18 22:46:48 INFO i.a.w.i.DefaultAirbyteDestination(start):69 - Running destination... 2022-08-18 22:46:48 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-bigquery:1.1.14 exists... 2022-08-18 22:46:49 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-bigquery:1.1.14 was found locally. 2022-08-18 22:46:49 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-18 22:46:49 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/1 --log-driver none --name destination-bigquery-write-167-1-xxkcr --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.1.14 -e WORKER_JOB_ATTEMPT=1 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/destination-bigquery:1.1.14 write --config destination_config.json --catalog destination_catalog.json 2022-08-18 22:46:49 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-mailchimp:0.2.14 exists... 2022-08-18 22:46:49 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-mailchimp:0.2.14 was found locally. 2022-08-18 22:46:49 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-18 22:46:49 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/1 --log-driver none --name source-mailchimp-read-167-1-wgnub --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/source-mailchimp:0.2.14 -e WORKER_JOB_ATTEMPT=1 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/source-mailchimp:0.2.14 read --config source_config.json --catalog source_catalog.json 2022-08-18 22:46:49 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):411 - Destination output thread started. 2022-08-18 22:46:49 INFO i.a.w.g.DefaultReplicationWorker(run):173 - Waiting for source and destination threads to complete. 2022-08-18 22:46:49 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):303 - Replication thread started. 2022-08-18 22:46:49 source > Starting syncing SourceMailchimp 2022-08-18 22:46:49 source > Syncing stream: campaigns 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: WRITE 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):301 - Selected loading method is set to: GCS 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"AVRO","flattening":"No flattening"} 2022-08-18 22:46:50 destination > 2022-08-18 22:46:50 INFO i.a.i.d.b.BigQueryUtils(isKeepFilesInGcs):317 - All tmp files will be removed from GCS when replication is finished 2022-08-18 22:46:51 destination > 2022-08-18 22:46:51 INFO i.a.i.d.b.BigQueryDestination(getGcsRecordConsumer):288 - Creating BigQuery staging message consumer with staging ID fe2af065-9851-44a9-a276-631c2f396560 at 2022-08-18T22:46:50.696Z 2022-08-18 22:46:51 destination > 2022-08-18 22:46:51 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_campaigns, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_ful_mailchimp_ctf_campaigns}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_campaigns}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append_dedup, stagedFiles=[]] 2022-08-18 22:46:51 destination > 2022-08-18 22:46:51 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_email_activity, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_qql_mailchimp_ctf_email_activity}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_email_activity}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append, stagedFiles=[]] 2022-08-18 22:46:51 destination > 2022-08-18 22:46:51 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_lists, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_nia_mailchimp_ctf_lists}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_lists}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append_dedup, stagedFiles=[]] 2022-08-18 22:46:51 destination > 2022-08-18 22:46:51 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):116 - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started. 2022-08-18 22:46:51 destination > 2022-08-18 22:46:51 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):98 - Preparing tmp tables in destination started for 3 streams 2022-08-18 22:46:51 destination > 2022-08-18 22:46:51 INFO i.a.i.d.b.BigQueryGcsOperations(createSchemaIfNotExists):86 - Creating dataset do 2022-08-18 22:46:52 destination > 2022-08-18 22:46:52 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_nia_mailchimp_ctf_lists}} 2022-08-18 22:46:52 destination > 2022-08-18 22:46:52 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_nia_mailchimp_ctf_lists}} 2022-08-18 22:46:52 destination > 2022-08-18 22:46:52 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_lists (dataset do): data_sync/do/do_mailchimp_ctf_lists/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ 2022-08-18 22:46:52 destination > 2022-08-18 22:46:52 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_lists/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ does not exist in bucket; creating... 2022-08-18 22:46:52 destination > 2022-08-18 22:46:52 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_lists/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ has been created in bucket. 2022-08-18 22:46:52 destination > 2022-08-18 22:46:52 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_ful_mailchimp_ctf_campaigns}} 2022-08-18 22:46:53 destination > 2022-08-18 22:46:53 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_ful_mailchimp_ctf_campaigns}} 2022-08-18 22:46:53 destination > 2022-08-18 22:46:53 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_campaigns (dataset do): data_sync/do/do_mailchimp_ctf_campaigns/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ 2022-08-18 22:46:53 destination > 2022-08-18 22:46:53 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_campaigns/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ does not exist in bucket; creating... 2022-08-18 22:46:53 destination > 2022-08-18 22:46:53 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_campaigns/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ has been created in bucket. 2022-08-18 22:46:53 destination > 2022-08-18 22:46:53 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_qql_mailchimp_ctf_email_activity}} 2022-08-18 22:46:53 destination > 2022-08-18 22:46:53 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_qql_mailchimp_ctf_email_activity}} 2022-08-18 22:46:53 destination > 2022-08-18 22:46:53 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_email_activity (dataset do): data_sync/do/do_mailchimp_ctf_email_activity/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ 2022-08-18 22:46:53 destination > 2022-08-18 22:46:53 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_email_activity/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ does not exist in bucket; creating... 2022-08-18 22:46:54 destination > 2022-08-18 22:46:54 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_email_activity/2022/08/18/22/fe2af065-9851-44a9-a276-631c2f396560/ has been created in bucket. 2022-08-18 22:46:54 destination > 2022-08-18 22:46:54 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):107 - Preparing tmp tables in destination completed. 2022-08-18 22:46:54 destination > 2022-08-18 22:46:54 INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$addRecord$0):47 - Starting a new buffer for stream mailchimp_ctf_campaigns (current state: 0 bytes in 0 buffers) 2022-08-18 22:46:54 destination > 2022-08-18 22:46:54 INFO i.a.i.d.g.u.GcsUtils(getDefaultAvroSchema):25 - Default schema. 2022-08-18 22:47:17 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 1000 (3 MB) 2022-08-18 22:47:44 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 2000 (8 MB) 2022-08-18 22:49:10 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 3000 (12 MB) 2022-08-18 22:49:27 source > Read 3423 records from campaigns stream 2022-08-18 22:49:27 source > Finished syncing campaigns 2022-08-18 22:49:27 source > SourceMailchimp runtimes: Syncing stream campaigns 0:02:38.044748 2022-08-18 22:49:27 source > Syncing stream: email_activity 2022-08-18 22:49:30 destination > 2022-08-18 22:49:30 INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$addRecord$0):47 - Starting a new buffer for stream mailchimp_ctf_email_activity (current state: 14 MB in 1 buffers) 2022-08-18 22:49:30 destination > 2022-08-18 22:49:30 INFO i.a.i.d.g.u.GcsUtils(getDefaultAvroSchema):25 - Default schema. 2022-08-18 22:49:39 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 4000 (14 MB) 2022-08-18 22:50:00 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 5000 (15 MB) 2022-08-18 22:50:30 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 6000 (16 MB) 2022-08-18 22:50:51 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 7000 (17 MB) 2022-08-18 22:51:14 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 8000 (18 MB) 2022-08-18 22:51:48 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 9000 (19 MB) 2022-08-18 22:52:13 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 10000 (20 MB) 2022-08-18 22:52:26 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 11000 (20 MB) 2022-08-18 22:52:34 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 12000 (21 MB) 2022-08-18 22:52:47 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 13000 (22 MB) 2022-08-18 22:53:02 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 14000 (23 MB) 2022-08-18 23:01:49 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 15000 (24 MB) 2022-08-18 23:11:24 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 16000 (25 MB) 2022-08-18 23:21:16 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 17000 (26 MB) 2022-08-18 23:28:01 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 18000 (27 MB) 2022-08-18 23:39:55 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 19000 (28 MB) 2022-08-18 23:42:50 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 20000 (29 MB) 2022-08-18 23:48:38 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 21000 (30 MB) 2022-08-18 23:48:42 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 22000 (31 MB) 2022-08-18 23:50:42 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 23000 (32 MB) 2022-08-18 23:53:13 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 24000 (33 MB) 2022-08-18 23:55:13 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 25000 (34 MB) 2022-08-18 23:59:04 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 26000 (34 MB) 2022-08-19 00:03:22 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 27000 (35 MB) 2022-08-19 00:07:17 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 28000 (36 MB) 2022-08-19 00:08:58 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):191 - Running sync worker cancellation... 2022-08-19 00:08:58 INFO i.a.w.g.DefaultReplicationWorker(cancel):449 - Cancelling replication worker... 2022-08-19 00:09:08 INFO i.a.w.g.DefaultReplicationWorker(cancel):457 - Cancelling destination... 2022-08-19 00:09:08 INFO i.a.w.i.DefaultAirbyteDestination(cancel):125 - Attempting to cancel destination process... 2022-08-19 00:09:08 INFO i.a.w.i.DefaultAirbyteDestination(cancel):130 - Destination process exists, cancelling... 2022-08-19 00:09:08 INFO i.a.w.g.DefaultReplicationWorker(run):178 - One of source or destination thread complete. Waiting on the other. 2022-08-19 00:09:08 WARN i.a.c.i.LineGobbler(voidCall):88 - airbyte-destination gobbler IOException: Stream closed. Typically happens when cancelling a job. 2022-08-19 00:09:09 INFO i.a.w.i.DefaultAirbyteDestination(cancel):132 - Cancelled destination process! 2022-08-19 00:09:09 INFO i.a.w.g.DefaultReplicationWorker(cancel):464 - Cancelling source... 2022-08-19 00:09:09 INFO i.a.w.i.DefaultAirbyteSource(cancel):142 - Attempting to cancel source process... 2022-08-19 00:09:09 INFO i.a.w.i.DefaultAirbyteSource(cancel):147 - Source process exists, cancelling... 2022-08-19 00:09:09 INFO i.a.w.g.DefaultReplicationWorker(run):180 - Source and destination threads complete. 2022-08-19 00:09:09 ERROR i.a.w.g.DefaultReplicationWorker(run):184 - Sync worker failed. io.airbyte.workers.exception.WorkerException: Source process exit with code 143. This warning is normal if the job was cancelled. at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:182) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:65) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:1589) [?:?] Suppressed: java.io.IOException: Stream closed at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:445) ~[?:?] at java.io.OutputStream.write(OutputStream.java:164) ~[?:?] at java.io.BufferedOutputStream.implWrite(BufferedOutputStream.java:216) ~[?:?] at java.io.BufferedOutputStream.write(BufferedOutputStream.java:205) ~[?:?] at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:313) ~[?:?] at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:385) ~[?:?] at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:361) ~[?:?] at sun.nio.cs.StreamEncoder.lockedWrite(StreamEncoder.java:162) ~[?:?] at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:143) ~[?:?] at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220) ~[?:?] at java.io.BufferedWriter.implFlushBuffer(BufferedWriter.java:178) ~[?:?] at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:163) ~[?:?] at java.io.BufferedWriter.implFlush(BufferedWriter.java:371) ~[?:?] at java.io.BufferedWriter.flush(BufferedWriter.java:359) ~[?:?] at io.airbyte.workers.internal.DefaultAirbyteDestination.notifyEndOfInput(DefaultAirbyteDestination.java:98) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.internal.DefaultAirbyteDestination.close(DefaultAirbyteDestination.java:111) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:141) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:65) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:1589) [?:?] 2022-08-19 00:09:09 INFO i.a.w.g.DefaultReplicationWorker(run):243 - sync summary: io.airbyte.config.ReplicationAttemptSummary@e885a9e[status=cancelled,recordsSynced=28277,bytesSynced=39020736,startTime=1660862808978,endTime=1660867749097,totalStats=io.airbyte.config.SyncStats@4212419d[recordsEmitted=28277,bytesEmitted=39020736,stateMessagesEmitted=136,recordsCommitted=0],streamStats=[io.airbyte.config.StreamSyncStats@5b4e1ef9[streamName=mailchimp_ctf_campaigns,stats=io.airbyte.config.SyncStats@6205625b[recordsEmitted=3423,bytesEmitted=14823344,stateMessagesEmitted=,recordsCommitted=]], io.airbyte.config.StreamSyncStats@2e15882f[streamName=mailchimp_ctf_email_activity,stats=io.airbyte.config.SyncStats@7e5b2ad9[recordsEmitted=24854,bytesEmitted=24197392,stateMessagesEmitted=,recordsCommitted=]]]] 2022-08-19 00:09:09 INFO i.a.w.g.DefaultReplicationWorker(run):270 - Source output at least one state message 2022-08-19 00:09:09 WARN i.a.w.g.DefaultReplicationWorker(run):283 - State capture: No state retained. 2022-08-19 00:09:09 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-19 00:09:09 INFO i.a.w.i.DefaultAirbyteSource(cancel):149 - Cancelled source process! 2022-08-19 00:09:09 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):195 - Interrupting worker thread... 2022-08-19 00:09:09 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):198 - Cancelling completable future... 2022-08-19 00:09:09 WARN i.a.w.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled. 2022-08-19 00:09:09 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):165 - sync summary: io.airbyte.config.StandardSyncOutput@33ba9e57[standardSyncSummary=io.airbyte.config.StandardSyncSummary@34715b1e[status=cancelled,recordsSynced=28277,bytesSynced=39020736,startTime=1660862808978,endTime=1660867749097,totalStats=io.airbyte.config.SyncStats@4212419d[recordsEmitted=28277,bytesEmitted=39020736,stateMessagesEmitted=136,recordsCommitted=0],streamStats=[io.airbyte.config.StreamSyncStats@5b4e1ef9[streamName=mailchimp_ctf_campaigns,stats=io.airbyte.config.SyncStats@6205625b[recordsEmitted=3423,bytesEmitted=14823344,stateMessagesEmitted=,recordsCommitted=]], io.airbyte.config.StreamSyncStats@2e15882f[streamName=mailchimp_ctf_email_activity,stats=io.airbyte.config.SyncStats@7e5b2ad9[recordsEmitted=24854,bytesEmitted=24197392,stateMessagesEmitted=,recordsCommitted=]]]],normalizationSummary=,state=,outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@75681256[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@6e117ecb[stream=io.airbyte.protocol.models.AirbyteStream@93e0d78[name=mailchimp_ctf_campaigns,jsonSchema={"type":"object","title":"Campaign","properties":{"id":{"type":"string","title":"Campaign ID","readOnly":true,"description":"A string that uniquely identifies this campaign."},"type":{"enum":["regular","plaintext","absplit","rss","variate"],"type":"string","title":"Campaign Type","description":"There are four types of [campaigns](https://mailchimp.com/help/getting-started-with-campaigns/) you can create in Mailchimp. A/B Split campaigns have been deprecated and variate campaigns should be used instead."},"status":{"enum":["save","paused","schedule","sending","sent","canceled","canceling","archived"],"type":"string","title":"Campaign Status","readOnly":true,"description":"The current status of the campaign."},"web_id":{"type":"integer","title":"Campaign Web ID","readOnly":true,"description":"The ID used in the Mailchimp web application. View this campaign in your Mailchimp account at `https://{dc}.admin.mailchimp.com/campaigns/show/?id={web_id}`."},"rss_opts":{"type":"object","title":"RSS Options","properties":{"feed_url":{"type":"string","title":"Feed URL","format":"uri","description":"The URL for the RSS feed."},"schedule":{"type":"object","title":"Sending Schedule","properties":{"hour":{"type":"integer","title":"Sending Hour","maximum":23,"minimum":0,"description":"The hour to send the campaign in local time. Acceptable hours are 0-23. For example, '4' would be 4am in [your account's default time zone](https://mailchimp.com/help/set-account-defaults/)."},"daily_send":{"type":"object","title":"Daily Sending Days","properties":{"friday":{"type":"boolean","title":"Friday","description":"Sends the daily RSS Campaign on Fridays."},"monday":{"type":"boolean","title":"Monday","description":"Sends the daily RSS Campaign on Mondays."},"sunday":{"type":"boolean","title":"Sunday","description":"Sends the daily RSS Campaign on Sundays."},"tuesday":{"type":"boolean","title":"tuesday","description":"Sends the daily RSS Campaign on Tuesdays."},"saturday":{"type":"boolean","title":"Saturday","description":"Sends the daily RSS Campaign on Saturdays."},"thursday":{"type":"boolean","title":"Thursday","description":"Sends the daily RSS Campaign on Thursdays."},"wednesday":{"type":"boolean","title":"Monday","description":"Sends the daily RSS Campaign on Wednesdays."}},"description":"The days of the week to send a daily RSS Campaign."},"weekly_send_day":{"enum":["sunday","monday","tuesday","wednesday","thursday","friday","saturday"],"type":"string","title":"Weekly Sending Day","description":"The day of the week to send a weekly RSS Campaign."},"monthly_send_date":{"type":"number","title":"Monthly Sending Day","maximum":31,"minimum":0,"description":"The day of the month to send a monthly RSS Campaign. Acceptable days are 0-31, where '0' is always the last day of a month. Months with fewer than the selected number of days will not have an RSS campaign sent out that day. For example, RSS Campaigns set to send on the 30th will not go out in February."}},"description":"The schedule for sending the RSS Campaign."},"frequency":{"enum":["daily","weekly","monthly"],"type":"string","title":"Frequency","description":"The frequency of the RSS Campaign."},"last_sent":{"type":"string","title":"Last Sent","readOnly":true,"description":"The date the campaign was last sent."},"constrain_rss_img":{"type":"boolean","title":"Constrain RSS Images","description":"Whether to add CSS to images in the RSS feed to constrain their width in campaigns."}},"description":"[RSS](https://mailchimp.com/help/share-your-blog-posts-with-mailchimp/) options for a campaign."},"settings":{"type":"object","title":"Campaign Settings","properties":{"title":{"type":"string","title":"Campaign Title","description":"The title of the campaign."},"to_name":{"type":"string","title":"To Name","description":"The campaign's custom 'To' name. Typically the first name [merge field](https://mailchimp.com/help/getting-started-with-merge-tags/)."},"reply_to":{"type":"string","title":"Reply To Address","description":"The reply-to email address for the campaign."},"timewarp":{"type":"boolean","title":"Timewarp Send","readOnly":true,"description":"Send this campaign using [Timewarp](https://mailchimp.com/help/use-timewarp/)."},"folder_id":{"type":"string","title":"Folder ID","description":"If the campaign is listed in a folder, the id for that folder."},"from_name":{"type":"string","title":"From Name","description":"The 'from' name on the campaign (not an email address)."},"auto_tweet":{"type":"boolean","title":"Auto-Tweet","description":"Automatically tweet a link to the [campaign archive](https://mailchimp.com/help/about-email-campaign-archives-and-pages/) page when the campaign is sent."},"inline_css":{"type":"boolean","title":"Inline CSS","description":"Automatically inline the CSS included with the campaign content."},"auto_footer":{"type":"boolean","title":"Auto-Footer","description":"Automatically append Mailchimp's [default footer](https://mailchimp.com/help/about-campaign-footers/) to the campaign."},"fb_comments":{"type":"boolean","title":"Facebook Comments","description":"Allows Facebook comments on the campaign (also force-enables the Campaign Archive toolbar). Defaults to `true`."},"template_id":{"type":"integer","title":"Template ID","readOnly":false,"description":"The id for the template used in this campaign."},"authenticate":{"type":"boolean","title":"Authentication","description":"Whether Mailchimp [authenticated](https://mailchimp.com/help/about-email-authentication/) the campaign. Defaults to `true`."},"auto_fb_post":{"type":"array","items":{"type":"string"},"title":"Auto Post to Facebook","description":"An array of [Facebook](https://mailchimp.com/help/connect-or-disconnect-the-facebook-integration/) page ids to auto-post to."},"preview_text":{"type":"string","title":"Campaign Preview Text","description":"The preview text for the campaign."},"subject_line":{"type":"string","title":"Campaign Subject Line","description":"The subject line for the campaign."},"drag_and_drop":{"type":"boolean","title":"Drag And Drop Campaign","readOnly":true,"description":"Whether the campaign uses the drag-and-drop editor."},"use_conversation":{"type":"boolean","title":"Conversation","description":"Use Mailchimp Conversation feature to manage out-of-office replies."}},"description":"The settings for your campaign, including subject, from name, reply-to address, and more."},"tracking":{"type":"object","title":"Campaign Tracking Options","properties":{"opens":{"type":"boolean","title":"Opens","description":"Whether to [track opens](https://mailchimp.com/help/about-open-tracking/). Defaults to `true`. Cannot be set to false for variate campaigns."},"capsule":{"type":"object","title":"Capsule CRM Tracking","properties":{"notes":{"type":"boolean","title":"Capsule Note","description":"Update contact notes for a campaign based on subscriber email addresses."}},"description":"Capsule tracking options for a campaign. Must be using Mailchimp's built-in Capsule integration."},"ecomm360":{"type":"boolean","title":"E-commerce Tracking","description":"Whether to enable [eCommerce360](https://mailchimp.com/help/connect-your-online-store-to-mailchimp/) tracking."},"clicktale":{"type":"string","title":"ClickTale Analytics Tracking","description":"The custom slug for [ClickTale](https://mailchimp.com/help/additional-tracking-options-for-campaigns/) tracking (max of 50 bytes)."},"salesforce":{"type":"object","title":"Salesforce CRM Tracking","properties":{"notes":{"type":"boolean","title":"Salesforce Note","description":"Update contact notes for a campaign based on subscriber email addresses."},"campaign":{"type":"boolean","title":"Salesforce Campaign","description":"Create a campaign in a connected Salesforce account."}},"description":"Salesforce tracking options for a campaign. Must be using Mailchimp's built-in [Salesforce integration](https://mailchimp.com/help/integrate-salesforce-with-mailchimp/)."},"html_clicks":{"type":"boolean","title":"HTML Click Tracking","description":"Whether to [track clicks](https://mailchimp.com/help/enable-and-view-click-tracking/) in the HTML version of the campaign. Defaults to `true`. Cannot be set to false for variate campaigns."},"text_clicks":{"type":"boolean","title":"Plain-Text Click Tracking","description":"Whether to [track clicks](https://mailchimp.com/help/enable-and-view-click-tracking/) in the plain-text version of the campaign. Defaults to `true`. Cannot be set to false for variate campaigns."},"goal_tracking":{"type":"boolean","title":"Mailchimp Goal Tracking","description":"Whether to enable [Goal](https://mailchimp.com/help/about-connected-sites/) tracking."},"google_analytics":{"type":"string","title":"Google Analytics Tracking","description":"The custom slug for [Google Analytics](https://mailchimp.com/help/integrate-google-analytics-with-mailchimp/) tracking (max of 50 bytes)."}},"description":"The tracking options for a campaign."},"send_time":{"type":"string","title":"Send Time","readOnly":true,"description":"The date and time a campaign was sent."},"recipients":{"type":"object","title":"List","properties":{"list_id":{"type":"string","title":"List ID","description":"The unique list id."},"list_name":{"type":"string","title":"List Name","readOnly":true,"description":"The name of the list."},"segment_opts":{"type":"object","title":"Segment Options","properties":{"match":{"enum":["any","all"],"type":"string","title":"Match Type","description":"Segment match type."},"conditions":{"type":"array","items":{"oneOf":[{"type":"object","title":"Aim Segment","properties":{"op":{"enum":["open","click","sent","noopen","noclick","nosent"],"type":"string","title":"Segment Operator","example":"open","description":"The status of the member with regard to their campaign interaction. One of the following: opened, clicked, was sent, didn't open, didn't click, or was not sent."},"field":{"enum":["aim"],"type":"string","title":"Segment Field","example":"aim","description":"Segment by interaction with a specific campaign."},"value":{"type":"string","title":"Segment Data","example":"any","description":"Either the web id value for a specific campaign or 'any' to account for subscribers who have/have not interacted with any campaigns."},"condition_type":{"enum":["Aim"],"type":"string","x-value":"Aim"}},"description":"Segment by interaction with a specific campaign."},{"type":"object","title":"Automation Segment","required":["field","op","value"],"properties":{"op":{"enum":["started","completed","not_started","not_completed"],"type":"string","title":"Segment Operator","example":"started","description":"The status of the member with regard to the automation workflow. One of the following: has started the workflow, has completed the workflow, has not started the workflow, or has not completed the workflow."},"field":{"enum":["automation"],"type":"string","title":"Segment Field","example":"automation","description":"Segment by interaction with an Automation workflow."},"value":{"type":"string","title":"Segment Data","example":"2135217","description":"The web id for the automation workflow to segment against."},"condition_type":{"enum":["Automation"],"type":"string","x-value":"Automation"}},"description":"Segment by interaction with an Automation workflow."},{"type":"object","title":"Poll Activity Segment","required":["field","op","value"],"properties":{"op":{"enum":["member","notmember"],"type":"string","title":"Segment Operator","example":"member","description":"Members have/have not interacted with a specific poll in a Mailchimp email."},"field":{"enum":["poll"],"type":"string","title":"Segment Field","example":"poll","description":"Segment by poll activity."},"value":{"type":"number","title":"Segment Operator","example":409,"description":"The id for the poll."},"condition_type":{"enum":["CampaignPoll"],"type":"string","x-value":"CampaignPoll"}},"description":"Segment by poll activity."},{"type":"object","title":"Conversation Segment","required":["field","op","value"],"properties":{"op":{"enum":["member","notmember"],"type":"string","title":"Segment Operator","example":"member","description":"The status of a member's interaction with a conversation. One of the following: has replied or has not replied."},"field":{"enum":["conversation"],"type":"string","title":"Segment Field","example":"conversation","description":"Segment by interaction with a campaign via Conversations."},"value":{"type":"string","title":"Segment Data","example":"any","description":"The web id value for a specific campaign or 'any' to account for subscribers who have/have not interacted with any campaigns."},"condition_type":{"enum":["Conversation"],"type":"string","x-value":"Conversation"}},"description":"Segment by interaction with a campaign via Conversations."},{"type":"object","title":"Date Segment","required":["field","op","value"],"properties":{"op":{"enum":["greater","less","is","not","blank","blank_not","within","notwithin"],"type":"string","title":"Segment Operator","example":"greater","description":"When the event took place: Before, after, is a specific date, is not a specific date, is blank, or is not blank."},"extra":{"type":"string","title":"Segment Extra Value","example":"2015-01-30","description":"When segmenting on 'date' or 'campaign', the date for the segment formatted as YYYY-MM-DD or the web id for the campaign."},"field":{"enum":["timestamp_opt","info_changed","ecomm_date"],"type":"string","title":"Segment Field","example":"timestamp_opt","description":"The type of date field to segment on: The opt-in time for a signup, the date the subscriber was last updated, or the date of their last ecomm purchase."},"value":{"type":"string","title":"Segment Data","example":"date","description":"What type of data to segment on: a specific date, a specific campaign, or the last campaign sent."},"condition_type":{"enum":["Date"],"type":"string","x-value":"Date"}},"description":"Segment by a specific date field."},{"type":"object","title":"Email Client Segment","required":["field","op","value"],"properties":{"op":{"enum":["client_is","client_not"],"type":"string","title":"Segment Operator","example":"client_is","description":"The operation to determine whether we select clients that match the value, or clients that do not match the value."},"field":{"enum":["email_client"],"type":"string","title":"Segment Field","example":"email_client","description":"Segment by use of a particular email client."},"value":{"type":"string","title":"Segment Data","example":"Gmail","description":"The name of the email client."},"condition_type":{"enum":["EmailClient"],"type":"string","x-value":"EmailClient"}},"description":"Segment by use of a particular email client."},{"type":"object","title":"Language Segment","required":["field","op","value"],"properties":{"op":{"enum":["is","not"],"type":"string","title":"Segment Operator","example":"is","description":"Whether the member's language is or is not set to a specific language."},"field":{"enum":["language"],"type":"string","title":"Segment Field","example":"language","description":"Segmenting based off of a subscriber's language."},"value":{"type":"string","title":"Segment Data","example":"en","description":"A two-letter language identifier."},"condition_type":{"enum":["Language"],"type":"string","x-value":"Language"}},"description":"Segment by language."},{"type":"object","title":"Member Rating Segment","required":["field","op","value"],"properties":{"op":{"enum":["is","not","greater","less"],"type":"string","title":"Segment Operator","example":"greater","description":"Members who have have a rating that is/not exactly a given number or members who have a rating greater/less than a given number."},"field":{"enum":["rating"],"type":"string","title":"Segment Field","example":"rating","description":"Segment by member rating."},"value":{"type":"number","title":"Segment Operator","example":4,"description":"The star rating number to segment against."},"condition_type":{"enum":["MemberRating"],"type":"string","x-value":"MemberRating"}},"description":"Segment by member rating."},{"type":"object","title":"Signup Source Segment","required":["field","condition_type","op"],"properties":{"op":{"enum":["source_is","source_not"],"type":"string","title":"Segment Operator","example":"source_is","description":"Whether the member's signup source was/was not a particular value."},"field":{"enum":["source"],"type":"string","title":"Segment Field","example":"source"},"value":{"type":"string","title":"Segment Data","example":"List Import","description":"The signup source."},"condition_type":{"enum":["SignupSource"],"type":"string","title":"Type","x-value":"SignupSource"}},"description":"Segment by signup source."},{"type":"object","title":"SurveyMonkey Segment","required":["field","op","value"],"properties":{"op":{"enum":["started","completed","not_started","not_completed"],"type":"string","title":"Segment Operator","example":"started","description":"The status of the member with regard to the survey. One of the following: has started the survey, has completed the survey, has not started the survey, or has not completed the survey."},"field":{"enum":["survey_monkey"],"type":"string","title":"Segment Field","example":"survey_monkey","description":"Segment by interaction with a SurveyMonkey survey."},"value":{"type":"string","title":"Survey ID","example":"32179586","description":"The unique ID of the SurveyMonkey survey."},"condition_type":{"enum":["SurveyMonkey"],"type":"string","x-value":"SurveyMonkey"}},"description":"Segment by interaction with a SurveyMonkey survey."},{"type":"object","title":"VIP Segment","required":["field","op"],"properties":{"op":{"enum":["member","notmember"],"type":"string","title":"Segment Operator","example":"member","description":"Whether the member is or is not marked as VIP."},"field":{"enum":["gmonkey"],"type":"string","title":"Segment Field","example":"gmonkey","description":"Segment by VIP status."},"condition_type":{"enum":["VIP"],"type":"string","x-value":"VIP"}},"description":"Segment by VIP status."},{"type":"object","title":"Interests Segment","properties":{"op":{"enum":["interestcontains","interestcontainsall","interestnotcontains"],"type":"string","title":"Segment Operator","example":"interestcontains","description":"Whether the member is a part of one, all, or none of the groups."},"field":{"type":"string","title":"Segment Field","example":"interests-123","description":"Segmenting based on interest group information. This should start with 'interests-' followed by the grouping id. Ex. 'interests-123'."},"value":{"type":"array","items":{"type":"string","example":["44401","44405","44409"]},"title":"Segment Value","description":"An array containing strings, each representing a group id."},"condition_type":{"enum":["Interests"],"type":"string","x-value":"Interests"}},"description":"Segment by an interest group merge field."},{"type":"object","title":"Ecommerce Category Segment","properties":{"op":{"enum":["is","not","contains","notcontain","starts","ends"],"type":"string","title":"Segment Operator","example":"is","description":"A member who has purchased from a category/specific item that is/is not a specific name, where the category/item name contains/doesn't contain a specific phrase or string, or a category/item name that starts/ends with a string."},"field":{"enum":["ecomm_cat","ecomm_prod"],"type":"string","title":"Segment Field","example":"ecomm_cat","description":"Segment by purchases in specific items or categories."},"value":{"type":"string","title":"Segment Data","example":"Product","description":"The ecommerce category/item information."},"condition_type":{"enum":["EcommCategory"],"type":"string","x-value":"EcommCategory"}},"description":"Segment by purchases in specific items or categories."},{"type":"object","title":"Ecommerce Number Segment","required":["field","op","value"],"properties":{"op":{"enum":["is","not","greater","less"],"type":"string","title":"Segment Operator","example":"greater","description":"Members who have spent exactly, have not spent exactly, spent more, or spent less than the segment value."},"field":{"enum":["ecomm_spent_avg","ecomm_orders","ecomm_prod_all","ecomm_avg_ord"],"type":"string","title":"Segment Field","example":"ecomm_orders","description":"Segment by average spent total, number of orders, total number of products purchased, or average number of products per order."},"value":{"type":"number","title":"Segment Operator","example":42,"description":"Members who have spent exactly, have not spent exactly, spent more, or spent less than this amount."},"condition_type":{"enum":["EcommNumber"],"type":"string","x-value":"EcommNumber"}},"description":"Segment by average spent total, number of orders, total number of products purchased, or average number of products per order."},{"type":"object","title":"Ecommerce Purchased Segment","properties":{"op":{"enum":["member","notmember"],"type":"string","title":"Segment Operator","example":"member","description":"Members who have have ('member') or have not ('notmember') purchased."},"field":{"enum":["ecomm_purchased"],"type":"string","title":"Segment Field","example":"ecomm_purchased","description":"Segment by whether someone has purchased anything."},"condition_type":{"enum":["EcommPurchased"],"type":"string","x-value":"EcommPurchased"}},"description":"Segment by whether someone has purchased anything."},{"type":"object","title":"Ecommerce Spent Segment","properties":{"op":{"enum":["greater","less"],"type":"string","title":"Segment Operator","example":"greater","description":"Members who have spent 'more' or 'less' than then specified value."},"field":{"enum":["ecomm_spent_one","ecomm_spent_all"],"type":"string","title":"Segment Field","example":"ecomm_spent_one","description":"Segment by amount spent on a single order or across all orders."},"value":{"type":"integer","title":"Segment Data","example":42,"description":"The total amount a member spent."},"condition_type":{"enum":["EcommSpent"],"type":"string","x-value":"EcommSpent"}},"description":"Segment by amount spent on a single order or across all orders."},{"type":"object","title":"Ecommerce Purchased Store Segment","properties":{"op":{"enum":["is","not"],"type":"string","title":"Segment Operator","example":"is","description":"Members who have or have not purchased from a specific store."},"field":{"enum":["ecomm_store"],"type":"string","title":"Segment Field","example":"ecomm_store","description":"Segment by purchases from a specific store."},"value":{"type":"string","title":"Segment Operator","example":"289","description":"The store id to segment against."},"condition_type":{"enum":["EcommStore"],"type":"string","x-value":"EcommStore"}},"description":"Segment by purchases from a specific store."},{"type":"object","title":"Goal Activity Segment","required":["field","op","value"],"properties":{"op":{"enum":["is","goal_not","contains","goal_notcontain","starts","ends"],"type":"string","title":"Segment Operator","example":"is","description":"Whether the website URL is/not exactly, contains/doesn't contain, starts with/ends with a string."},"field":{"enum":["goal"],"type":"string","title":"Segment Field","example":"goal","description":"Segment by Goal activity."},"value":{"type":"string","title":"Segment Value","description":"The URL to check Goal activity against."},"condition_type":{"enum":["GoalActivity"],"type":"string","x-value":"GoalActivity"}},"description":"Segment by Goal activity."},{"type":"object","title":"Goal Timestamp Segment","required":["field","op","value"],"properties":{"op":{"enum":["greater","less","is"],"type":"string","title":"Segment Operator","example":"greater","description":"Whether the website activity happened after, before, or at a given timestamp."},"field":{"enum":["goal_last_visited"],"type":"string","title":"Segment Field","example":"goal_last_visited","description":"Segment by most recent interaction with a website."},"value":{"type":"string","title":"Segment Value","example":"2015-07-20 19:45:21","description":"The date to check Goal activity against."},"condition_type":{"enum":["GoalTimestamp"],"type":"string","x-value":"GoalTimestamp"}},"description":"Segment by most recent interaction with a website."},{"type":"object","title":"Similar Subscribers Segment Member Segment","required":["field","op","value"],"properties":{"op":{"enum":["fuzzy_is","fuzzy_not"],"type":"string","title":"Segment Operator","example":"fuzzy_is","description":"Members who are/are not apart of a 'similar subscribers' segment."},"field":{"enum":["fuzzy_segment"],"type":"string","title":"Segment Field","example":"fuzzy_segment","description":"Segment by similar subscribers."},"value":{"type":"number","title":"Segment Operator","example":48433,"description":"The id for the 'similar subscribers' segment."},"condition_type":{"enum":["FuzzySegment"],"type":"string","x-value":"FuzzySegment"}},"description":"Segment by similar subscribers."},{"type":"object","title":"Static Segment Member Segment","required":["field","op","value"],"properties":{"op":{"enum":["static_is","static_not"],"type":"string","title":"Segment Operator","example":"static_is","description":"Members who are/are not apart of a static segment."},"field":{"enum":["static_segment"],"type":"string","title":"Segment Field","example":"static_segment","description":"Segment by a given static segment."},"value":{"type":"number","title":"Segment Operator","example":48433,"description":"The id for the static segment."},"condition_type":{"enum":["StaticSegment"],"type":"string","x-value":"StaticSegment"}},"description":"Segment by a given static segment."},{"type":"object","title":"Location-Based Segment","required":["field","op","value"],"properties":{"op":{"enum":["ipgeocountry","ipgeonotcountry","ipgeostate","ipgeonotstate"],"type":"string","title":"Segment Operator","example":"ipgeocountry","description":"Segment members who are within a specific country or US state."},"field":{"enum":["ipgeo"],"type":"string","title":"Segment Field","example":"ipgeo","description":"Segmenting subscribers who are within a specific location."},"value":{"type":"string","title":"Segment Data","example":"US","description":"The two-letter country code or US state abbreviation."},"condition_type":{"enum":["IPGeoCountryState"],"type":"string","x-value":"IPGeoCountryState"}},"description":"Segment by a specific country or US state."},{"type":"object","title":"Geolocation Segment","required":["field","op","value","addr","lat","lng"],"properties":{"op":{"enum":["ipgeoin","ipgeonotin"],"type":"string","title":"Segment Operator","example":"ipgeoin","description":"Segment members who are within a specific geographic region."},"lat":{"type":"string","title":"Segment Location Latitude","example":"33.7489954","description":"The latitude of the target location."},"lng":{"type":"string","title":"Segment Location Longitude","example":"-84.3879824","description":"The longitude of the target location."},"addr":{"type":"string","title":"Segment Location Address","example":"Atlanta, GA, USA","description":"The address of the target location."},"field":{"enum":["ipgeo"],"type":"string","title":"Segment Field","example":"ipgeo","description":"Segmenting subscribers who are within a specific location."},"value":{"type":"integer","title":"Segment Data","example":42,"description":"The radius of the target location."},"condition_type":{"enum":["IPGeoIn"],"type":"string","x-value":"IPGeoIn"}},"description":"Segment by a specific geographic region."},{"type":"object","title":"US Zip Code Segment","required":["field","op","value","extra"],"properties":{"op":{"enum":["ipgeoinzip"],"type":"string","title":"Segment Operator","example":"ipgeoinzip","description":"Segment members who are within a specific US zip code."},"extra":{"type":"integer","title":"Extra Data","example":30318,"description":"The zip code to segment against."},"field":{"enum":["ipgeo"],"type":"string","title":"Segment Field","example":"ipgeo","description":"Segmenting subscribers who are within a specific location."},"value":{"type":"integer","title":"Segment Data","example":25,"description":"The radius of the target location."},"condition_type":{"enum":["IPGeoInZip"],"type":"string","x-value":"IPGeoInZip"}},"description":"Segment by a specific US ZIP code."},{"type":"object","title":"Unknown Location-Based Segment","required":["field","op"],"properties":{"op":{"enum":["ipgeounknown"],"type":"string","title":"Segment Operator","example":"ipgeounknown","description":"Segment members for which location information is unknown."},"field":{"enum":["ipgeo"],"type":"string","title":"Segment Field","example":"ipgeo","description":"Segmenting subscribers who are within a specific location."},"condition_type":{"enum":["IPGeoUnknown"],"type":"string","x-value":"IPGeoUnknown"}},"description":"Segment members whose location information is unknown."},{"type":"object","title":"Zip Code Location-Based Segment","required":["field","op","value"],"properties":{"op":{"enum":["ipgeoiszip","ipgeonotzip"],"type":"string","title":"Segment Operator","example":"ipgeonotzip","description":"Segment members who are/are not within a specific US zip code."},"field":{"enum":["ipgeo"],"type":"string","title":"Segment Field","example":"ipgeo","description":"Segmenting subscribers who are within a specific location."},"value":{"type":"integer","title":"Segment Data","example":30318,"description":"The 5-digit zip code."},"condition_type":{"enum":["IPGeoZip"],"type":"string","x-value":"IPGeoZip"}},"description":"Segment by a specific US ZIP code."},{"type":"object","title":"Social Profiles Age Segment","required":["field","op","value"],"properties":{"op":{"enum":["is","not"],"type":"string","title":"Segment Operator","example":"is","description":"Members who are/not the exact criteria listed."},"field":{"enum":["social_age"],"type":"string","title":"Segment Field","example":"social_age","description":"Segment by age ranges in Social Profiles data."},"value":{"enum":["18-24","25-34","35-54","55+"],"type":"string","title":"Segment Operator","example":"35-54","description":"The age range to segment."},"condition_type":{"enum":["SocialAge"],"type":"string","x-value":"SocialAge"}},"description":"Segment by age ranges in Social Profiles data."},{"type":"object","title":"Social Profiles Gender Segment","required":["field","op","value"],"properties":{"op":{"enum":["is","not"],"type":"string","title":"Segment Operator","example":"is","description":"Members who are/not the exact criteria listed."},"field":{"enum":["social_gender"],"type":"string","title":"Segment Field","example":"social_gender","description":"Segment by listed gender in Social Profiles data."},"value":{"enum":["male","female"],"type":"string","title":"Segment Operator","example":"female","description":"The Social Profiles gender to segment."},"condition_type":{"enum":["SocialGender"],"type":"string","x-value":"SocialGender"}},"description":"Segment by listed gender in Social Profiles data."},{"type":"object","title":"Social Profiles Influence Segment","required":["field","op","value"],"properties":{"op":{"enum":["is","not","greater","less"],"type":"string","title":"Segment Operator","example":"greater","description":"Members who have a rating that is/not or greater/less than the rating provided."},"field":{"enum":["social_influence"],"type":"string","title":"Segment Field","example":"social_influence","description":"Segment by influence rating in Social Profiles data."},"value":{"type":"number","title":"Segment Operator","example":2,"description":"The Social Profiles influence rating to segment."},"condition_type":{"enum":["SocialInfluence"],"type":"string","x-value":"SocialInfluence"}},"description":"Segment by influence rating in Social Profiles data."},{"type":"object","title":"Social Profiles Social Network Segment","required":["field","op","value"],"properties":{"op":{"enum":["member","notmember"],"type":"string","title":"Segment Operator","example":"member","description":"Members who are/not on a given social network."},"field":{"enum":["social_network"],"type":"string","title":"Segment Field","example":"social_network","description":"Segment by social network in Social Profiles data."},"value":{"enum":["twitter","facebook","linkedin","flickr","foursquare","lastfm","myspace","quora","vimeo","yelp","youtube"],"type":"string","title":"Segment Operator","example":"twitter","description":"The social network to segment against."},"condition_type":{"enum":["SocialNetworkMember"],"type":"string","x-value":"SocialNetworkMember"}},"description":"Segment by social network in Social Profiles data."},{"type":"object","title":"Social Profiles Social Network Follow Segment","required":["field","op","value"],"properties":{"op":{"enum":["follow","notfollow"],"type":"string","title":"Segment Operator","example":"follow","description":"Members who are/not following a linked account on a given social network."},"field":{"enum":["social_network"],"type":"string","title":"Segment Field","example":"social_network","description":"Segment by social network in Social Profiles data."},"value":{"enum":["twitter_follow"],"type":"string","title":"Segment Operator","example":"twitter_follow","description":"The social network to segment against."},"condition_type":{"enum":["SocialNetworkFollow"],"type":"string","x-value":"SocialNetworkFollow"}},"description":"Segment by social network in Social Profiles data."},{"type":"object","title":"Address Merge Field Segment","required":["field","op"],"properties":{"op":{"enum":["contains","notcontain","blank","blank_not"],"type":"string","title":"Segment Operator","example":"contains","description":"Whether the member's address merge field contains/does not contain a value or is/is not blank."},"field":{"type":"string","title":"Segment Field","example":"MMERGE3","description":"An address-type merge field to segment."},"value":{"type":"string","title":"Segment Value","example":"Atlanta","description":"The value to segment a text merge field with."},"condition_type":{"enum":["AddressMerge"],"type":"string","x-value":"AddressMerge"}},"description":"Segment by an address-type merge field."},{"type":"object","title":"Address/Zip Merge Field Segment","required":["field","op","value","extra"],"properties":{"op":{"enum":["geoin"],"type":"string","title":"Segment Operator","example":"geoin","description":"Whether the member's address merge field is within a given distance from a city or zip."},"extra":{"type":"string","title":"Segment Extra","example":"30318","description":"The city or the zip being used to segment against."},"field":{"type":"string","title":"Segment Field","example":"MMERGE2","description":"An address or zip-type merge field to segment."},"value":{"type":"string","title":"Segment Value","example":"25","description":"The distance from the city/zip."},"condition_type":{"enum":["ZipMerge"],"type":"string","x-value":"ZipMerge"}},"description":"Segment by an address-type merge field within a given distance."},{"type":"object","title":"Birthday Merge Field Segment","required":["field","op"],"properties":{"op":{"enum":["is","not","blank","blank_not"],"type":"string","title":"Segment Operator","example":"is","description":"Whether the member's birthday merge information is/is not a certain date or is/is not blank."},"field":{"type":"string","title":"Segment Field","example":"MMERGE4","description":"A date merge field to segment."},"value":{"type":"string","title":"Segment Value","example":"01/30","description":"A date to segment against (mm/dd)."},"condition_type":{"enum":["BirthdayMerge"],"type":"string","x-value":"BirthdayMerge"}},"description":"Segment by a contact's birthday."},{"type":"object","title":"Date Merge Field Segment","required":["field","op"],"properties":{"op":{"enum":["is","not","less","blank","blank_not","greater"],"type":"string","title":"Segment Operator","example":"is","description":"Whether the member's merge information is/is not, is greater/less than a value or is/is not blank."},"field":{"type":"string","title":"Segment Field","example":"MMERGE5","description":"A date merge field to segment."},"value":{"type":"string","title":"Segment Value","example":"01/30/2015","description":"A date to segment against."},"condition_type":{"enum":["DateMerge"],"type":"string","x-value":"DateMerge"}},"description":"Segment by a given date merge field."},{"type":"object","title":"Dropdown/Radio Merge Field Segment","required":["field","op"],"properties":{"op":{"enum":["is","not","blank","blank_not","notcontain","contains"],"type":"string","title":"Segment Operator","example":"is","description":"Whether the member's merge information is/is not a value or is/is not blank."},"field":{"type":"string","title":"Segment Field","example":"MMERGE6","description":"A merge field to segment."},"value":{"type":"string","title":"Segment Value","example":"Second Choice","description":"The value to segment a text merge field with."},"condition_type":{"enum":["SelectMerge"],"type":"string","x-value":"SelectMerge"}},"description":"An individual segment condition"},{"type":"object","title":"Text or Number Merge Field Segment","required":["field","op"],"properties":{"op":{"enum":["is","not","contains","notcontain","starts","ends","greater","less","blank","blank_not"],"type":"string","title":"Segment Operator","example":"contains","description":"Whether the member's merge information is/is not, contains/does not contain, starts/ends with, or is greater/less than a value"},"field":{"type":"string","title":"Segment Field","example":"MMERGE7","description":"A text or number merge field to segment."},"value":{"type":"string","title":"Segment Value","example":"Freddie's Jokes","description":"The value to segment a text or number merge field with."},"condition_type":{"enum":["TextMerge"],"type":"string","x-value":"TextMerge"}},"description":"Segment by a given text or number merge field."},{"type":"object","title":"Email Segment","required":["field","op"],"properties":{"op":{"enum":["is","not","contains","notcontain","starts","ends","greater","less"],"type":"string","title":"Segment Operator","description":"Whether the email address is/not exactly, contains/doesn't contain, starts/ends with a string."},"field":{"enum":["merge0","EMAIL"],"type":"string","title":"Segment Field","example":"EMAIL","description":"Segmenting based off of a subscriber's email address."},"value":{"type":"string","title":"Segment Value","example":"urist.mcvankab@freddiesjokes.com","description":"The value to compare the email against."},"condition_type":{"enum":["EmailAddress"],"type":"string","x-value":"EmailAddress"}},"description":"Segment by email address."},{"type":"object","title":"Predicted Gender Segment","required":["field","op","value"],"properties":{"op":{"enum":["is","not"],"type":"string","title":"Segment Operator","example":"is","description":"Members who are/not the exact criteria listed."},"field":{"enum":["predicted_gender"],"type":"string","title":"Segment Field","description":"Segment by predicted gender."},"value":{"enum":["male","female"],"type":"string","title":"Segment Operator","example":"female","description":"The predicted gender to segment."},"condition_type":{"enum":["PredictedGender"],"type":"string","x-value":"PredictedGender"}},"description":"Segment by predicted gender."},{"type":"object","title":"Predicted Age Segment","required":["field","op","value"],"properties":{"op":{"enum":["is"],"type":"string","title":"Segment Operator","example":"is","description":"Members who are/not the exact criteria listed."},"field":{"enum":["predicted_age_range"],"type":"string","title":"Segment Field","description":"Segment by predicted age."},"value":{"enum":["18-24","25-34","35-44","45-54","55-64","65+"],"type":"string","title":"Segment Operator","example":"female","description":"The predicted age to segment."},"condition_type":{"enum":["PredictedAge"],"type":"string","x-value":"PredictedAge"}},"description":"Segment by predicted age."},{"type":"object","title":"New Subscribers Prebuilt Segment","properties":{"op":{"enum":["date_within"],"type":"string","title":"Segment Operator","description":"Whe the event took place, namely within a time frame."},"field":{"enum":["timestamp_opt"],"type":"string","title":"Segment Field","description":"Segment by when people subscribed."},"value":{"type":"string","title":"Segment Data","description":"What type of data to segment on: a specific date, a specific campaign, or the last campaign sent."},"condition_type":{"enum":["NewSubscribers"],"type":"string","x-value":"NewSubscribers"}},"description":"Segment by when people subscribed."}],"x-discriminator":{"propertyName":"condition_type"}},"title":"Segment Type","description":"Segment match conditions. There are multiple possible types, see the [condition types documentation](https://mailchimp.com/developer/marketing/docs/alternative-schemas/#segment-condition-schemas)."},"saved_segment_id":{"type":"integer","title":"Saved Segment ID","description":"The id for an existing saved segment."},"prebuilt_segment_id":{"type":"string","title":"Prebuilt Segment Id","example":"subscribers-female","description":"The prebuilt segment id, if a prebuilt segment has been designated for this campaign."}},"description":"An object representing all segmentation options. This object should contain a `saved_segment_id` to use an existing segment, or you can create a new segment by including both `match` and `conditions` options."},"segment_text":{"type":"string","title":"Segment Text","readOnly":true,"description":"A description of the [segment](https://mailchimp.com/help/create-and-send-to-a-segment/) used for the campaign. Formatted as a string marked up with HTML."},"list_is_active":{"type":"boolean","title":"List Status","readOnly":true,"description":"The status of the list used, namely if it's deleted or disabled."},"recipient_count":{"type":"integer","title":"Recipient Count","readOnly":true,"description":"Count of the recipients on the associated list. Formatted as an integer."}},"description":"List settings for the campaign."},"resendable":{"type":"boolean","title":"Resendable","readOnly":true,"description":"Determines if the campaign qualifies to be resent to non-openers."},"archive_url":{"type":"string","title":"Archive URL","readOnly":true,"description":"The link to the campaign's archive version in ISO 8601 format."},"create_time":{"type":"string","title":"Create Time","readOnly":true,"description":"The date and time the campaign was created in ISO 8601 format."},"emails_sent":{"type":"integer","title":"Emails Sent","readOnly":true,"description":"The total number of emails sent for this campaign."},"social_card":{"type":"object","title":"Campaign Social Card","properties":{"title":{"type":"string","title":"Title","description":"The title for the card. Typically the subject line of the campaign."},"image_url":{"type":"string","title":"Image URL","description":"The url for the header image for the card."},"description":{"type":"string","title":"Campaign Description","description":"A short summary of the campaign to display."}},"description":"The preview for the campaign, rendered by social networks like Facebook and Twitter. [Learn more](https://mailchimp.com/help/enable-and-customize-social-cards/)."},"content_type":{"enum":["template","html","url","multichannel"],"type":"string","title":"Content Type","description":"How the campaign's content is put together."},"ab_split_opts":{"type":"object","title":"A/B Testing Options","readOnly":true,"properties":{"subject_a":{"type":"string","title":"Subject Line Group A","description":"For campaigns split on 'Subject Line', the subject line for Group A."},"subject_b":{"type":"string","title":"Subject Line Group B","description":"For campaigns split on 'Subject Line', the subject line for Group B."},"wait_time":{"type":"integer","title":"Wait Time","description":"The amount of time to wait before picking a winner. This cannot be changed after a campaign is sent."},"split_size":{"type":"integer","title":"Split Size","maximum":50,"minimum":1,"description":"The size of the split groups. Campaigns split based on 'schedule' are forced to have a 50/50 split. Valid split integers are between 1-50."},"split_test":{"enum":["subject","from_name","schedule"],"type":"string","title":"Split Test","description":"The type of AB split to run."},"wait_units":{"enum":["hours","days"],"type":"string","title":"Wait Time","description":"How unit of time for measuring the winner ('hours' or 'days'). This cannot be changed after a campaign is sent."},"from_name_a":{"type":"string","title":"From Name Group A","description":"For campaigns split on 'From Name', the name for Group A."},"from_name_b":{"type":"string","title":"From Name Group B","description":"For campaigns split on 'From Name', the name for Group B."},"pick_winner":{"enum":["opens","clicks","manual"],"type":"string","title":"Pick Winner","description":"How we should evaluate a winner. Based on 'opens', 'clicks', or 'manual'."},"send_time_a":{"type":"string","title":"Send Time Group A","description":"The send time for Group A."},"send_time_b":{"type":"string","title":"Send Time Group B","description":"The send time for Group B."},"reply_email_a":{"type":"string","title":"Reply Email Group A","description":"For campaigns split on 'From Name', the reply-to address for Group A."},"reply_email_b":{"type":"string","title":"Reply Email Group B","description":"For campaigns split on 'From Name', the reply-to address for Group B."},"send_time_winner":{"type":"string","title":"Send Time Winner","description":"The send time for the winning version."}},"description":"[A/B Testing](https://mailchimp.com/help/about-ab-testing-campaigns/) options for a campaign."},"report_summary":{"type":"object","title":"Campaign Report Summary","properties":{"opens":{"type":"integer","title":"Automation Opens","readOnly":true,"description":"The total number of opens for a campaign."},"clicks":{"type":"integer","title":"Total Clicks","readOnly":true,"description":"The total number of clicks for an campaign."},"ecommerce":{"type":"object","title":"E-Commerce Report","properties":{"total_spent":{"type":"number","title":"Total Spent","readOnly":true,"description":"The total spent for a campaign. Calculated as the sum of all order totals with no deductions."},"total_orders":{"type":"integer","title":"Total Orders","readOnly":true,"description":"The total orders for a campaign."},"total_revenue":{"type":"number","title":"Total Revenue","readOnly":true,"description":"The total revenue for a campaign. Calculated as the sum of all order totals minus shipping and tax totals."}},"description":"E-Commerce stats for a campaign."},"open_rate":{"type":"number","title":"Open Rate","readOnly":true,"description":"The number of unique opens divided by the total number of successful deliveries."},"click_rate":{"type":"number","title":"Click Rate","readOnly":true,"description":"The number of unique clicks divided by the total number of successful deliveries."},"unique_opens":{"type":"integer","title":"Unique Opens","readOnly":true,"description":"The number of unique opens."},"subscriber_clicks":{"type":"integer","title":"Unique Subscriber Clicks","readOnly":true,"description":"The number of unique clicks."}},"description":"For sent campaigns, a summary of opens, clicks, and e-commerce data."},"delivery_status":{"type":"object","title":"Campaign Delivery Status","properties":{"status":{"enum":["delivering","delivered","canceling","canceled"],"type":"string","title":"Campaign Delivery Status","readOnly":true,"description":"The current state of a campaign delivery."},"enabled":{"type":"boolean","title":"Delivery Status Enabled","readOnly":true,"description":"Whether Campaign Delivery Status is enabled for this account and campaign."},"can_cancel":{"type":"boolean","title":"Campaign Cancelable","readOnly":true,"description":"Whether a campaign send can be canceled."},"emails_sent":{"type":"integer","title":"Emails Sent","readOnly":true,"description":"The total number of emails confirmed sent for this campaign so far."},"emails_canceled":{"type":"integer","title":"Emails Canceled","readOnly":true,"description":"The total number of emails canceled for this campaign."}},"description":"Updates on campaigns in the process of sending."},"long_archive_url":{"type":"string","title":"Long Archive URL","readOnly":true,"description":"The original link to the campaign's archive version."},"variate_settings":{"type":"object","title":"A/B Test Options","properties":{"contents":{"type":"array","items":{"type":"string"},"title":"Content Descriptions","readOnly":true,"description":"Descriptions of possible email contents. To set campaign contents, make a PUT request to /campaigns/{campaign_id}/content with the field 'variate_contents'."},"test_size":{"type":"integer","title":"Test Size","description":"The percentage of recipients to send the test combinations to, must be a value between 10 and 100."},"wait_time":{"type":"integer","title":"Wait Time","description":"The number of minutes to wait before choosing the winning campaign. The value of wait_time must be greater than 0 and in whole hours, specified in minutes."},"from_names":{"type":"array","items":{"type":"string"},"title":"From Names","description":"The possible from names. The number of from_names provided must match the number of reply_to_addresses. If no from_names are provided, settings.from_name will be used."},"send_times":{"type":"array","items":{"type":"string"},"title":"Send Times","description":"The possible send times to test. The times provided should be in the format YYYY-MM-DD HH:MM:SS. If send_times are provided to test, the test_size will be set to 100% and winner_criteria will be ignored."},"combinations":{"type":"array","items":{"type":"object","properties":{"id":{"type":"string","title":"ID","description":"Unique ID for the combination."},"reply_to":{"type":"integer","title":"Reply To","description":"The index of `variate_settings.reply_to_addresses` used."},"from_name":{"type":"integer","title":"From Name","description":"The index of `variate_settings.from_names` used."},"send_time":{"type":"integer","title":"Send Time","description":"The index of `variate_settings.send_times` used."},"recipients":{"type":"integer","title":"Recipients","description":"The number of recipients for this combination."},"subject_line":{"type":"integer","title":"Subject Line","description":"The index of `variate_settings.subject_lines` used."},"content_description":{"type":"integer","title":"Content Description","description":"The index of `variate_settings.contents` used."}}},"title":"Combinations","readOnly":true,"description":"Combinations of possible variables used to build emails."},"subject_lines":{"type":"array","items":{"type":"string"},"title":"Subject Lines","description":"The possible subject lines to test. If no subject lines are provided, settings.subject_line will be used."},"winner_criteria":{"enum":["opens","clicks","manual","total_revenue"],"type":"string","title":"Winning Criteria","description":"The combination that performs the best. This may be determined automatically by click rate, open rate, or total revenue -- or you may choose manually based on the reporting data you find the most valuable. For Multivariate Campaigns testing send_time, winner_criteria is ignored. For Multivariate Campaigns with 'manual' as the winner_criteria, the winner must be chosen in the Mailchimp web application."},"reply_to_addresses":{"type":"array","items":{"type":"string"},"title":"Reply To Addresses","description":"The possible reply-to addresses. The number of reply_to_addresses provided must match the number of from_names. If no reply_to_addresses are provided, settings.reply_to will be used."},"winning_campaign_id":{"type":"string","title":"Winning Campaign ID","readOnly":true,"description":"ID of the campaign that was sent to the remaining recipients based on the winning combination."},"winning_combination_id":{"type":"string","title":"Winning Combination ID","readOnly":true,"description":"ID for the winning combination."}},"description":"The settings specific to A/B test campaigns."},"parent_campaign_id":{"type":"string","title":"Parent Campaign ID","readOnly":true,"description":"If this campaign is the child of another campaign, this identifies the parent campaign. For Example, for RSS or Automation children."},"needs_block_refresh":{"type":"boolean","title":"Needs Block Refresh","readOnly":true,"description":"Determines if the campaign needs its blocks refreshed by opening the web-based campaign editor. Deprecated and will always return false."}},"description":"A summary of an individual campaign's settings and content."},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[create_time],sourceDefinedPrimaryKey=[[id]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[create_time],destinationSyncMode=append_dedup,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@37a91eea[stream=io.airbyte.protocol.models.AirbyteStream@2e650eb[name=mailchimp_ctf_email_activity,jsonSchema={"type":"object","title":"Email Activity","properties":{"ip":{"type":["string","null"],"title":"Action ip address","description":"The IP address recorded for the action."},"url":{"type":["string","null"],"title":"Click url","description":"If the action is a 'click', the URL on which the member clicked."},"type":{"enum":["hard","soft"],"type":["string","null"],"title":"Type","description":"If the action is a 'bounce', the type of bounce received: 'hard', 'soft'."},"action":{"enum":["open","click","bounce"],"type":["string","null"],"title":"action","description":"One of the following actions: 'open', 'click', or 'bounce'"},"list_id":{"type":"string","title":"The unique id for the list.","description":"The unique id for the list."},"email_id":{"type":"string","title":"email MD5 hash.","description":"The MD5 hash of the lowercase version of the list member's email address."},"timestamp":{"type":["string","null"],"title":"Action date and time","format":"date-time","description":"The date and time recorded for the action in ISO 8601 format."},"campaign_id":{"type":"string","title":"The unique id for the campaign.","description":"The unique id for the campaign."},"email_address":{"type":"string","title":"Email address for a subscriber.","description":"Email address for a subscriber."},"list_is_active":{"type":"boolean","title":"The status of the list used.","description":"The status of the list used, namely if it's deleted or disabled."}},"description":"A list of member's subscriber activity in a specific campaign."},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[timestamp],sourceDefinedPrimaryKey=[],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[timestamp],destinationSyncMode=append,primaryKey=[],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@6764915a[stream=io.airbyte.protocol.models.AirbyteStream@1e480503[name=mailchimp_ctf_lists,jsonSchema={"type":"object","title":"Subscriber List","properties":{"id":{"type":"string","title":"List ID","readOnly":true,"description":"A string that uniquely identifies this list."},"name":{"type":"string","title":"List Name","description":"The name of the list."},"stats":{"type":"object","title":"Statistics","readOnly":true,"properties":{"open_rate":{"type":"number","title":"Open Rate","readOnly":true,"description":"The average open rate (a percentage represented as a number between 0 and 100) per campaign for the list (not returned if we haven't calculated it yet)."},"click_rate":{"type":"number","title":"Click Rate","readOnly":true,"description":"The average click rate (a percentage represented as a number between 0 and 100) per campaign for the list (not returned if we haven't calculated it yet)."},"avg_sub_rate":{"type":"number","title":"Average Subscription Rate","readOnly":true,"description":"The average number of subscriptions per month for the list (not returned if we haven't calculated it yet)."},"member_count":{"type":"integer","title":"Member Count","readOnly":true,"description":"The number of active members in the list."},"cleaned_count":{"type":"integer","title":"Cleaned Count","readOnly":true,"description":"The number of members cleaned from the list."},"last_sub_date":{"type":"string","title":"Date of Last List Subscribe","readOnly":true,"description":"The date and time of the last time someone subscribed to this list in ISO 8601 format."},"avg_unsub_rate":{"type":"number","title":"Average Unsubscription Rate","readOnly":true,"description":"The average number of unsubscriptions per month for the list (not returned if we haven't calculated it yet)."},"campaign_count":{"type":"integer","title":"Campaign Count","readOnly":true,"description":"The number of campaigns in any status that use this list."},"total_contacts":{"type":"integer","title":"Total Contacts","readOnly":true,"description":"The number of contacts in the list, including subscribed, unsubscribed, pending, cleaned, deleted, transactional, and those that need to be reconfirmed."},"last_unsub_date":{"type":"string","title":"Date of Last List Unsubscribe","readOnly":true,"description":"The date and time of the last time someone unsubscribed from this list in ISO 8601 format."},"target_sub_rate":{"type":"number","title":"Average Subscription Rate","readOnly":true,"description":"The target number of subscriptions per month for the list to keep it growing (not returned if we haven't calculated it yet)."},"merge_field_count":{"type":"integer","title":"Merge Var Count","readOnly":true,"description":"The number of merge vars for this list (not EMAIL, which is required)."},"unsubscribe_count":{"type":"integer","title":"Unsubscribe Count","readOnly":true,"description":"The number of members who have unsubscribed from the list."},"campaign_last_sent":{"type":"string","title":"Campaign Last Sent","readOnly":true,"description":"The date and time the last campaign was sent to this list in ISO 8601 format. This is updated when a campaign is sent to 10 or more recipients."},"member_count_since_send":{"type":"integer","title":"Member Count Since Send","readOnly":true,"description":"The number of active members in the list since the last campaign was sent."},"cleaned_count_since_send":{"type":"integer","title":"Cleaned Count Since Send","readOnly":true,"description":"The number of members cleaned from the list since the last campaign was sent."},"unsubscribe_count_since_send":{"type":"integer","title":"Unsubscribe Count Since Send","readOnly":true,"description":"The number of members who have unsubscribed since the last campaign was sent."}},"description":"Stats for the list. Many of these are cached for at least five minutes."},"web_id":{"type":"integer","title":"List Web ID","readOnly":true,"description":"The ID used in the Mailchimp web application. View this list in your Mailchimp account at `https://{dc}.admin.mailchimp.com/lists/members/?id={web_id}`."},"contact":{"type":"object","title":"List Contact","properties":{"zip":{"type":"string","title":"Postal Code","description":"The postal or zip code for the list contact."},"city":{"type":"string","title":"City","description":"The city for the list contact."},"phone":{"type":"string","title":"Phone Number","description":"The phone number for the list contact."},"state":{"type":"string","title":"State","description":"The state for the list contact."},"company":{"type":"string","title":"Company Name","description":"The company name for the list."},"country":{"type":"string","title":"Country Code","description":"A two-character ISO3166 country code. Defaults to US if invalid."},"address1":{"type":"string","title":"Address","description":"The street address for the list contact."},"address2":{"type":"string","title":"Address","description":"The street address for the list contact."}},"description":"[Contact information displayed in campaign footers](https://mailchimp.com/help/about-campaign-footers/) to comply with international spam laws."},"modules":{"type":"array","items":{"type":"string"},"title":"Modules","readOnly":true,"description":"Any list-specific modules installed for this list."},"visibility":{"enum":["pub","prv"],"type":"string","title":"Visibility","description":"Whether this list is [public or private](https://mailchimp.com/help/about-list-publicity/)."},"has_welcome":{"type":"boolean","title":"Has Welcome","default":false,"example":false,"description":"Whether or not this list has a welcome automation connected. Welcome Automations: welcomeSeries, singleWelcome, emailFollowup."},"list_rating":{"type":"integer","title":"List Rating","readOnly":true,"description":"An auto-generated activity score for the list (0-5)."},"date_created":{"type":"string","title":"Creation Date","format":"date-time","readOnly":true,"description":"The date and time that this list was created in ISO 8601 format."},"double_optin":{"type":"boolean","title":"Double Opt In","default":false,"description":"Whether or not to require the subscriber to confirm subscription via email."},"beamer_address":{"type":"string","title":"Beamer Address","readOnly":true,"description":"The list's [Email Beamer](https://mailchimp.com/help/use-email-beamer-to-create-a-campaign/) address."},"use_archive_bar":{"type":"boolean","title":"Use Archive Bar","default":false,"description":"Whether campaigns for this list use the [Archive Bar](https://mailchimp.com/help/about-email-campaign-archives-and-pages/) in archives by default."},"campaign_defaults":{"type":"object","title":"Campaign Defaults","properties":{"subject":{"type":"string","title":"Subject","description":"The default subject line for campaigns sent to this list."},"language":{"type":"string","title":"Language","description":"The default language for this lists's forms."},"from_name":{"type":"string","title":"Sender's Name","description":"The default from name for campaigns sent to this list."},"from_email":{"type":"string","title":"Sender's Email Address","description":"The default from email for campaigns sent to this list."}},"description":"[Default values for campaigns](https://mailchimp.com/help/edit-your-emails-subject-preview-text-from-name-or-from-email-address/) created for this list."},"email_type_option":{"type":"boolean","title":"Email Type Option","description":"Whether the list supports [multiple formats for emails](https://mailchimp.com/help/change-list-name-and-defaults/). When set to `true`, subscribers can choose whether they want to receive HTML or plain-text emails. When set to `false`, subscribers will receive HTML emails, with a plain-text alternative backup."},"subscribe_url_long":{"type":"string","title":"Subscribe URL Long","readOnly":true,"description":"The full version of this list's subscribe form (host will vary)."},"notify_on_subscribe":{"type":"string","title":"Notify on Subscribe","default":false,"description":"The email address to send [subscribe notifications](https://mailchimp.com/help/change-subscribe-and-unsubscribe-notifications/) to."},"permission_reminder":{"type":"string","title":"Permission Reminder","description":"The [permission reminder](https://mailchimp.com/help/edit-the-permission-reminder/) for the list."},"subscribe_url_short":{"type":"string","title":"Subscribe URL Short","readOnly":true,"description":"Our [EepURL shortened](https://mailchimp.com/help/share-your-signup-form/) version of this list's subscribe form."},"marketing_permissions":{"type":"boolean","title":"Marketing Permissions","default":false,"description":"Whether or not the list has marketing permissions (eg. GDPR) enabled."},"notify_on_unsubscribe":{"type":"string","title":"Notify on Unsubscribe","default":false,"description":"The email address to send [unsubscribe notifications](https://mailchimp.com/help/change-subscribe-and-unsubscribe-notifications/) to."}},"description":"Information about a specific list."},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[date_created],sourceDefinedPrimaryKey=[[id]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[date_created],destinationSyncMode=append_dedup,primaryKey=[[id]],additionalProperties={}]],additionalProperties={}],failures=[]] 2022-08-19 00:09:09 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):312 - Stopping temporal heartbeating... 2022-08-19 00:08:52 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.access_token: is missing but it is required, $.auth_type: must be a constant value oauth2.0 2022-08-19 00:08:52 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.apikey: object found, string expected 2022-08-19 00:08:52 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.method: must be a constant value Standard 2022-08-19 00:08:52 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.credential.hmac_key_access_id: object found, string expected, $.credential.hmac_key_secret: object found, string expected 2022-08-19 00:08:52 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/2/logs.log 2022-08-19 00:08:52 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-19 00:08:52 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-mailchimp:0.2.14 exists... 2022-08-19 00:08:52 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-mailchimp:0.2.14 was found locally. 2022-08-19 00:08:52 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-19 00:08:52 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/2 --log-driver none --name source-mailchimp-check-167-2-zymbm --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/source-mailchimp:0.2.14 -e WORKER_JOB_ATTEMPT=2 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/source-mailchimp:0.2.14 check --config source_config.json 2022-08-19 00:08:53 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Check succeeded 2022-08-19 00:08:53 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-19 00:08:53 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/2/logs.log 2022-08-19 00:08:53 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-19 00:08:53 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-bigquery:1.1.14 exists... 2022-08-19 00:08:53 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-bigquery:1.1.14 was found locally. 2022-08-19 00:08:53 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-19 00:08:53 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/2 --log-driver none --name destination-bigquery-check-167-2-glwzj --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.1.14 -e WORKER_JOB_ATTEMPT=2 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/destination-bigquery:1.1.14 check --config source_config.json 2022-08-19 00:08:54 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:54 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {check=null, config=source_config.json} 2022-08-19 00:08:54 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:54 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-19 00:08:54 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:54 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: CHECK 2022-08-19 00:08:54 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:54 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'} 2022-08-19 00:08:55 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:55 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-19 00:08:55 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:55 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-19 00:08:55 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:55 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-19 00:08:55 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:55 INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):301 - Selected loading method is set to: GCS 2022-08-19 00:08:56 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:56 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"CSV","flattening":"No flattening"} 2022-08-19 00:08:56 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:56 INFO i.a.i.d.s.S3Destination(testSingleUpload):81 - Started testing if all required credentials assigned to user for single file uploading 2022-08-19 00:08:57 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:57 INFO i.a.i.d.s.S3Destination(testSingleUpload):91 - Finished checking for normal upload mode 2022-08-19 00:08:57 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:57 INFO i.a.i.d.s.S3Destination(testMultipartUpload):95 - Started testing if all required credentials assigned to user for multipart upload 2022-08-19 00:08:57 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:57 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to na_airbyte_sync/data_sync/do/test_1660867737428 with full ID ABPnzm4alRd8sXjrCKgDeHWU-lc8XmMairrTVjkOjkWQwTSM0BTFMuRKBpd_ccxVasbLRAY 2022-08-19 00:08:57 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:57 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-19 00:08:57 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:57 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-19 00:08:57 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:57 WARN a.m.s.MultiPartOutputStream(close):160 - [MultipartOutputStream for parts 1 - 10000] is already closed 2022-08-19 00:08:57 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:57 INFO a.m.s.StreamTransferManager(complete):367 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660867737428 with id ABPnzm4al...xVasbLRAY]: Uploading leftover stream [Part number 1 containing 3.34 MB] 2022-08-19 00:08:58 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:58 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660867737428 with id ABPnzm4al...xVasbLRAY]: Finished uploading [Part number 1 containing 3.34 MB] 2022-08-19 00:08:59 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:59 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to na_airbyte_sync/data_sync/do/test_1660867737428 with id ABPnzm4al...xVasbLRAY]: Completed 2022-08-19 00:08:59 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:08:59 INFO i.a.i.d.s.S3Destination(testMultipartUpload):119 - Finished verification for multipart upload mode 2022-08-19 00:09:00 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):61 - 2022-08-19 00:09:00 INFO i.a.i.b.IntegrationRunner(runInternal):152 - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-19 00:09:00 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-19 00:09:00 INFO i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/167/2/logs.log 2022-08-19 00:09:00 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.40.0-alpha 2022-08-19 00:09:00 INFO i.a.c.EnvConfigs(getEnvOrDefault):977 - Using default value for environment variable METRIC_CLIENT: '' 2022-08-19 00:09:00 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 2022-08-19 00:09:00 INFO i.a.w.g.DefaultReplicationWorker(run):119 - start sync worker. job id: 167 attempt id: 2 2022-08-19 00:09:00 INFO i.a.w.g.DefaultReplicationWorker(run):131 - configured sync modes: {null.campaigns=incremental - append_dedup, null.lists=incremental - append_dedup, null.email_activity=incremental - append} 2022-08-19 00:09:00 INFO i.a.w.i.DefaultAirbyteDestination(start):69 - Running destination... 2022-08-19 00:09:00 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/destination-bigquery:1.1.14 exists... 2022-08-19 00:09:00 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/destination-bigquery:1.1.14 was found locally. 2022-08-19 00:09:00 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-19 00:09:00 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/2 --log-driver none --name destination-bigquery-write-167-2-yosju --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.1.14 -e WORKER_JOB_ATTEMPT=2 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/destination-bigquery:1.1.14 write --config destination_config.json --catalog destination_catalog.json 2022-08-19 00:09:00 INFO i.a.c.i.LineGobbler(voidCall):83 - Checking if airbyte/source-mailchimp:0.2.14 exists... 2022-08-19 00:09:00 INFO i.a.c.i.LineGobbler(voidCall):83 - airbyte/source-mailchimp:0.2.14 was found locally. 2022-08-19 00:09:00 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 167 2022-08-19 00:09:00 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/167/2 --log-driver none --name source-mailchimp-read-167-2-qxmqy --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e WORKER_CONNECTOR_IMAGE=airbyte/source-mailchimp:0.2.14 -e WORKER_JOB_ATTEMPT=2 -e AIRBYTE_VERSION=0.40.0-alpha -e WORKER_JOB_ID=167 airbyte/source-mailchimp:0.2.14 read --config source_config.json --catalog source_catalog.json 2022-08-19 00:09:00 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):411 - Destination output thread started. 2022-08-19 00:09:00 INFO i.a.w.g.DefaultReplicationWorker(run):173 - Waiting for source and destination threads to complete. 2022-08-19 00:09:00 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):303 - Replication thread started. 2022-08-19 00:09:01 source > Starting syncing SourceMailchimp 2022-08-19 00:09:01 source > Syncing stream: campaigns 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: WRITE 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):301 - Selected loading method is set to: GCS 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"AVRO","flattening":"No flattening"} 2022-08-19 00:09:02 destination > 2022-08-19 00:09:02 INFO i.a.i.d.b.BigQueryUtils(isKeepFilesInGcs):317 - All tmp files will be removed from GCS when replication is finished 2022-08-19 00:09:03 destination > 2022-08-19 00:09:03 INFO i.a.i.d.b.BigQueryDestination(getGcsRecordConsumer):288 - Creating BigQuery staging message consumer with staging ID b991fdd6-1bca-4ae9-9a38-47e906c5318e at 2022-08-19T00:09:02.603Z 2022-08-19 00:09:03 destination > 2022-08-19 00:09:03 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_campaigns, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_qbo_mailchimp_ctf_campaigns}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_campaigns}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append_dedup, stagedFiles=[]] 2022-08-19 00:09:03 destination > 2022-08-19 00:09:03 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_email_activity, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_tbs_mailchimp_ctf_email_activity}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_email_activity}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append, stagedFiles=[]] 2022-08-19 00:09:03 destination > 2022-08-19 00:09:03 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$createWriteConfigs$1):86 - BigQuery write config: BigQueryWriteConfig[streamName=mailchimp_ctf_lists, namespace=null, datasetId=do, datasetLocation=US, tmpTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_wmm_mailchimp_ctf_lists}}, targetTableId=GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_raw_mailchimp_ctf_lists}}, tableSchema=Schema{fields=[Field{name=_airbyte_ab_id, type=STRING, mode=null, description=null, policyTags=null}, Field{name=_airbyte_emitted_at, type=TIMESTAMP, mode=null, description=null, policyTags=null}, Field{name=_airbyte_data, type=STRING, mode=null, description=null, policyTags=null}]}, syncMode=append_dedup, stagedFiles=[]] 2022-08-19 00:09:03 destination > 2022-08-19 00:09:03 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):116 - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started. 2022-08-19 00:09:03 destination > 2022-08-19 00:09:03 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):98 - Preparing tmp tables in destination started for 3 streams 2022-08-19 00:09:03 destination > 2022-08-19 00:09:03 INFO i.a.i.d.b.BigQueryGcsOperations(createSchemaIfNotExists):86 - Creating dataset do 2022-08-19 00:09:03 destination > 2022-08-19 00:09:03 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_wmm_mailchimp_ctf_lists}} 2022-08-19 00:09:04 destination > 2022-08-19 00:09:04 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_wmm_mailchimp_ctf_lists}} 2022-08-19 00:09:04 destination > 2022-08-19 00:09:04 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_lists (dataset do): data_sync/do/do_mailchimp_ctf_lists/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ 2022-08-19 00:09:04 destination > 2022-08-19 00:09:04 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_lists/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ does not exist in bucket; creating... 2022-08-19 00:09:04 destination > 2022-08-19 00:09:04 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_lists/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ has been created in bucket. 2022-08-19 00:09:04 destination > 2022-08-19 00:09:04 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_qbo_mailchimp_ctf_campaigns}} 2022-08-19 00:09:05 destination > 2022-08-19 00:09:05 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_qbo_mailchimp_ctf_campaigns}} 2022-08-19 00:09:05 destination > 2022-08-19 00:09:05 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_campaigns (dataset do): data_sync/do/do_mailchimp_ctf_campaigns/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ 2022-08-19 00:09:05 destination > 2022-08-19 00:09:05 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_campaigns/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ does not exist in bucket; creating... 2022-08-19 00:09:05 destination > 2022-08-19 00:09:05 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_campaigns/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ has been created in bucket. 2022-08-19 00:09:05 destination > 2022-08-19 00:09:05 INFO i.a.i.d.b.BigQueryGcsOperations(createTmpTableIfNotExists):94 - Creating tmp table GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_tbs_mailchimp_ctf_email_activity}} 2022-08-19 00:09:05 destination > 2022-08-19 00:09:05 INFO i.a.i.d.b.BigQueryUtils(createPartitionedTable):131 - Partitioned table created successfully: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=do, tableId=_airbyte_tmp_tbs_mailchimp_ctf_email_activity}} 2022-08-19 00:09:05 destination > 2022-08-19 00:09:05 INFO i.a.i.d.b.BigQueryGcsOperations(createStageIfNotExists):101 - Creating staging path for stream mailchimp_ctf_email_activity (dataset do): data_sync/do/do_mailchimp_ctf_email_activity/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ 2022-08-19 00:09:05 destination > 2022-08-19 00:09:05 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_email_activity/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ does not exist in bucket; creating... 2022-08-19 00:09:06 destination > 2022-08-19 00:09:06 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object na_airbyte_sync/data_sync/do/do_mailchimp_ctf_email_activity/2022/08/19/00/b991fdd6-1bca-4ae9-9a38-47e906c5318e/ has been created in bucket. 2022-08-19 00:09:06 destination > 2022-08-19 00:09:06 INFO i.a.i.d.b.BigQueryStagingConsumerFactory(lambda$onStartFunction$3):107 - Preparing tmp tables in destination completed. 2022-08-19 00:09:06 destination > 2022-08-19 00:09:06 INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$addRecord$0):47 - Starting a new buffer for stream mailchimp_ctf_campaigns (current state: 0 bytes in 0 buffers) 2022-08-19 00:09:06 destination > 2022-08-19 00:09:06 INFO i.a.i.d.g.u.GcsUtils(getDefaultAvroSchema):25 - Default schema. 2022-08-19 00:09:25 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 1000 (3 MB) 2022-08-19 00:09:55 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 2000 (8 MB) 2022-08-19 00:11:32 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 3000 (12 MB) 2022-08-19 00:11:52 source > Read 3423 records from campaigns stream 2022-08-19 00:11:52 source > Finished syncing campaigns 2022-08-19 00:11:52 source > SourceMailchimp runtimes: Syncing stream campaigns 0:02:50.989224 2022-08-19 00:11:52 source > Syncing stream: email_activity 2022-08-19 00:11:56 destination > 2022-08-19 00:11:56 INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$addRecord$0):47 - Starting a new buffer for stream mailchimp_ctf_email_activity (current state: 14 MB in 1 buffers) 2022-08-19 00:11:56 destination > 2022-08-19 00:11:56 INFO i.a.i.d.g.u.GcsUtils(getDefaultAvroSchema):25 - Default schema. 2022-08-19 00:12:10 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 4000 (14 MB) 2022-08-19 00:12:40 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 5000 (15 MB) 2022-08-19 00:13:08 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 6000 (16 MB) 2022-08-19 00:13:33 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 7000 (17 MB) 2022-08-19 00:13:57 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 8000 (18 MB) 2022-08-19 00:14:34 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 9000 (19 MB) 2022-08-19 00:15:01 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 10000 (20 MB) 2022-08-19 00:15:19 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 11000 (20 MB) 2022-08-19 00:15:30 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 12000 (21 MB) 2022-08-19 00:15:47 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 13000 (22 MB) 2022-08-19 00:16:11 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):329 - Records read: 14000 (23 MB) 2022-08-19 00:23:40 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):191 - Running sync worker cancellation... 2022-08-19 00:23:40 INFO i.a.w.g.DefaultReplicationWorker(cancel):449 - Cancelling replication worker... 2022-08-19 00:23:50 INFO i.a.w.g.DefaultReplicationWorker(cancel):457 - Cancelling destination... 2022-08-19 00:23:50 INFO i.a.w.i.DefaultAirbyteDestination(cancel):125 - Attempting to cancel destination process... 2022-08-19 00:23:50 INFO i.a.w.i.DefaultAirbyteDestination(cancel):130 - Destination process exists, cancelling... 2022-08-19 00:23:50 INFO i.a.w.g.DefaultReplicationWorker(run):178 - One of source or destination thread complete. Waiting on the other. 2022-08-19 00:23:50 INFO i.a.w.i.DefaultAirbyteDestination(cancel):132 - Cancelled destination process! 2022-08-19 00:23:50 INFO i.a.w.g.DefaultReplicationWorker(cancel):464 - Cancelling source... 2022-08-19 00:23:50 INFO i.a.w.i.DefaultAirbyteSource(cancel):142 - Attempting to cancel source process... 2022-08-19 00:23:50 INFO i.a.w.i.DefaultAirbyteSource(cancel):147 - Source process exists, cancelling... 2022-08-19 00:23:50 INFO i.a.w.g.DefaultReplicationWorker(run):180 - Source and destination threads complete. 2022-08-19 00:23:50 INFO i.a.w.i.DefaultAirbyteSource(cancel):149 - Cancelled source process! 2022-08-19 00:23:50 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):195 - Interrupting worker thread... 2022-08-19 00:23:50 INFO i.a.w.t.TemporalAttemptExecution(lambda$getCancellationChecker$3):198 - Cancelling completable future... 2022-08-19 00:23:50 WARN i.a.w.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled. 2022-08-19 00:23:50 WARN i.a.w.t.CancellationHandler$TemporalCancellationHandler(checkAndHandleCancellation):53 - Job either timed out or was cancelled. 2022-08-19 00:23:50 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-19 00:23:50 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):312 - Stopping temporal heartbeating... 2022-08-19 00:23:50 ERROR i.a.w.g.DefaultReplicationWorker(run):184 - Sync worker failed. io.airbyte.workers.exception.WorkerException: Source process exit with code 143. This warning is normal if the job was cancelled. at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:182) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:65) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:1589) [?:?] Suppressed: java.io.IOException: Stream closed at java.lang.ProcessBuilder$NullOutputStream.write(ProcessBuilder.java:445) ~[?:?] at java.io.OutputStream.write(OutputStream.java:164) ~[?:?] at java.io.BufferedOutputStream.implWrite(BufferedOutputStream.java:216) ~[?:?] at java.io.BufferedOutputStream.write(BufferedOutputStream.java:205) ~[?:?] at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:313) ~[?:?] at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:385) ~[?:?] at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:361) ~[?:?] at sun.nio.cs.StreamEncoder.lockedWrite(StreamEncoder.java:162) ~[?:?] at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:143) ~[?:?] at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220) ~[?:?] at java.io.BufferedWriter.implFlushBuffer(BufferedWriter.java:178) ~[?:?] at java.io.BufferedWriter.flushBuffer(BufferedWriter.java:163) ~[?:?] at java.io.BufferedWriter.implFlush(BufferedWriter.java:371) ~[?:?] at java.io.BufferedWriter.flush(BufferedWriter.java:359) ~[?:?] at io.airbyte.workers.internal.DefaultAirbyteDestination.notifyEndOfInput(DefaultAirbyteDestination.java:98) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.internal.DefaultAirbyteDestination.close(DefaultAirbyteDestination.java:111) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:141) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:65) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:1589) [?:?] 2022-08-19 00:23:50 WARN i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=112408e6-70ee-39a3-b352-ac477df77b51, activityType=Replicate, attempt=1 java.lang.RuntimeException: java.util.concurrent.CancellationException at io.airbyte.workers.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:310) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.sync.ReplicationActivityImpl.replicate(ReplicationActivityImpl.java:119) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:578) ~[?:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?] at java.lang.Thread.run(Thread.java:1589) [?:?] Caused by: java.util.concurrent.CancellationException at java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2510) ~[?:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getCancellationChecker$3(TemporalAttemptExecution.java:201) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.CancellationHandler$TemporalCancellationHandler.checkAndHandleCancellation(CancellationHandler.java:52) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getCancellationChecker$4(TemporalAttemptExecution.java:204) ~[io.airbyte-airbyte-workers-0.40.0-alpha.jar:?] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577) ~[?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358) ~[?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[?:?] ... 3 more 2022-08-19 00:23:50 INFO i.a.w.g.DefaultReplicationWorker(run):243 - sync summary: io.airbyte.config.ReplicationAttemptSummary@762da3b7[status=cancelled,recordsSynced=14917,bytesSynced=25586121,startTime=1660867740738,endTime=1660868630970,totalStats=io.airbyte.config.SyncStats@207c358a[recordsEmitted=14917,bytesEmitted=25586121,stateMessagesEmitted=7,recordsCommitted=0],streamStats=[io.airbyte.config.StreamSyncStats@172dab33[streamName=mailchimp_ctf_campaigns,stats=io.airbyte.config.SyncStats@304dce9a[recordsEmitted=3423,bytesEmitted=14823353,stateMessagesEmitted=,recordsCommitted=]], io.airbyte.config.StreamSyncStats@2be6f33f[streamName=mailchimp_ctf_email_activity,stats=io.airbyte.config.SyncStats@6a7743ac[recordsEmitted=11494,bytesEmitted=10762768,stateMessagesEmitted=,recordsCommitted=]]]] 2022-08-19 00:23:50 INFO i.a.w.g.DefaultReplicationWorker(run):270 - Source output at least one state message 2022-08-19 00:23:50 WARN i.a.w.g.DefaultReplicationWorker(run):283 - State capture: No state retained.