2022-10-10 18:34:05 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.client_id: is missing but it is required, $.client_secret: is missing but it is required, $.access_token: is missing but it is required, $.auth_type: must be a constant value OAuth2.0 2022-10-10 18:34:05 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.token: object found, string expected 2022-10-10 18:34:05 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.format_type: does not have a value in the enumeration [Avro], $.compression_codec: string found, object expected, $.compression_codec: should be valid to one and only one of the schemas 2022-10-10 18:34:05 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.flattening: is missing but it is required, $.format_type: does not have a value in the enumeration [CSV] 2022-10-10 18:34:05 INFO i.a.v.j.JsonSchemaValidator(test):71 - JSON schema validation failed. errors: $.format_type: does not have a value in the enumeration [JSONL] 2022-10-10 18:34:31 destination > 2022-10-10 18:34:27 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-10-10 18:34:05 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Cloud storage job log path: /workspace/932/0/logs.log 2022-10-10 18:34:05 INFO i.a.w.t.TemporalAttemptExecution(get):111 - Executing worker wrapper. Airbyte version: 0.40.6 2022-10-10 18:34:05 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = source-notion-check-932-0-wukts for airbyte/source-notion:0.1.10 with resources io.airbyte.config.ResourceRequirements@7a220907[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-10 18:34:05 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:05 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2022-10-10 18:34:05 INFO i.a.w.p.KubeProcessFactory(create):103 - source-notion-check-932-0-wukts stdoutLocalPort = 9018 2022-10-10 18:34:05 INFO i.a.w.p.KubeProcessFactory(create):106 - source-notion-check-932-0-wukts stderrLocalPort = 9019 2022-10-10 18:34:05 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:05 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):589 - Creating stdout socket server... 2022-10-10 18:34:05 INFO i.a.w.p.KubePodProcess():520 - Creating pod source-notion-check-932-0-wukts... 2022-10-10 18:34:05 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):607 - Creating stderr socket server... 2022-10-10 18:34:05 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):309 - Waiting for init container to be ready before copying files... 2022-10-10 18:34:05 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):313 - Init container present.. 2022-10-10 18:34:07 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):316 - Init container ready.. 2022-10-10 18:34:07 INFO i.a.w.p.KubePodProcess():551 - Copying files... 2022-10-10 18:34:07 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: source_config.json 2022-10-10 18:34:07 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/3a5e4494-cd8c-4032-9663-5ed9c19a1564/source_config.json dev/source-notion-check-932-0-wukts:/config/source_config.json -c init 2022-10-10 18:34:07 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:08 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:08 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: FINISHED_UPLOADING 2022-10-10 18:34:08 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/9cf7a537-d0bf-47bc-b7ea-083dec7ce371/FINISHED_UPLOADING dev/source-notion-check-932-0-wukts:/config/FINISHED_UPLOADING -c init 2022-10-10 18:34:08 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:08 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:08 INFO i.a.w.p.KubePodProcess():554 - Waiting until pod is ready... 2022-10-10 18:34:09 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):598 - Setting stdout... 2022-10-10 18:34:09 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):610 - Setting stderr... 2022-10-10 18:34:09 INFO i.a.w.p.KubePodProcess():570 - Reading pod IP... 2022-10-10 18:34:09 INFO i.a.w.p.KubePodProcess():572 - Pod IP: 14.0.3.224 2022-10-10 18:34:09 INFO i.a.w.p.KubePodProcess():579 - Using null stdin output stream... 2022-10-10 18:34:10 INFO i.a.w.i.DefaultAirbyteStreamFactory(internalLog):103 - Check succeeded 2022-10-10 18:34:12 INFO i.a.w.p.KubePodProcess(close):737 - (pod: dev / source-notion-check-932-0-wukts) - Closed all resources for pod 2022-10-10 18:34:12 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:12 INFO i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling... 2022-10-10 18:34:12 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK ----- 2022-10-10 18:34:12 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:12 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Cloud storage job log path: /workspace/932/0/logs.log 2022-10-10 18:34:12 INFO i.a.w.t.TemporalAttemptExecution(get):111 - Executing worker wrapper. Airbyte version: 0.40.6 2022-10-10 18:34:12 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = destination-s3-check-932-0-ncuwl for airbyte/destination-s3:0.3.15 with resources io.airbyte.config.ResourceRequirements@7a220907[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-10 18:34:12 INFO i.a.w.p.KubeProcessFactory(create):103 - destination-s3-check-932-0-ncuwl stdoutLocalPort = 9020 2022-10-10 18:34:12 INFO i.a.w.p.KubeProcessFactory(create):106 - destination-s3-check-932-0-ncuwl stderrLocalPort = 9021 2022-10-10 18:34:12 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:12 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK ----- 2022-10-10 18:34:12 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):589 - Creating stdout socket server... 2022-10-10 18:34:12 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:12 INFO i.a.w.p.KubePodProcess():520 - Creating pod destination-s3-check-932-0-ncuwl... 2022-10-10 18:34:12 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):607 - Creating stderr socket server... 2022-10-10 18:34:12 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):309 - Waiting for init container to be ready before copying files... 2022-10-10 18:34:12 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):313 - Init container present.. 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):316 - Init container ready.. 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess():551 - Copying files... 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: source_config.json 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/f64a20c8-27f5-40bc-b7bb-826416ef3753/source_config.json dev/destination-s3-check-932-0-ncuwl:/config/source_config.json -c init 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: FINISHED_UPLOADING 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/138573f5-8c1c-45e7-a4c7-a729df4fec34/FINISHED_UPLOADING dev/destination-s3-check-932-0-ncuwl:/config/FINISHED_UPLOADING -c init 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:14 INFO i.a.w.p.KubePodProcess():554 - Waiting until pod is ready... 2022-10-10 18:34:15 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):598 - Setting stdout... 2022-10-10 18:34:15 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):610 - Setting stderr... 2022-10-10 18:34:16 INFO i.a.w.p.KubePodProcess():570 - Reading pod IP... 2022-10-10 18:34:16 INFO i.a.w.p.KubePodProcess():572 - Pod IP: 14.0.3.225 2022-10-10 18:34:16 INFO i.a.w.p.KubePodProcess():579 - Using null stdin output stream... 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {check=null, config=source_config.json} 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.destination.s3.S3Destination 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: CHECK 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'} 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword requires - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"Parquet","page_size_kb":1024,"block_size_mb":128,"compression_codec":"UNCOMPRESSED","dictionary_encoding":true,"max_padding_size_mb":8,"dictionary_page_size_kb":1024} 2022-10-10 18:34:17 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:17 INFO i.a.i.d.s.S3DestinationConfig(createS3Client):190 - Creating S3 client... 2022-10-10 18:34:18 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:18 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):102 - Storage Object redshift-dev-s3-bucket/csv_path_dev_2 does not exist in bucket; creating... 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO i.a.i.d.s.S3StorageOperations(createBucketObjectIfNotExists):104 - Storage Object redshift-dev-s3-bucket/csv_path_dev_2 has been created in bucket. 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO i.a.i.d.s.S3Destination(testIAMUserHasListObjectPermission):156 - Started testing if IAM user can call listObjects on the destination bucket 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO i.a.i.d.s.S3Destination(testIAMUserHasListObjectPermission):159 - Finished checking for listObjects permission 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO i.a.i.d.s.S3Destination(testSingleUpload):81 - Started testing if all required credentials assigned to user for single file uploading 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO i.a.i.d.s.S3Destination(testSingleUpload):91 - Finished checking for normal upload mode 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO i.a.i.d.s.S3Destination(testMultipartUpload):95 - Started testing if all required credentials assigned to user for multipart upload 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to redshift-dev-s3-bucket/csv_path_dev_2/test_1665426859180 with full ID bSm_emgtQSirx1TpWOVa4bbubD9TFxmFesw8ivPNzTf922Z_lMG0oBWo_TQ_XzZUY1Oxqr9x2T_oFRqKmEIqujDT0I8Pn4ZbVD5TqY212gbqdR0Yhb8BxAn2rPN5RdrK 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 WARN a.m.s.MultiPartOutputStream(close):160 - [MultipartOutputStream for parts 1 - 10000] is already closed 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO a.m.s.StreamTransferManager(complete):367 - [Manager uploading to redshift-dev-s3-bucket/csv_path_dev_2/test_1665426859180 with id bSm_emgtQ...2rPN5RdrK]: Uploading leftover stream [Part number 1 containing 3.34 MB] 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to redshift-dev-s3-bucket/csv_path_dev_2/test_1665426859180 with id bSm_emgtQ...2rPN5RdrK]: Finished uploading [Part number 1 containing 3.34 MB] 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to redshift-dev-s3-bucket/csv_path_dev_2/test_1665426859180 with id bSm_emgtQ...2rPN5RdrK]: Completed 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO i.a.i.d.s.S3Destination(testMultipartUpload):119 - Finished verification for multipart upload mode 2022-10-10 18:34:19 INFO i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$0):62 - 2022-10-10 18:34:19 INFO i.a.i.b.IntegrationRunner(runInternal):152 - Completed integration: io.airbyte.integrations.destination.s3.S3Destination 2022-10-10 18:34:20 INFO i.a.w.p.KubePodProcess(close):737 - (pod: dev / destination-s3-check-932-0-ncuwl) - Closed all resources for pod 2022-10-10 18:34:20 INFO i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling... 2022-10-10 18:34:20 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:20 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK ----- 2022-10-10 18:34:20 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:21 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Cloud storage job log path: /workspace/932/0/logs.log 2022-10-10 18:34:21 INFO i.a.w.t.TemporalAttemptExecution(get):111 - Executing worker wrapper. Airbyte version: 0.40.6 2022-10-10 18:34:21 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):49 - Using default value for environment variable LOG_CONNECTOR_MESSAGES: 'false' 2022-10-10 18:34:21 INFO i.a.c.EnvConfigs(getEnvOrDefault):1096 - Using default value for environment variable METRIC_CLIENT: '' 2022-10-10 18:34:21 WARN i.a.m.l.MetricClientFactory(initialize):60 - Metric client is already initialized to 2022-10-10 18:34:21 INFO i.a.c.f.EnvVariableFeatureFlags(getEnvOrDefault):49 - Using default value for environment variable LOG_CONNECTOR_MESSAGES: 'false' 2022-10-10 18:34:21 INFO i.a.w.g.DefaultReplicationWorker(run):122 - start sync worker. job id: 932 attempt id: 0 2022-10-10 18:34:21 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:21 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START REPLICATION ----- 2022-10-10 18:34:21 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:21 INFO i.a.w.g.DefaultReplicationWorker(run):135 - configured sync modes: {null.databases=incremental - append} 2022-10-10 18:34:21 INFO i.a.w.i.DefaultAirbyteDestination(start):69 - Running destination... 2022-10-10 18:34:21 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = destination-s3-write-932-0-kuuwa for airbyte/destination-s3:0.3.15 with resources io.airbyte.config.ResourceRequirements@15899c33[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-10 18:34:21 INFO i.a.w.p.KubeProcessFactory(create):103 - destination-s3-write-932-0-kuuwa stdoutLocalPort = 9024 2022-10-10 18:34:21 INFO i.a.w.p.KubeProcessFactory(create):106 - destination-s3-write-932-0-kuuwa stderrLocalPort = 9025 2022-10-10 18:34:21 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):589 - Creating stdout socket server... 2022-10-10 18:34:21 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):607 - Creating stderr socket server... 2022-10-10 18:34:21 INFO i.a.w.p.KubePodProcess():520 - Creating pod destination-s3-write-932-0-kuuwa... 2022-10-10 18:34:21 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):309 - Waiting for init container to be ready before copying files... 2022-10-10 18:34:21 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):313 - Init container present.. 2022-10-10 18:34:22 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):316 - Init container ready.. 2022-10-10 18:34:22 INFO i.a.w.p.KubePodProcess():551 - Copying files... 2022-10-10 18:34:22 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: destination_config.json 2022-10-10 18:34:22 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/e14696a4-8f36-4fa3-9e10-a9842031f945/destination_config.json dev/destination-s3-write-932-0-kuuwa:/config/destination_config.json -c init 2022-10-10 18:34:22 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: destination_catalog.json 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/1859e6a1-a50e-43bd-a287-7b843b113d73/destination_catalog.json dev/destination-s3-write-932-0-kuuwa:/config/destination_catalog.json -c init 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: FINISHED_UPLOADING 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/4a681dfd-10fe-43e0-b03d-2a0cdb678a13/FINISHED_UPLOADING dev/destination-s3-write-932-0-kuuwa:/config/FINISHED_UPLOADING -c init 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:23 INFO i.a.w.p.KubePodProcess():554 - Waiting until pod is ready... 2022-10-10 18:34:25 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):598 - Setting stdout... 2022-10-10 18:34:25 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):610 - Setting stderr... 2022-10-10 18:34:26 INFO i.a.w.p.KubePodProcess():570 - Reading pod IP... 2022-10-10 18:34:26 INFO i.a.w.p.KubePodProcess():572 - Pod IP: 14.0.3.228 2022-10-10 18:34:26 INFO i.a.w.p.KubePodProcess():575 - Creating stdin socket... 2022-10-10 18:34:26 INFO i.a.w.p.KubeProcessFactory(create):100 - Attempting to start pod = source-notion-read-932-0-gzhfl for airbyte/source-notion:0.1.10 with resources io.airbyte.config.ResourceRequirements@21006df3[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=] 2022-10-10 18:34:26 INFO i.a.w.p.KubeProcessFactory(create):103 - source-notion-read-932-0-gzhfl stdoutLocalPort = 9030 2022-10-10 18:34:26 INFO i.a.w.p.KubeProcessFactory(create):106 - source-notion-read-932-0-gzhfl stderrLocalPort = 9031 2022-10-10 18:34:26 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):589 - Creating stdout socket server... 2022-10-10 18:34:26 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):607 - Creating stderr socket server... 2022-10-10 18:34:26 INFO i.a.w.p.KubePodProcess():520 - Creating pod source-notion-read-932-0-gzhfl... 2022-10-10 18:34:26 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):309 - Waiting for init container to be ready before copying files... 2022-10-10 18:34:26 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):313 - Init container present.. 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):316 - Init container ready.. 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess():551 - Copying files... 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: input_state.json 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/724384d8-5d5c-4ecb-9e4c-007fbec6f83c/input_state.json dev/source-notion-read-932-0-gzhfl:/config/input_state.json -c init 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: source_config.json 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/af4d8359-a2f5-4958-b2cb-1edc6b5a57b7/source_config.json dev/source-notion-read-932-0-gzhfl:/config/source_config.json -c init 2022-10-10 18:34:28 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: source_catalog.json 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/85293c0a-6b05-4de5-b5e8-c4e203b06436/source_catalog.json dev/source-notion-read-932-0-gzhfl:/config/source_catalog.json -c init 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):258 - Uploading file: FINISHED_UPLOADING 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):266 - kubectl cp /tmp/a2b60316-70e9-41cd-99e8-c9efdbcbe35b/FINISHED_UPLOADING dev/source-notion-read-932-0-gzhfl:/config/FINISHED_UPLOADING -c init 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):269 - Waiting for kubectl cp to complete 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):283 - kubectl cp complete, closing process 2022-10-10 18:34:29 INFO i.a.w.p.KubePodProcess():554 - Waiting until pod is ready... 2022-10-10 18:34:30 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):598 - Setting stdout... 2022-10-10 18:34:30 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):610 - Setting stderr... 2022-10-10 18:34:31 INFO i.a.w.p.KubePodProcess():570 - Reading pod IP... 2022-10-10 18:34:31 INFO i.a.w.p.KubePodProcess():572 - Pod IP: 14.0.3.229 2022-10-10 18:34:31 INFO i.a.w.p.KubePodProcess():579 - Using null stdin output stream... 2022-10-10 18:34:31 INFO i.a.w.g.DefaultReplicationWorker(run):177 - Waiting for source and destination threads to complete. 2022-10-10 18:34:31 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):435 - Destination output thread started. 2022-10-10 18:34:31 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):322 - Replication thread started. 2022-10-10 18:34:32 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):355 - Source has no more messages, closing connection. 2022-10-10 18:34:32 INFO i.a.w.p.KubePodProcess(close):737 - (pod: dev / source-notion-read-932-0-gzhfl) - Closed all resources for pod 2022-10-10 18:34:32 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):363 - Total records read: 2 (1 KB) 2022-10-10 18:34:32 INFO i.a.w.g.DefaultReplicationWorker(run):182 - One of source or destination thread complete. Waiting on the other. 2022-10-10 18:34:35 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):445 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@1365813b[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@578d8d6[type=STREAM,stream=io.airbyte.protocol.models.AirbyteStreamState@7968e24e[streamDescriptor=io.airbyte.protocol.models.StreamDescriptor@77db88d7[name=databases,namespace=,additionalProperties={}],streamState={"last_edited_time":"2022-10-10T13:11:00.000Z"},additionalProperties={}],global=,data={"databases":{"last_edited_time":"2022-10-10T13:11:00.000Z"}},additionalProperties={}],trace=,additionalProperties={}] 2022-10-10 18:34:35 INFO i.a.w.p.KubePodProcess(close):737 - (pod: dev / destination-s3-write-932-0-kuuwa) - Closed all resources for pod 2022-10-10 18:34:35 INFO i.a.w.g.DefaultReplicationWorker(run):184 - Source and destination threads complete. 2022-10-10 18:34:36 INFO i.a.w.g.DefaultReplicationWorker(run):279 - Source output at least one state message 2022-10-10 18:34:36 INFO i.a.w.g.DefaultReplicationWorker(run):285 - State capture: Updated state to: Optional[io.airbyte.config.State@654fa82c[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"databases"},"stream_state":{"last_edited_time":"2022-10-10T13:11:00.000Z"}},"data":{"databases":{"last_edited_time":"2022-10-10T13:11:00.000Z"}}}]]] 2022-10-10 18:34:36 INFO i.a.w.g.DefaultReplicationWorker(run):300 - sync summary: { "status" : "completed", "recordsSynced" : 1, "bytesSynced" : 1546, "startTime" : 1665426861307, "endTime" : 1665426876000, "totalStats" : { "recordsEmitted" : 1, "bytesEmitted" : 1546, "sourceStateMessagesEmitted" : 1, "destinationStateMessagesEmitted" : 1, "recordsCommitted" : 1, "meanSecondsBeforeSourceStateMessageEmitted" : 0, "maxSecondsBeforeSourceStateMessageEmitted" : 0, "maxSecondsBetweenStateMessageEmittedandCommitted" : 3, "meanSecondsBetweenStateMessageEmittedandCommitted" : 3 }, "streamStats" : [ { "streamName" : "databases", "stats" : { "recordsEmitted" : 1, "bytesEmitted" : 1546, "recordsCommitted" : 1 } } ] } 2022-10-10 18:34:36 INFO i.a.w.g.DefaultReplicationWorker(run):301 - failures: [ ] 2022-10-10 18:34:36 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:36 INFO i.a.w.t.TemporalAttemptExecution(get):132 - Stopping cancellation check scheduling... 2022-10-10 18:34:36 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):166 - sync summary: io.airbyte.config.StandardSyncOutput@635eeba0[standardSyncSummary=io.airbyte.config.StandardSyncSummary@77aa0d7b[status=completed,recordsSynced=1,bytesSynced=1546,startTime=1665426861307,endTime=1665426876000,totalStats=io.airbyte.config.SyncStats@6c7698a6[recordsEmitted=1,bytesEmitted=1546,sourceStateMessagesEmitted=1,destinationStateMessagesEmitted=1,recordsCommitted=1,meanSecondsBeforeSourceStateMessageEmitted=0,maxSecondsBeforeSourceStateMessageEmitted=0,maxSecondsBetweenStateMessageEmittedandCommitted=3,meanSecondsBetweenStateMessageEmittedandCommitted=3,additionalProperties={}],streamStats=[io.airbyte.config.StreamSyncStats@4edd9306[streamName=databases,stats=io.airbyte.config.SyncStats@5aca7296[recordsEmitted=1,bytesEmitted=1546,sourceStateMessagesEmitted=,destinationStateMessagesEmitted=,recordsCommitted=1,meanSecondsBeforeSourceStateMessageEmitted=,maxSecondsBeforeSourceStateMessageEmitted=,maxSecondsBetweenStateMessageEmittedandCommitted=,meanSecondsBetweenStateMessageEmittedandCommitted=,additionalProperties={}]]]],normalizationSummary=,state=io.airbyte.config.State@654fa82c[state=[{"type":"STREAM","stream":{"stream_descriptor":{"name":"databases"},"stream_state":{"last_edited_time":"2022-10-10T13:11:00.000Z"}},"data":{"databases":{"last_edited_time":"2022-10-10T13:11:00.000Z"}}}]],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@7d2c6d61[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@254779ca[stream=io.airbyte.protocol.models.AirbyteStream@7a08bc7[name=databases,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":"string"},"url":{"type":"string"},"icon":{"anyOf":[{"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"file":{"type":["null","object"],"properties":{"url":{"type":"string"},"expiry_time":{"type":"string"}},"additionalProperties":true},"type":{"enum":["file","external"]},"caption":{"type":["null","array"],"items":{"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"href":{"type":["null","string"]},"text":{"type":["null","object"],"properties":{"link":{"type":["null","object"],"properties":{"url":{"type":["null","string"]},"type":{"enum":["url"]}},"additionalProperties":true},"content":{"type":["null","string"]}},"additionalProperties":true},"type":{"type":["null","string"]},"rich_text":{"type":["null","object"],"properties":{"link":{"type":["null","object"],"properties":{"url":{"type":["null","string"]},"type":{"enum":["url"]}},"additionalProperties":true},"content":{"type":["null","string"]}},"additionalProperties":true},"plain_text":{"type":["null","string"]},"annotations":{"type":["null","object"],"properties":{"bold":{"type":["null","boolean"]},"code":{"type":["null","boolean"]},"color":{"type":["null","string"]},"italic":{"type":["null","boolean"]},"underline":{"type":["null","boolean"]},"strikethrough":{"type":["null","boolean"]}},"additionalProperties":true}}}},"external":{"type":["null","object"],"properties":{"url":{"type":"string"}},"additionalProperties":true}},"additionalProperties":true},{"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"type":{"type":"string"},"emoji":{"type":"string"}},"additionalProperties":true}],"$schema":"http://json-schema.org/draft-07/schema#"},"cover":{"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"file":{"type":["null","object"],"properties":{"url":{"type":"string"},"expiry_time":{"type":"string"}},"additionalProperties":true},"type":{"enum":["file","external"]},"caption":{"type":["null","array"],"items":{"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"href":{"type":["null","string"]},"text":{"type":["null","object"],"properties":{"link":{"type":["null","object"],"properties":{"url":{"type":["null","string"]},"type":{"enum":["url"]}},"additionalProperties":true},"content":{"type":["null","string"]}},"additionalProperties":true},"type":{"type":["null","string"]},"rich_text":{"type":["null","object"],"properties":{"link":{"type":["null","object"],"properties":{"url":{"type":["null","string"]},"type":{"enum":["url"]}},"additionalProperties":true},"content":{"type":["null","string"]}},"additionalProperties":true},"plain_text":{"type":["null","string"]},"annotations":{"type":["null","object"],"properties":{"bold":{"type":["null","boolean"]},"code":{"type":["null","boolean"]},"color":{"type":["null","string"]},"italic":{"type":["null","boolean"]},"underline":{"type":["null","boolean"]},"strikethrough":{"type":["null","boolean"]}},"additionalProperties":true}}}},"external":{"type":["null","object"],"properties":{"url":{"type":"string"}},"additionalProperties":true}},"additionalProperties":true},"title":{"type":"array","items":{"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"href":{"type":["null","string"]},"text":{"type":["null","object"],"properties":{"link":{"type":["null","object"],"properties":{"url":{"type":["null","string"]},"type":{"enum":["url"]}},"additionalProperties":true},"content":{"type":["null","string"]}},"additionalProperties":true},"type":{"type":["null","string"]},"rich_text":{"type":["null","object"],"properties":{"link":{"type":["null","object"],"properties":{"url":{"type":["null","string"]},"type":{"enum":["url"]}},"additionalProperties":true},"content":{"type":["null","string"]}},"additionalProperties":true},"plain_text":{"type":["null","string"]},"annotations":{"type":["null","object"],"properties":{"bold":{"type":["null","boolean"]},"code":{"type":["null","boolean"]},"color":{"type":["null","string"]},"italic":{"type":["null","boolean"]},"underline":{"type":["null","boolean"]},"strikethrough":{"type":["null","boolean"]}},"additionalProperties":true}}}},"object":{"enum":["database"]},"parent":{"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"type":{"enum":["database_id","page_id","workspace"]},"page_id":{"type":"string"},"workspace":{"type":"boolean"},"database_id":{"type":"string"}},"additionalProperties":true},"archived":{"type":"boolean"},"is_inline":{"type":["null","boolean"]},"created_by":{"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":"string"},"bot":{"type":["null","object"],"properties":{"owner":{"type":"object","properties":{"type":{"type":"string"},"workspace":{"type":["null","boolean"]}}}},"additionalProperties":true},"name":{"type":"string"},"type":{"enum":["person","bot"]},"object":{"enum":["user"]},"person":{"type":["null","object"],"properties":{"email":{"type":"string"}},"additionalProperties":true},"avatar_url":{"type":["null","string"]}},"additionalProperties":true},"properties":{"type":"array","items":{"type":["null","object"],"properties":{"name":{"type":["null","string"]},"value":{"type":"object","oneOf":[{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"type":{"enum":["title","rich_text","date","people","files","checkbox","url","email","phone_number","created_time","created_by","last_edited_time","last_edited_by"]}},"additionalProperties":true},{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"type":{"enum":["number"]},"format":{"enum":["number","number_with_commas","percent","dollar","canadian_dollar","euro","pound","yen","ruble","rupee","won","yuan","real","lira","rupiah","franc","hong_kong_dollar","new_zealand_dollar","krona","norwegian_krone","mexican_peso","rand","new_taiwan_dollar","danish_krone","zloty","baht","forint","koruna","shekel","chilean_peso","philippine_peso","dirham","colombian_peso","riyal","ringgit","leu"]}}},{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"type":{"enum":["select","multi_select"]},"options":{"type":"array","items":{"type":["null","object"],"$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":"string"},"name":{"type":"string"},"color":{"enum":["default","gray","brown","orange","yellow","green","blue","purple","pink","red"]}},"additionalProperties":true}}}},{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"type":{"enum":["formula"]},"expression":{"type":"string"}}},{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"type":{"enum":["relation"]},"database_id":{"type":"string"},"synced_property_id":{"type":["null","string"]},"synced_property_name":{"type":["null","string"]}}},{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"type":{"enum":["rollup"]},"function":{"enum":["count_all","count_values","count_unique_values","count_empty","count_not_empty","percent_empty","percent_not_empty","sum","average","median","min","max","range","show_original"]},"rollup_property_id":{"type":"string"},"relation_property_id":{"type":"string"},"rollup_property_name":{"type":"string"},"relation_property_name":{"type":"string"}}}],"additionalProperties":true}},"additionalProperties":true}},"created_time":{"type":"string"},"last_edited_by":{"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":"string"},"bot":{"type":["null","object"],"properties":{"owner":{"type":"object","properties":{"type":{"type":"string"},"workspace":{"type":["null","boolean"]}}}},"additionalProperties":true},"name":{"type":"string"},"type":{"enum":["person","bot"]},"object":{"enum":["user"]},"person":{"type":["null","object"],"properties":{"email":{"type":"string"}},"additionalProperties":true},"avatar_url":{"type":["null","string"]}},"additionalProperties":true},"last_edited_time":{"type":"string"}},"additionalProperties":true},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[last_edited_time],sourceDefinedPrimaryKey=[[id]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[last_edited_time],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}]],additionalProperties={}],failures=[]] 2022-10-10 18:34:36 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):312 - Stopping temporal heartbeating... 2022-10-10 18:34:36 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END REPLICATION ----- 2022-10-10 18:34:36 INFO i.a.c.i.LineGobbler(voidCall):114 - 2022-10-10 18:34:31 destination > 2022-10-10 18:34:27 INFO i.a.i.b.IntegrationRunner(runInternal):104 - Running integration: io.airbyte.integrations.destination.s3.S3Destination 2022-10-10 18:34:31 destination > 2022-10-10 18:34:27 INFO i.a.i.b.IntegrationRunner(runInternal):105 - Command: WRITE 2022-10-10 18:34:31 destination > 2022-10-10 18:34:27 INFO i.a.i.b.IntegrationRunner(runInternal):106 - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-10-10 18:34:31 destination > 2022-10-10 18:34:28 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-10 18:34:31 destination > 2022-10-10 18:34:28 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-10 18:34:31 destination > 2022-10-10 18:34:28 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-10 18:34:31 destination > 2022-10-10 18:34:28 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword requires - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-10-10 18:34:31 destination > 2022-10-10 18:34:28 INFO i.a.i.d.s.S3FormatConfigs(getS3FormatConfig):22 - S3 format config: {"format_type":"Parquet","page_size_kb":1024,"block_size_mb":128,"compression_codec":"UNCOMPRESSED","dictionary_encoding":true,"max_padding_size_mb":8,"dictionary_page_size_kb":1024} 2022-10-10 18:34:31 destination > 2022-10-10 18:34:28 INFO i.a.i.d.s.S3DestinationConfig(createS3Client):190 - Creating S3 client... 2022-10-10 18:34:31 destination > 2022-10-10 18:34:28 INFO i.a.i.d.s.SerializedBufferFactory(getCreateFunction):49 - S3 format config: S3ParquetFormatConfig{compressionCodec=UNCOMPRESSED, blockSize=134217728, maxPaddingSize=8388608, pageSize=1048576, dictionaryPageSize=1048576, dictionaryEncoding=true, } 2022-10-10 18:34:31 destination > 2022-10-10 18:34:28 INFO i.a.i.d.s.S3ConsumerFactory(lambda$toWriteConfig$0):83 - Write config: WriteConfig{streamName=databases, namespace=null, outputBucketPath=csv_path_dev_2, pathFormat=csv_path_dev_2/${NAMESPACE}/${STREAM_NAME}/${YEAR}_${MONTH}_${DAY}_${EPOCH}_, fullOutputPath=csv_path_dev_2/databases/2022_10_10_1665426868159_, syncMode=append} 2022-10-10 18:34:31 destination > 2022-10-10 18:34:29 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):116 - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started. 2022-10-10 18:34:31 destination > 2022-10-10 18:34:29 INFO i.a.i.d.s.S3ConsumerFactory(lambda$onStartFunction$1):90 - Preparing bucket in destination started for 1 streams 2022-10-10 18:34:31 destination > 2022-10-10 18:34:29 INFO i.a.i.d.s.S3ConsumerFactory(lambda$onStartFunction$1):104 - Preparing storage area in destination completed. 2022-10-10 18:34:32 source > Starting syncing SourceNotion 2022-10-10 18:34:32 source > Using start_date: 2021-03-01T00:00:00.000Z 2022-10-10 18:34:32 source > Syncing stream: databases 2022-10-10 18:34:32 source > Read 1 records from databases stream 2022-10-10 18:34:32 source > Finished syncing databases 2022-10-10 18:34:32 source > SourceNotion runtimes: Syncing stream databases 0:00:00.214583 2022-10-10 18:34:32 source > Finished syncing SourceNotion 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$addRecord$0):47 - Starting a new buffer for stream databases (current state: 0 bytes in 0 buffers) 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getAvroSchema):133 - Schema name "caption.items" contains illegal character(s) and is standardized to "caption_items" 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getAvroSchema):133 - Schema name "caption.items" contains illegal character(s) and is standardized to "caption_items" 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getAvroSchema):133 - Schema name "title.items" contains illegal character(s) and is standardized to "title_items" 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "object" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "object" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getAvroSchema):133 - Schema name "properties.items" contains illegal character(s) and is standardized to "properties_items" 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "format" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getAvroSchema):133 - Schema name "options.items" contains illegal character(s) and is standardized to "options_items" 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "color" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "function" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "type" has no type specification. It will default to string 2022-10-10 18:34:32 destination > 2022-10-10 18:34:32 WARN i.a.i.d.s.a.JsonToAvroSchemaConverter(getTypes):69 - Field "object" has no type specification. It will default to string 2022-10-10 18:34:33 destination > 2022-10-10 18:34:33 WARN o.a.h.u.NativeCodeLoader():60 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):62 - Airbyte message consumer: succeeded. 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO i.a.i.d.b.BufferedStreamConsumer(close):171 - executing on success close procedure. 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO i.a.i.d.r.SerializedBufferingStrategy(flushAll):100 - Flushing all 1 current buffers (1 KB in total) 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO i.a.i.d.r.SerializedBufferingStrategy(flushAll):102 - Flushing buffer of stream databases (1 KB) 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO i.a.i.d.s.S3ConsumerFactory(lambda$flushBufferFunction$2):121 - Flushing buffer for stream databases (1 KB) to storage 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO i.a.i.d.s.p.ParquetSerializedBuffer(flush):106 - Finished writing data to 89dfb044-d4e7-4637-90f8-452efe0db0c216330884859604633514.parquet (1 KB) 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to redshift-dev-s3-bucket/csv_path_dev_2/databases/2022_10_10_1665426868159_0.parquet with full ID Puz0l1Wk9CFZwzg1vVq34QH_soWPu8FL6SoI9zwB5uJNJZPUwBNJ0gdobHYqkHTIyWsrdbCIf.VDsmEvBA_quoa1zJzFtfyGguaQ7osBPATuSEAfKyjEnvSyZGLTvXDp 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-10-10 18:34:34 destination > 2022-10-10 18:34:34 INFO a.m.s.StreamTransferManager(complete):367 - [Manager uploading to redshift-dev-s3-bucket/csv_path_dev_2/databases/2022_10_10_1665426868159_0.parquet with id Puz0l1Wk9...yZGLTvXDp]: Uploading leftover stream [Part number 1 containing 0.05 MB] 2022-10-10 18:34:35 destination > 2022-10-10 18:34:35 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to redshift-dev-s3-bucket/csv_path_dev_2/databases/2022_10_10_1665426868159_0.parquet with id Puz0l1Wk9...yZGLTvXDp]: Finished uploading [Part number 1 containing 0.05 MB] 2022-10-10 18:34:35 destination > 2022-10-10 18:34:35 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to redshift-dev-s3-bucket/csv_path_dev_2/databases/2022_10_10_1665426868159_0.parquet with id Puz0l1Wk9...yZGLTvXDp]: Completed 2022-10-10 18:34:35 destination > 2022-10-10 18:34:35 INFO i.a.i.d.s.S3StorageOperations(loadDataIntoBucket):198 - Uploaded buffer file to storage: 89dfb044-d4e7-4637-90f8-452efe0db0c216330884859604633514.parquet -> csv_path_dev_2/databases/2022_10_10_1665426868159_0.parquet (filename: 2022_10_10_1665426868159_0.parquet) 2022-10-10 18:34:35 destination > 2022-10-10 18:34:35 INFO i.a.i.d.r.SerializedBufferingStrategy(close):122 - Closing buffer for stream databases 2022-10-10 18:34:35 destination > 2022-10-10 18:34:35 INFO i.a.i.b.IntegrationRunner(runInternal):152 - Completed integration: io.airbyte.integrations.destination.s3.S3Destination