2022-08-17 16:20:53 INFO i.a.w.w.WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.36.0-alpha 2022-08-17 16:20:53 INFO i.a.w.t.TemporalAttemptExecution(get):107 - Cloud storage job log path: /workspace/2919/0/logs.log 2022-08-17 16:21:28 destination > SLF4J: Class path contains multiple SLF4J bindings. 2022-08-17 16:20:53 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.36.0-alpha 2022-08-17 16:20:53 INFO i.a.w.DefaultReplicationWorker(run):104 - start sync worker. job id: 2919 attempt id: 0 2022-08-17 16:20:53 INFO i.a.w.DefaultReplicationWorker(run):116 - configured sync modes: {null.issues=incremental - append_dedup, null.projects=full_refresh - append} 2022-08-17 16:20:53 INFO i.a.w.p.a.DefaultAirbyteDestination(start):69 - Running destination... 2022-08-17 16:20:53 INFO i.a.w.p.KubeProcessFactory(create):113 - Attempting to start pod = destination-redshift-sync-2919-0-cdvro for airbyte/destination-redshift:0.3.29 2022-08-17 16:20:53 INFO i.a.w.p.KubeProcessFactory(create):116 - destination-redshift-sync-2919-0-cdvro stdoutLocalPort = 9002 2022-08-17 16:20:53 INFO i.a.w.p.KubeProcessFactory(create):119 - destination-redshift-sync-2919-0-cdvro stderrLocalPort = 9003 2022-08-17 16:20:53 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):583 - Creating stdout socket server... 2022-08-17 16:20:53 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):601 - Creating stderr socket server... 2022-08-17 16:20:53 INFO i.a.w.p.KubePodProcess():521 - Creating pod destination-redshift-sync-2919-0-cdvro... 2022-08-17 16:21:23 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):305 - Waiting for init container to be ready before copying files... 2022-08-17 16:21:23 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):318 - Init container present.. 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):321 - Init container ready.. 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess():546 - Copying files... 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: destination_config.json 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/a90b40ec-e718-4857-b523-3c28d2021573/destination_config.json airbyte/destination-redshift-sync-2919-0-cdvro:/config/destination_config.json -c init 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: destination_catalog.json 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/d1e47882-d312-4afb-bc79-b6fcdff2a7fb/destination_catalog.json airbyte/destination-redshift-sync-2919-0-cdvro:/config/destination_catalog.json -c init 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: FINISHED_UPLOADING 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/84010648-55d2-4d4a-9f6e-d14a758ec5a9/FINISHED_UPLOADING airbyte/destination-redshift-sync-2919-0-cdvro:/config/FINISHED_UPLOADING -c init 2022-08-17 16:21:25 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:21:26 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:21:26 INFO i.a.w.p.KubePodProcess():549 - Waiting until pod is ready... 2022-08-17 16:21:27 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):592 - Setting stdout... 2022-08-17 16:21:27 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):604 - Setting stderr... 2022-08-17 16:21:28 INFO i.a.w.p.KubePodProcess():564 - Reading pod IP... 2022-08-17 16:21:28 INFO i.a.w.p.KubePodProcess():566 - Pod IP: 100.65.99.211 2022-08-17 16:21:28 INFO i.a.w.p.KubePodProcess():569 - Creating stdin socket... 2022-08-17 16:21:28 INFO i.a.w.p.KubeProcessFactory(create):113 - Attempting to start pod = source-jira-sync-2919-0-nfavr for airbyte/source-jira:0.2.20 2022-08-17 16:21:28 INFO i.a.w.p.KubeProcessFactory(create):116 - source-jira-sync-2919-0-nfavr stdoutLocalPort = 9022 2022-08-17 16:21:28 INFO i.a.w.p.KubeProcessFactory(create):119 - source-jira-sync-2919-0-nfavr stderrLocalPort = 9023 2022-08-17 16:21:28 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):583 - Creating stdout socket server... 2022-08-17 16:21:28 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):601 - Creating stderr socket server... 2022-08-17 16:21:28 INFO i.a.w.p.KubePodProcess():521 - Creating pod source-jira-sync-2919-0-nfavr... 2022-08-17 16:22:26 INFO i.a.w.t.TemporalAttemptExecution(get):107 - Cloud storage job log path: /workspace/2919/0/logs.log 2022-08-17 16:21:28 destination > SLF4J: Found binding in [jar:file:/airbyte/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2022-08-17 16:21:28 destination > SLF4J: Found binding in [jar:file:/airbyte/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] 2022-08-17 16:21:28 destination > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2022-08-17 16:21:28 destination > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 2022-08-17 16:22:03 destination > 2022-08-17 16:21:31 INFO i.a.i.d.r.RedshiftDestination(main):77 - starting destination: class io.airbyte.integrations.destination.redshift.RedshiftDestination 2022-08-17 16:22:03 destination > 2022-08-17 16:21:31 INFO i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json} 2022-08-17 16:22:03 destination > 2022-08-17 16:21:31 INFO i.a.i.b.IntegrationRunner(runInternal):121 - Running integration: io.airbyte.integrations.destination.redshift.RedshiftDestination 2022-08-17 16:22:03 source > Starting syncing SourceJira 2022-08-17 16:22:03 destination > 2022-08-17 16:21:31 INFO i.a.i.b.IntegrationRunner(runInternal):122 - Command: WRITE 2022-08-17 16:22:03 destination > 2022-08-17 16:21:31 INFO i.a.i.b.IntegrationRunner(runInternal):123 - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'} 2022-08-17 16:22:03 destination > 2022-08-17 16:21:32 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-17 16:22:03 destination > 2022-08-17 16:21:32 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword 2022-08-17 16:22:03 source > Syncing stream: issues 2022-08-17 16:22:03 destination > 2022-08-17 16:21:32 INFO i.a.i.d.j.c.SwitchingDestination(getConsumer):65 - Using destination type: COPY_S3 2022-08-17 16:22:03 destination > 2022-08-17 16:21:32 INFO i.a.i.d.s.S3DestinationConfig(createS3Client):169 - Creating S3 client... 2022-08-17 16:22:03 destination > 2022-08-17 16:21:34 INFO i.a.i.d.b.BufferedStreamConsumer(startTracked):116 - class io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer started. 2022-08-17 16:22:06 source > Read 2 records from issues stream 2022-08-17 16:22:06 source > Finished syncing issues 2022-08-17 16:22:06 source > SourceJira runtimes: Syncing stream issues 0:00:03.031365 2022-08-17 16:22:06 source > Syncing stream: projects 2022-08-17 16:22:06 source > Read 13 records from projects stream 2022-08-17 16:22:06 source > Finished syncing projects 2022-08-17 16:22:06 source > SourceJira runtimes: Syncing stream issues 0:00:03.031365 Syncing stream projects 0:00:00.024164 2022-08-17 16:22:06 source > Finished syncing SourceJira 2022-08-17 16:22:07 destination > 2022-08-17 16:22:07 INFO i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):65 - Airbyte message consumer: succeeded. 2022-08-17 16:22:07 destination > 2022-08-17 16:22:07 INFO i.a.i.d.b.BufferedStreamConsumer(close):170 - executing on success close procedure. 2022-08-17 16:22:07 destination > 2022-08-17 16:22:07 INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(lambda$flushAll$1):84 - Flushing jira_info_issues: 2 records (138 KB) 2022-08-17 16:22:07 destination > 2022-08-17 16:22:07 INFO i.a.i.d.j.c.s.S3StreamCopier(prepareStagingFile):95 - S3 upload part size: 10 MB 2022-08-17 16:22:07 destination > 2022-08-17 16:22:07 INFO i.a.i.d.s.c.S3CsvWriter():58 - Full S3 path for stream 'jira_info_issues': s3://spark-prd-etl/google/analytics/jira_info_issues/2022_08_17_1660753294413_fe570db5-5f17-4313-8e26-4bdcbcd975ec.csv 2022-08-17 16:22:07 destination > 2022-08-17 16:22:07 INFO i.a.i.d.s.u.StreamTransferManagerHelper(getDefault):55 - PartSize arg is set to 10 MB 2022-08-17 16:22:08 destination > 2022-08-17 16:22:08 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to spark-prd-etl/google/analytics/jira_info_issues/2022_08_17_1660753294413_fe570db5-5f17-4313-8e26-4bdcbcd975ec.csv with full ID MnyzRHvMnNRmthh9PkB1j9SXY.KKegFyyld5jd.9Rl7n.qvYO_PQBsMF6AKkygBxhsTf3eU02tSdVRc.ruJaqQtt01mqrFpU.IwtWMdNGA59fTyRt895awzFcoJX2rGE 2022-08-17 16:22:08 destination > 2022-08-17 16:22:08 INFO i.a.i.d.r.InMemoryRecordBufferingStrategy(lambda$flushAll$1):84 - Flushing jira_info_projects: 13 records (138 KB) 2022-08-17 16:22:08 destination > 2022-08-17 16:22:08 INFO i.a.i.d.j.c.s.S3StreamCopier(prepareStagingFile):95 - S3 upload part size: 10 MB 2022-08-17 16:22:08 destination > 2022-08-17 16:22:08 INFO i.a.i.d.s.c.S3CsvWriter():58 - Full S3 path for stream 'jira_info_projects': s3://spark-prd-etl/google/analytics/jira_info_projects/2022_08_17_1660753294414_8b304213-d803-4bfe-871c-2d080d4bcbae.csv 2022-08-17 16:22:08 destination > 2022-08-17 16:22:08 INFO i.a.i.d.s.u.StreamTransferManagerHelper(getDefault):55 - PartSize arg is set to 10 MB 2022-08-17 16:22:08 destination > 2022-08-17 16:22:08 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to spark-prd-etl/google/analytics/jira_info_projects/2022_08_17_1660753294414_8b304213-d803-4bfe-871c-2d080d4bcbae.csv with full ID lygyewZYNF.DpqATUKccgPGyJbpgdkNBe_fC8rd3p3WwMsRLdQBJ_w8qgREOW5nUDG3NHN.PryU8WU7W.dUPrSt8wjS22nhwfpX1KqrXhE13.jBEhM8KzDI5ZgUabydZ 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 INFO i.a.i.d.s.w.BaseS3Writer(close):113 - Uploading remaining data for stream 'jira_info_issues'. 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 WARN a.m.s.MultiPartOutputStream(close):160 - [MultipartOutputStream for parts 1 - 10000] is already closed 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 INFO a.m.s.StreamTransferManager(complete):367 - [Manager uploading to spark-prd-etl/google/analytics/jira_info_issues/2022_08_17_1660753294413_fe570db5-5f17-4313-8e26-4bdcbcd975ec.csv with id MnyzRHvMn...FcoJX2rGE]: Uploading leftover stream [Part number 1 containing 0.02 MB] 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to spark-prd-etl/google/analytics/jira_info_issues/2022_08_17_1660753294413_fe570db5-5f17-4313-8e26-4bdcbcd975ec.csv with id MnyzRHvMn...FcoJX2rGE]: Finished uploading [Part number 1 containing 0.02 MB] 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to spark-prd-etl/google/analytics/jira_info_issues/2022_08_17_1660753294413_fe570db5-5f17-4313-8e26-4bdcbcd975ec.csv with id MnyzRHvMn...FcoJX2rGE]: Completed 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 INFO i.a.i.d.s.w.BaseS3Writer(close):115 - Upload completed for stream 'jira_info_issues'. 2022-08-17 16:22:09 destination > 2022-08-17 16:22:09 INFO i.a.i.d.j.c.s.S3StreamCopier(createDestinationSchema):152 - Creating schema in destination if it doesn't exist: wine_stg 2022-08-17 16:22:11 destination > 2022-08-17 16:22:11 INFO i.a.i.d.j.c.s.S3StreamCopier(createTemporaryTable):158 - Preparing tmp table in destination for stream: jira_info_issues, schema: wine_stg, tmp table name: _airbyte_tmp_dyv_jira_info_issues. 2022-08-17 16:22:12 destination > 2022-08-17 16:22:12 INFO i.a.i.d.r.RedshiftStreamCopier(copyStagingFileToTemporaryTable):85 - Starting copy to tmp table: _airbyte_tmp_dyv_jira_info_issues in destination for stream: jira_info_issues, schema: wine_stg, . 2022-08-17 16:22:15 destination > 2022-08-17 16:22:15 INFO i.a.i.d.r.RedshiftStreamCopier(copyStagingFileToTemporaryTable):89 - Copy to tmp table _airbyte_tmp_dyv_jira_info_issues in destination for stream jira_info_issues complete. 2022-08-17 16:22:15 destination > 2022-08-17 16:22:15 INFO i.a.i.d.j.c.s.S3StreamCopier(createDestinationTable):174 - Preparing table _airbyte_raw_jira_info_issues in destination. 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO i.a.i.d.j.c.s.S3StreamCopier(createDestinationTable):176 - Table _airbyte_tmp_dyv_jira_info_issues in destination prepared. 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO i.a.i.d.j.c.s.S3StreamCopier(generateMergeStatement):183 - Preparing to merge tmp table _airbyte_tmp_dyv_jira_info_issues to dest table: _airbyte_raw_jira_info_issues, schema: wine_stg, in destination. 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO i.a.i.d.s.w.BaseS3Writer(close):113 - Uploading remaining data for stream 'jira_info_projects'. 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 WARN a.m.s.MultiPartOutputStream(close):160 - [MultipartOutputStream for parts 1 - 10000] is already closed 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO a.m.s.StreamTransferManager(complete):367 - [Manager uploading to spark-prd-etl/google/analytics/jira_info_projects/2022_08_17_1660753294414_8b304213-d803-4bfe-871c-2d080d4bcbae.csv with id lygyewZYN...5ZgUabydZ]: Uploading leftover stream [Part number 1 containing 0.01 MB] 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to spark-prd-etl/google/analytics/jira_info_projects/2022_08_17_1660753294414_8b304213-d803-4bfe-871c-2d080d4bcbae.csv with id lygyewZYN...5ZgUabydZ]: Finished uploading [Part number 1 containing 0.01 MB] 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to spark-prd-etl/google/analytics/jira_info_projects/2022_08_17_1660753294414_8b304213-d803-4bfe-871c-2d080d4bcbae.csv with id lygyewZYN...5ZgUabydZ]: Completed 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO i.a.i.d.s.w.BaseS3Writer(close):115 - Upload completed for stream 'jira_info_projects'. 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO i.a.i.d.j.c.s.S3StreamCopier(createDestinationSchema):152 - Creating schema in destination if it doesn't exist: wine_stg 2022-08-17 16:22:17 destination > 2022-08-17 16:22:17 INFO i.a.i.d.j.c.s.S3StreamCopier(createTemporaryTable):158 - Preparing tmp table in destination for stream: jira_info_projects, schema: wine_stg, tmp table name: _airbyte_tmp_gky_jira_info_projects. 2022-08-17 16:22:18 destination > 2022-08-17 16:22:18 INFO i.a.i.d.r.RedshiftStreamCopier(copyStagingFileToTemporaryTable):85 - Starting copy to tmp table: _airbyte_tmp_gky_jira_info_projects in destination for stream: jira_info_projects, schema: wine_stg, . 2022-08-17 16:22:19 destination > 2022-08-17 16:22:19 INFO i.a.i.d.r.RedshiftStreamCopier(copyStagingFileToTemporaryTable):89 - Copy to tmp table _airbyte_tmp_gky_jira_info_projects in destination for stream jira_info_projects complete. 2022-08-17 16:22:19 destination > 2022-08-17 16:22:19 INFO i.a.i.d.j.c.s.S3StreamCopier(createDestinationTable):174 - Preparing table _airbyte_raw_jira_info_projects in destination. 2022-08-17 16:22:20 destination > 2022-08-17 16:22:20 INFO i.a.i.d.j.c.s.S3StreamCopier(createDestinationTable):176 - Table _airbyte_tmp_gky_jira_info_projects in destination prepared. 2022-08-17 16:22:20 destination > 2022-08-17 16:22:20 INFO i.a.i.d.j.c.s.S3StreamCopier(generateMergeStatement):183 - Preparing to merge tmp table _airbyte_tmp_gky_jira_info_projects to dest table: _airbyte_raw_jira_info_projects, schema: wine_stg, in destination. 2022-08-17 16:22:23 destination > 2022-08-17 16:22:23 INFO i.a.i.d.j.c.s.S3StreamCopier(removeFileAndDropTmpTable):198 - S3 staging file google/analytics/jira_info_issues/2022_08_17_1660753294413_fe570db5-5f17-4313-8e26-4bdcbcd975ec.csv cleaned. 2022-08-17 16:22:23 destination > 2022-08-17 16:22:23 INFO i.a.i.d.j.c.s.S3StreamCopier(removeFileAndDropTmpTable):202 - Begin cleaning _airbyte_tmp_dyv_jira_info_issues tmp table in destination. 2022-08-17 16:22:23 destination > 2022-08-17 16:22:23 INFO i.a.i.d.j.c.s.S3StreamCopier(removeFileAndDropTmpTable):204 - _airbyte_tmp_dyv_jira_info_issues tmp table in destination cleaned. 2022-08-17 16:22:23 destination > 2022-08-17 16:22:23 INFO i.a.i.d.r.RedshiftStreamCopier(removeFileAndDropTmpTable):105 - Begin cleaning s3 manifest file google/analytics//0f124367-6ed6-43e2-89de-fbca4619beb7/wine_stg/cd77c11a-00ed-4f3d-b792-f7be184cb21e.manifest. 2022-08-17 16:22:23 destination > 2022-08-17 16:22:23 INFO i.a.i.d.r.RedshiftStreamCopier(removeFileAndDropTmpTable):109 - S3 manifest file google/analytics//0f124367-6ed6-43e2-89de-fbca4619beb7/wine_stg/cd77c11a-00ed-4f3d-b792-f7be184cb21e.manifest cleaned. 2022-08-17 16:22:24 destination > 2022-08-17 16:22:24 INFO i.a.i.d.j.c.s.S3StreamCopier(removeFileAndDropTmpTable):198 - S3 staging file google/analytics/jira_info_projects/2022_08_17_1660753294414_8b304213-d803-4bfe-871c-2d080d4bcbae.csv cleaned. 2022-08-17 16:22:24 destination > 2022-08-17 16:22:24 INFO i.a.i.d.j.c.s.S3StreamCopier(removeFileAndDropTmpTable):202 - Begin cleaning _airbyte_tmp_gky_jira_info_projects tmp table in destination. 2022-08-17 16:22:25 destination > 2022-08-17 16:22:25 INFO i.a.i.d.j.c.s.S3StreamCopier(removeFileAndDropTmpTable):204 - _airbyte_tmp_gky_jira_info_projects tmp table in destination cleaned. 2022-08-17 16:22:25 destination > 2022-08-17 16:22:25 INFO i.a.i.d.r.RedshiftStreamCopier(removeFileAndDropTmpTable):105 - Begin cleaning s3 manifest file google/analytics//0f124367-6ed6-43e2-89de-fbca4619beb7/wine_stg/fca6b80a-128e-4f24-82dc-b8eb8dcf3ea5.manifest. 2022-08-17 16:22:25 destination > 2022-08-17 16:22:25 INFO i.a.i.d.r.RedshiftStreamCopier(removeFileAndDropTmpTable):109 - S3 manifest file google/analytics//0f124367-6ed6-43e2-89de-fbca4619beb7/wine_stg/fca6b80a-128e-4f24-82dc-b8eb8dcf3ea5.manifest cleaned. 2022-08-17 16:22:25 destination > 2022-08-17 16:22:25 INFO i.a.i.b.IntegrationRunner(runInternal):169 - Completed integration: io.airbyte.integrations.destination.redshift.RedshiftDestination 2022-08-17 16:22:25 destination > 2022-08-17 16:22:25 INFO i.a.i.d.r.RedshiftDestination(main):79 - completed destination: class io.airbyte.integrations.destination.redshift.RedshiftDestination 2022-08-17 16:21:58 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):305 - Waiting for init container to be ready before copying files... 2022-08-17 16:21:58 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):318 - Init container present.. 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):321 - Init container ready.. 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess():546 - Copying files... 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: input_state.json 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/a5bab78b-40b0-4087-b4ce-ea4162880a04/input_state.json airbyte/source-jira-sync-2919-0-nfavr:/config/input_state.json -c init 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: source_config.json 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/891daf22-e31f-48fd-ad1f-92bd722505d7/source_config.json airbyte/source-jira-sync-2919-0-nfavr:/config/source_config.json -c init 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: source_catalog.json 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/3c3ac094-b38f-4754-9033-060effc44ed7/source_catalog.json airbyte/source-jira-sync-2919-0-nfavr:/config/source_catalog.json -c init 2022-08-17 16:22:00 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:22:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:22:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: FINISHED_UPLOADING 2022-08-17 16:22:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/b5726a05-0292-47b3-babb-9631cd8b17b4/FINISHED_UPLOADING airbyte/source-jira-sync-2919-0-nfavr:/config/FINISHED_UPLOADING -c init 2022-08-17 16:22:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:22:01 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:22:01 INFO i.a.w.p.KubePodProcess():549 - Waiting until pod is ready... 2022-08-17 16:22:02 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):592 - Setting stdout... 2022-08-17 16:22:02 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):604 - Setting stderr... 2022-08-17 16:22:03 INFO i.a.w.p.KubePodProcess():564 - Reading pod IP... 2022-08-17 16:22:03 INFO i.a.w.p.KubePodProcess():566 - Pod IP: 100.65.100.34 2022-08-17 16:22:03 INFO i.a.w.p.KubePodProcess():573 - Using null stdin output stream... 2022-08-17 16:22:03 INFO i.a.w.DefaultReplicationWorker(run):158 - Waiting for source and destination threads to complete. 2022-08-17 16:22:03 INFO i.a.w.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$6):339 - Destination output thread started. 2022-08-17 16:22:03 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):279 - Replication thread started. 2022-08-17 16:22:07 INFO i.a.w.p.KubePodProcess(close):713 - (pod: airbyte / source-jira-sync-2919-0-nfavr) - Closed all resources for pod 2022-08-17 16:22:07 INFO i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):305 - Total records read: 16 (28 KB) 2022-08-17 16:22:07 INFO i.a.w.DefaultReplicationWorker(run):163 - One of source or destination thread complete. Waiting on the other. 2022-08-17 16:22:25 INFO i.a.w.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$6):349 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@225d8b53[type=STATE,log=,spec=,connectionStatus=,catalog=,record=,state=io.airbyte.protocol.models.AirbyteStateMessage@401d8ad9[data={"issues":{"updated":"2022-08-17T13:16:11.489000-03:00"}},additionalProperties={}],additionalProperties={}] 2022-08-17 16:22:26 INFO i.a.w.p.KubePodProcess(close):713 - (pod: airbyte / destination-redshift-sync-2919-0-cdvro) - Closed all resources for pod 2022-08-17 16:22:26 INFO i.a.w.DefaultReplicationWorker(run):165 - Source and destination threads complete. 2022-08-17 16:22:26 INFO i.a.w.DefaultReplicationWorker(run):228 - sync summary: io.airbyte.config.ReplicationAttemptSummary@4e381f8b[status=completed,recordsSynced=15,bytesSynced=29188,startTime=1660753253364,endTime=1660753346001,totalStats=io.airbyte.config.SyncStats@16f16831[recordsEmitted=15,bytesEmitted=29188,stateMessagesEmitted=1,recordsCommitted=15],streamStats=[io.airbyte.config.StreamSyncStats@3336f0c1[streamName=jira_info_projects,stats=io.airbyte.config.SyncStats@54c4bc8c[recordsEmitted=13,bytesEmitted=10060,stateMessagesEmitted=,recordsCommitted=13]], io.airbyte.config.StreamSyncStats@5a0e35d8[streamName=jira_info_issues,stats=io.airbyte.config.SyncStats@47a36e48[recordsEmitted=2,bytesEmitted=19128,stateMessagesEmitted=,recordsCommitted=2]]]] 2022-08-17 16:22:26 INFO i.a.w.DefaultReplicationWorker(run):248 - Source output at least one state message 2022-08-17 16:22:26 INFO i.a.w.DefaultReplicationWorker(run):254 - State capture: Updated state to: Optional[io.airbyte.config.State@3fd918a6[state={"issues":{"updated":"2022-08-17T13:16:11.489000-03:00"}}]] 2022-08-17 16:22:26 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-17 16:22:26 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$1):147 - sync summary: io.airbyte.config.StandardSyncOutput@4edbf29c[standardSyncSummary=io.airbyte.config.StandardSyncSummary@3daf451d[status=completed,recordsSynced=15,bytesSynced=29188,startTime=1660753253364,endTime=1660753346001,totalStats=io.airbyte.config.SyncStats@16f16831[recordsEmitted=15,bytesEmitted=29188,stateMessagesEmitted=1,recordsCommitted=15],streamStats=[io.airbyte.config.StreamSyncStats@3336f0c1[streamName=jira_info_projects,stats=io.airbyte.config.SyncStats@54c4bc8c[recordsEmitted=13,bytesEmitted=10060,stateMessagesEmitted=,recordsCommitted=13]], io.airbyte.config.StreamSyncStats@5a0e35d8[streamName=jira_info_issues,stats=io.airbyte.config.SyncStats@47a36e48[recordsEmitted=2,bytesEmitted=19128,stateMessagesEmitted=,recordsCommitted=2]]]],normalizationSummary=,state=io.airbyte.config.State@3fd918a6[state={"issues":{"updated":"2022-08-17T13:16:11.489000-03:00"}}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@68ca70a7[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@6fc61f31[stream=io.airbyte.protocol.models.AirbyteStream@7e185f47[name=jira_info_issues,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":"string","readOnly":true,"description":"The ID of the issue."},"key":{"type":"string","readOnly":true,"description":"The key of the issue."},"self":{"type":"string","format":"uri","readOnly":true,"description":"The URL of the issue details."},"names":{"type":"object","readOnly":true,"description":"The ID and name of each field present on the issue.","additionalProperties":{"type":"string","readOnly":true}},"expand":{"xml":{"attribute":true},"type":"string","readOnly":true,"description":"Expand options that include additional issue details in the response."},"fields":{"type":"object","properties":{"created":{"type":["string","null"],"format":"date-time","description":"This field is not shown in schema / swagger, but exists in records and we use it as cursor fiekd. Updated may be absent. Added to solve the #4341"},"updated":{"type":["string","null"],"format":"date-time","description":"This field is not shown in schema / swagger, but exists in records and we use it as cursor fiekd. Updated may be absent. Added to solve the #4341"}},"additionalProperties":{}},"schema":{"type":"object","readOnly":true,"description":"The schema describing each field present on the issue."},"editmeta":{"readOnly":true,"description":"The metadata for the fields on the issue that can be amended."},"changelog":{"readOnly":true,"description":"Details of changelogs associated with the issue."},"projectId":{"type":"string","readOnly":true,"description":"The ID of the project containing the issue."},"operations":{"readOnly":true,"description":"The operations that can be performed on the issue."},"projectKey":{"type":"string","readOnly":true,"description":"The key of the project containing the issue."},"properties":{"type":"object","readOnly":true,"description":"Details of the issue properties identified in the request.","additionalProperties":{"readOnly":true}},"transitions":{"type":"array","readOnly":true,"description":"The transitions that can be performed on the issue."},"renderedFields":{"type":"object","readOnly":true,"description":"The rendered value of each field present on the issue.","additionalProperties":{"readOnly":true}},"fieldsToInclude":{"type":"object"},"versionedRepresentations":{"type":"object","readOnly":true,"description":"The versions of each field on the issue.","additionalProperties":{"type":"object","readOnly":true,"additionalProperties":{"readOnly":true}}}},"additionalProperties":false},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[fields, updated],sourceDefinedPrimaryKey=[[id]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[fields, updated],destinationSyncMode=append_dedup,primaryKey=[[id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@39d87a34[stream=io.airbyte.protocol.models.AirbyteStream@22c7ae0[name=jira_info_projects,jsonSchema={"type":"object","$schema":"http://json-schema.org/draft-07/schema#","properties":{"id":{"type":"string","description":"The ID of the project."},"key":{"type":"string","readOnly":true,"description":"The key of the project."},"url":{"type":"string","readOnly":true,"description":"A link to information about this project, such as project documentation."},"lead":{"readOnly":true,"description":"The username of the project lead."},"name":{"type":"string","readOnly":true,"description":"The name of the project."},"self":{"type":"string","format":"uri","readOnly":true,"description":"The URL of the project details."},"uuid":{"type":"string","format":"uuid","readOnly":true,"description":"Unique ID for next-gen projects."},"email":{"type":"string","description":"An email address associated with the project."},"roles":{"type":"object","readOnly":true,"description":"The name and self URL for each role defined in the project. For more information, see [Create project role](#api-rest-api-3-role-post).","additionalProperties":{"type":"string","format":"uri","readOnly":true}},"style":{"enum":["classic","next-gen"],"type":"string","readOnly":true,"description":"The type of the project."},"expand":{"xml":{"attribute":true},"type":"string","readOnly":true,"description":"Expand options that include additional project details in the response."},"deleted":{"type":"boolean","readOnly":true,"description":"Whether the project is marked as deleted."},"insight":{"readOnly":true,"description":"Insights about the project."},"archived":{"type":"boolean","readOnly":true,"description":"Whether the project is archived."},"versions":{"type":"array","readOnly":true,"description":"The versions defined in the project. For more information, see [Create version](#api-rest-api-3-version-post)."},"deletedBy":{"readOnly":true,"description":"The user who marked the project as deleted."},"favourite":{"type":"boolean","description":"Whether the project is selected as a favorite."},"isPrivate":{"type":"boolean","readOnly":true,"description":"Whether the project is private."},"archivedBy":{"readOnly":true,"description":"The user who archived the project."},"avatarUrls":{"readOnly":true,"description":"The URLs of the project's avatars."},"components":{"type":"array","readOnly":true,"description":"List of the components contained in the project."},"issueTypes":{"type":"array","readOnly":true,"description":"List of the issue types available in the project."},"properties":{"type":"object","readOnly":true,"description":"Map of project properties","additionalProperties":{"readOnly":true}},"simplified":{"type":"boolean","readOnly":true,"description":"Whether the project is simplified."},"deletedDate":{"type":"string","format":"date-time","readOnly":true,"description":"The date when the project was marked as deleted."},"description":{"type":"string","readOnly":true,"description":"A brief description of the project."},"permissions":{"readOnly":true,"description":"User permissions on the project"},"archivedDate":{"type":"string","format":"date-time","readOnly":true,"description":"The date when the project was archived."},"assigneeType":{"enum":["PROJECT_LEAD","UNASSIGNED"],"type":"string","readOnly":true,"description":"The default assignee when creating issues for this project."},"projectTypeKey":{"enum":["software","service_desk","business"],"type":"string","readOnly":true,"description":"The [project type](https://confluence.atlassian.com/x/GwiiLQ#Jiraapplicationsoverview-Productfeaturesandprojecttypes) of the project."},"projectCategory":{"readOnly":true,"description":"The category the project belongs to."},"retentionTillDate":{"type":"string","format":"date-time","readOnly":true,"description":"The date when the project is deleted permanently."},"issueTypeHierarchy":{"readOnly":true,"description":"The issue type hierarchy for the project"}},"description":"Details about a project.","additionalProperties":false},supportedSyncModes=[full_refresh],sourceDefinedCursor=,defaultCursorField=[],sourceDefinedPrimaryKey=[[id]],namespace=,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=append,primaryKey=[[id]],additionalProperties={}]],additionalProperties={}],failures=[]] 2022-08-17 16:22:26 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):235 - Stopping temporal heartbeating... 2022-08-17 16:23:00 normalization > Traceback (most recent call last): 2022-08-17 16:23:00 normalization > Running: transform-config --config destination_config.json --integration-type redshift --out /config 2022-08-17 16:23:00 normalization > Namespace(config='destination_config.json', integration_type=, out='/config') 2022-08-17 16:23:00 normalization > File "/usr/local/bin/transform-catalog", line 8, in 2022-08-17 16:22:26 INFO i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.36.0-alpha 2022-08-17 16:22:26 INFO i.a.w.DefaultNormalizationWorker(run):47 - Running normalization. 2022-08-17 16:22:26 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):122 - Running with normalization version: airbyte/normalization:0.1.75 2022-08-17 16:22:26 INFO i.a.w.p.KubeProcessFactory(create):113 - Attempting to start pod = normalization-sync-2919-0-pzvdu for airbyte/normalization:0.1.75 2022-08-17 16:22:26 INFO i.a.w.p.KubeProcessFactory(create):116 - normalization-sync-2919-0-pzvdu stdoutLocalPort = 9030 2022-08-17 16:22:26 INFO i.a.w.p.KubeProcessFactory(create):119 - normalization-sync-2919-0-pzvdu stderrLocalPort = 9031 2022-08-17 16:22:26 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):583 - Creating stdout socket server... 2022-08-17 16:22:26 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):601 - Creating stderr socket server... 2022-08-17 16:22:26 INFO i.a.w.p.KubePodProcess():521 - Creating pod normalization-sync-2919-0-pzvdu... 2022-08-17 16:22:56 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):305 - Waiting for init container to be ready before copying files... 2022-08-17 16:22:56 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):318 - Init container present.. 2022-08-17 16:22:57 INFO i.a.w.p.KubePodProcess(waitForInitPodToRun):321 - Init container ready.. 2022-08-17 16:22:57 INFO i.a.w.p.KubePodProcess():546 - Copying files... 2022-08-17 16:22:57 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: destination_config.json 2022-08-17 16:22:57 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/2872b0cb-3cb2-4b8e-8f32-c029172d9452/destination_config.json airbyte/normalization-sync-2919-0-pzvdu:/config/destination_config.json -c init 2022-08-17 16:22:57 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: destination_catalog.json 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/df7de89a-67fe-4b78-b7cb-35b1934e5e9a/destination_catalog.json airbyte/normalization-sync-2919-0-pzvdu:/config/destination_catalog.json -c init 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):254 - Uploading file: FINISHED_UPLOADING 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):262 - kubectl cp /tmp/63159c18-63aa-4106-a1b9-27bf6ebfd562/FINISHED_UPLOADING airbyte/normalization-sync-2919-0-pzvdu:/config/FINISHED_UPLOADING -c init 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):265 - Waiting for kubectl cp to complete 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess(copyFilesToKubeConfigVolume):279 - kubectl cp complete, closing process 2022-08-17 16:22:58 INFO i.a.w.p.KubePodProcess():549 - Waiting until pod is ready... 2022-08-17 16:22:59 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$9):592 - Setting stdout... 2022-08-17 16:22:59 INFO i.a.w.p.KubePodProcess(lambda$setupStdOutAndStdErrListeners$10):604 - Setting stderr... 2022-08-17 16:23:00 INFO i.a.w.p.KubePodProcess():564 - Reading pod IP... 2022-08-17 16:23:00 INFO i.a.w.p.KubePodProcess():566 - Pod IP: 100.65.113.248 2022-08-17 16:23:00 INFO i.a.w.p.KubePodProcess():573 - Using null stdin output stream... 2022-08-17 16:23:00 INFO i.a.w.p.KubePodProcess(close):713 - (pod: airbyte / normalization-sync-2919-0-pzvdu) - Closed all resources for pod 2022-08-17 16:23:00 INFO i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):158 - Completing future exceptionally... io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:61) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:19) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:833) [?:?] Caused by: io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:58) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] ... 3 more Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:160) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:46) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:19) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:833) [?:?] 2022-08-17 16:23:00 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling... 2022-08-17 16:23:00 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):235 - Stopping temporal heartbeating... 2022-08-17 16:23:00 WARN i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=089919f6-2a97-339e-82ed-d9d4e87b6169, activityType=Normalize, attempt=1 java.lang.RuntimeException: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:233) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.normalize(NormalizationActivityImpl.java:72) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.lang.Thread.run(Thread.java:833) [?:?] Caused by: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed. at io.temporal.serviceclient.CheckedExceptionWrapper.wrap(CheckedExceptionWrapper.java:56) ~[temporal-serviceclient-1.8.1.jar:?] at io.temporal.internal.sync.WorkflowInternal.wrap(WorkflowInternal.java:448) ~[temporal-sdk-1.8.1.jar:?] at io.temporal.activity.Activity.wrap(Activity.java:51) ~[temporal-sdk-1.8.1.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:135) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$1(NormalizationActivityImpl.java:98) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:228) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] ... 14 more Caused by: java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed. at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:129) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$1(NormalizationActivityImpl.java:98) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:228) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] ... 14 more Caused by: io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:61) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:19) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] ... 1 more Caused by: io.airbyte.workers.WorkerException: Normalization Failed. at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:58) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:19) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] ... 1 more Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:160) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:46) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:19) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.36.0-alpha.jar:?] at java.lang.Thread.run(Thread.java:833) [?:?] 2022-08-17 16:23:00 WARN i.a.c.i.LineGobbler(voidCall):86 - generic gobbler IOException: Socket closed. Typically happens when cancelling a job. 2022-08-17 16:23:00 WARN i.a.c.i.LineGobbler(voidCall):86 - generic gobbler IOException: Socket closed. Typically happens when cancelling a job. 2022-08-17 16:23:00 normalization > sys.exit(main()) 2022-08-17 16:23:00 normalization > transform_redshift 2022-08-17 16:23:00 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 82, in main 2022-08-17 16:23:00 normalization > Running: transform-catalog --integration-type redshift --profile-config-dir /config --catalog destination_catalog.json --out /config/models/generated/ --json-column _airbyte_data 2022-08-17 16:23:00 normalization > TransformCatalog().run(args) 2022-08-17 16:23:00 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 35, in run 2022-08-17 16:23:00 normalization > Processing destination_catalog.json... 2022-08-17 16:23:00 normalization > self.process_catalog() 2022-08-17 16:23:00 normalization > Generating airbyte_ctes/wine_stg/jira_info_issues_ab1.sql from jira_info_issues 2022-08-17 16:23:00 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py", line 62, in process_catalog 2022-08-17 16:23:00 normalization > WARN: Unknown type for column editmeta at jira_info_issues 2022-08-17 16:23:00 normalization > processor.process(catalog_file=catalog_file, json_column_name=json_col, default_schema=schema) 2022-08-17 16:23:00 normalization > WARN: Unknown type for column changelog at jira_info_issues 2022-08-17 16:23:00 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py", line 74, in process 2022-08-17 16:23:00 normalization > WARN: Unknown type for column operations at jira_info_issues 2022-08-17 16:23:00 normalization > nested_processors = stream_processor.process() 2022-08-17 16:23:00 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py", line 282, in process 2022-08-17 16:23:00 normalization > Generating airbyte_ctes/wine_stg/jira_info_issues_ab2.sql from jira_info_issues 2022-08-17 16:23:00 normalization > self.generate_scd_type_2_model(from_table, column_names), 2022-08-17 16:23:00 normalization > Generating airbyte_views/wine_stg/jira_info_issues_stg.sql from jira_info_issues 2022-08-17 16:23:00 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py", line 659, in generate_scd_type_2_model 2022-08-17 16:23:00 normalization > 2022-08-17 16:23:00 normalization > cursor_field = self.get_cursor_field(column_names) 2022-08-17 16:23:00 normalization > Showing destination_catalog.json to diagnose/debug errors (1): 2022-08-17 16:23:00 normalization > File "/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py", line 944, in get_cursor_field 2022-08-17 16:23:00 normalization > 2022-08-17 16:23:00 normalization > raise ValueError(f"Unsupported nested cursor field {'.'.join(self.cursor_field)} for stream {self.stream_name}") 2022-08-17 16:23:00 normalization > { 2022-08-17 16:23:00 normalization > ValueError: Unsupported nested cursor field fields.updated for stream jira_info_issues 2022-08-17 16:23:00 normalization > "streams": [ 2022-08-17 16:23:00 normalization > { 2022-08-17 16:23:00 normalization > "stream": { 2022-08-17 16:23:00 normalization > "name": "jira_info_issues", 2022-08-17 16:23:00 normalization > "json_schema": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "$schema": "http://json-schema.org/draft-07/schema#", 2022-08-17 16:23:00 normalization > "properties": { 2022-08-17 16:23:00 normalization > "id": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The ID of the issue." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "key": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The key of the issue." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "self": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "format": "uri", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The URL of the issue details." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "names": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The ID and name of each field present on the issue.", 2022-08-17 16:23:00 normalization > "additionalProperties": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true 2022-08-17 16:23:00 normalization > } 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "expand": { 2022-08-17 16:23:00 normalization > "xml": { 2022-08-17 16:23:00 normalization > "attribute": true 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "Expand options that include additional issue details in the response." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "fields": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "properties": { 2022-08-17 16:23:00 normalization > "created": { 2022-08-17 16:23:00 normalization > "type": [ 2022-08-17 16:23:00 normalization > "string", 2022-08-17 16:23:00 normalization > "null" 2022-08-17 16:23:00 normalization > ], 2022-08-17 16:23:00 normalization > "format": "date-time", 2022-08-17 16:23:00 normalization > "description": "This field is not shown in schema / swagger, but exists in records and we use it as cursor fiekd. Updated may be absent. Added to solve the #4341" 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "updated": { 2022-08-17 16:23:00 normalization > "type": [ 2022-08-17 16:23:00 normalization > "string", 2022-08-17 16:23:00 normalization > "null" 2022-08-17 16:23:00 normalization > ], 2022-08-17 16:23:00 normalization > "format": "date-time", 2022-08-17 16:23:00 normalization > "description": "This field is not shown in schema / swagger, but exists in records and we use it as cursor fiekd. Updated may be absent. Added to solve the #4341" 2022-08-17 16:23:00 normalization > } 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "additionalProperties": {} 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "schema": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The schema describing each field present on the issue." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "editmeta": { 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The metadata for the fields on the issue that can be amended." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "changelog": { 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "Details of changelogs associated with the issue." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "projectId": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The ID of the project containing the issue." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "operations": { 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The operations that can be performed on the issue." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "projectKey": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The key of the project containing the issue." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "properties": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "Details of the issue properties identified in the request.", 2022-08-17 16:23:00 normalization > "additionalProperties": { 2022-08-17 16:23:00 normalization > "readOnly": true 2022-08-17 16:23:00 normalization > } 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "transitions": { 2022-08-17 16:23:00 normalization > "type": "array", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The transitions that can be performed on the issue." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "renderedFields": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The rendered value of each field present on the issue.", 2022-08-17 16:23:00 normalization > "additionalProperties": { 2022-08-17 16:23:00 normalization > "readOnly": true 2022-08-17 16:23:00 normalization > } 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "fieldsToInclude": { 2022-08-17 16:23:00 normalization > "type": "object" 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "versionedRepresentations": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The versions of each field on the issue.", 2022-08-17 16:23:00 normalization > "additionalProperties": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "additionalProperties": { 2022-08-17 16:23:00 normalization > "readOnly": true 2022-08-17 16:23:00 normalization > } 2022-08-17 16:23:00 normalization > } 2022-08-17 16:23:00 normalization > } 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "additionalProperties": false 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "supported_sync_modes": [ 2022-08-17 16:23:00 normalization > "full_refresh", 2022-08-17 16:23:00 normalization > "incremental" 2022-08-17 16:23:00 normalization > ], 2022-08-17 16:23:00 normalization > "source_defined_cursor": true, 2022-08-17 16:23:00 normalization > "default_cursor_field": [ 2022-08-17 16:23:00 normalization > "fields", 2022-08-17 16:23:00 normalization > "updated" 2022-08-17 16:23:00 normalization > ], 2022-08-17 16:23:00 normalization > "source_defined_primary_key": [ 2022-08-17 16:23:00 normalization > [ 2022-08-17 16:23:00 normalization > "id" 2022-08-17 16:23:00 normalization > ] 2022-08-17 16:23:00 normalization > ] 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "sync_mode": "incremental", 2022-08-17 16:23:00 normalization > "cursor_field": [ 2022-08-17 16:23:00 normalization > "fields", 2022-08-17 16:23:00 normalization > "updated" 2022-08-17 16:23:00 normalization > ], 2022-08-17 16:23:00 normalization > "destination_sync_mode": "append_dedup", 2022-08-17 16:23:00 normalization > "primary_key": [ 2022-08-17 16:23:00 normalization > [ 2022-08-17 16:23:00 normalization > "id" 2022-08-17 16:23:00 normalization > ] 2022-08-17 16:23:00 normalization > ] 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > { 2022-08-17 16:23:00 normalization > "stream": { 2022-08-17 16:23:00 normalization > "name": "jira_info_projects", 2022-08-17 16:23:00 normalization > "json_schema": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "$schema": "http://json-schema.org/draft-07/schema#", 2022-08-17 16:23:00 normalization > "properties": { 2022-08-17 16:23:00 normalization > "id": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "description": "The ID of the project." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "key": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The key of the project." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "url": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "A link to information about this project, such as project documentation." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "lead": { 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The username of the project lead." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "name": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The name of the project." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "self": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "format": "uri", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The URL of the project details." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "uuid": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "format": "uuid", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "Unique ID for next-gen projects." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "email": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "description": "An email address associated with the project." 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "roles": { 2022-08-17 16:23:00 normalization > "type": "object", 2022-08-17 16:23:00 normalization > "readOnly": true, 2022-08-17 16:23:00 normalization > "description": "The name and self URL for each role defined in the project. For more information, see [Create project role](#api-rest-api-3-role-post).", 2022-08-17 16:23:00 normalization > "additionalProperties": { 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "format": "uri", 2022-08-17 16:23:00 normalization > "readOnly": true 2022-08-17 16:23:00 normalization > } 2022-08-17 16:23:00 normalization > }, 2022-08-17 16:23:00 normalization > "style": { 2022-08-17 16:23:00 normalization > "enum": [ 2022-08-17 16:23:00 normalization > "classic", 2022-08-17 16:23:00 normalization > "next-gen" 2022-08-17 16:23:00 normalization > ], 2022-08-17 16:23:00 normalization > "type": "string", 2022-08-17 16:23:00 normalization > "readOnly": true,