2022-06-29 06:25:54 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):344 - Records read: 531875000 (282 GB) 2022-06-29 06:25:55 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):344 - Records read: 531876000 (282 GB) 2022-06-29 06:25:55 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):344 - Records read: 531877000 (282 GB) 2022-06-29 06:25:55 source > 2022-06-29 06:25:54 INFO i.d.e.EmbeddedEngine(stop):1047 - Stopping the embedded engine 2022-06-29 06:25:55 source > 2022-06-29 06:25:54 INFO i.d.e.EmbeddedEngine(stop):1055 - Waiting for PT5M for connector to stop 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.e.EmbeddedEngine(run):846 - Stopping the task and engine 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.c.c.BaseSourceTask(stop):238 - Stopping down connector 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.u.Threads(threadFactory):270 - Requested thread factory for connector PostgresConnector, id = sse named = keep-alive 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.u.Threads$3(newThread):287 - Creating thread debezium-postgresconnector-sse-keep-alive 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.c.p.PostgresSchema(printReplicaIdentityInfo):103 - REPLICA IDENTITY for 'dp.dp_indicator_daily' is 'DEFAULT'; UPDATE and DELETE events will contain previous values only for PK columns 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.c.p.PostgresStreamingChangeEventSource(searchWalPosition):314 - Searching for WAL resume position 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.c.p.PostgresStreamingChangeEventSource(searchWalPosition):335 - WAL resume position 'null' discovered 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.j.JdbcConnection(lambda$doClose$3):956 - Connection gracefully closed 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.j.JdbcConnection(lambda$doClose$3):956 - Connection gracefully closed 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.c.p.c.PostgresReplicationConnection(initPublication):145 - Initializing PgOutput logical decoder publication 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.u.Threads(threadFactory):270 - Requested thread factory for connector PostgresConnector, id = sse named = keep-alive 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.u.Threads$3(newThread):287 - Creating thread debezium-postgresconnector-sse-keep-alive 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.c.p.PostgresStreamingChangeEventSource(processMessages):202 - Processing messages 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.j.JdbcConnection(lambda$doClose$3):956 - Connection gracefully closed 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.p.ChangeEventSourceCoordinator(streamEvents):175 - Finished streaming 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.p.ChangeEventSourceCoordinator(streamingConnected):234 - Connected metrics set to 'false' 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.j.JdbcConnection(lambda$doClose$3):956 - Connection gracefully closed 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.d.j.JdbcConnection(lambda$doClose$3):956 - Connection gracefully closed 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.a.i.d.i.DebeziumRecordPublisher(lambda$start$1):85 - Debezium engine shutdown. 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.a.i.s.p.PostgresCdcStateHandler(saveState):32 - debezium state: {"{\"schema\":null,\"payload\":[\"sse\",{\"server\":\"sse\"}]}":"{\"last_snapshot_record\":true,\"lsn\":23879882514432,\"txId\":856821367,\"ts_usec\":1656483952394000,\"snapshot\":true}"} 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.a.i.s.r.AbstractDbSource(lambda$read$2):134 - Closing database connection pool. 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO c.z.h.HikariDataSource(close):350 - HikariPool-2 - Shutdown initiated... 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO c.z.h.HikariDataSource(close):352 - HikariPool-2 - Shutdown completed. 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.a.i.s.r.AbstractDbSource(lambda$read$2):136 - Closed database connection pool. 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.a.i.b.IntegrationRunner(runInternal):171 - Completed integration: io.airbyte.integrations.base.ssh.SshWrappedSource 2022-06-29 06:25:55 source > 2022-06-29 06:25:55 INFO i.a.i.b.a.AdaptiveSourceRunner$Runner(run):87 - Completed source: io.airbyte.integrations.base.ssh.SshWrappedSource 2022-06-29 06:25:56 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):347 - Source has no more messages, closing connection. 2022-06-29 06:25:56 INFO i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$6):355 - Total records read: 531877648 (282 GB) 2022-06-29 06:25:56 WARN i.a.w.g.DefaultReplicationWorker(lambda$getReplicationRunnable$5):358 - Schema validation errors found for stream dpdp_indicator_daily. Error messages: [$.measured_source is of an incorrect type. Expected it to be string, $.day is of an incorrect type. Expected it to be date, $.invoice is of an incorrect type. Expected it to be number, $.estimated_source is of an incorrect type. Expected it to be string, $.measured is of an incorrect type. Expected it to be number, $.estimated is of an incorrect type. Expected it to be number, $.gtb_reading is of an incorrect type. Expected it to be number, $.measured_source_year is of an incorrect type. Expected it to be number, $.distributor_billing is of an incorrect type. Expected it to be number, $.estimated_source_year is of an incorrect type. Expected it to be number, $.predicted is of an incorrect type. Expected it to be number, $.manual_reading is of an incorrect type. Expected it to be number, $.iot_reading is of an incorrect type. Expected it to be number, $.distributor_reading is of an incorrect type. Expected it to be number, $.ipmvp_prediction is of an incorrect type. Expected it to be number, $._ab_cdc_deleted_at is of an incorrect type. Expected it to be string] 2022-06-29 06:25:56 INFO i.a.w.g.DefaultReplicationWorker(run):174 - One of source or destination thread complete. Waiting on the other. 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):65 - Airbyte message consumer: succeeded. 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.b.BufferedStreamConsumer(close):171 - executing on success close procedure. 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.r.SerializedBufferingStrategy(flushAll):103 - Flushing all 1 current buffers (16 MB in total) 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.r.SerializedBufferingStrategy(lambda$flushAll$2):106 - Flushing buffer of stream dp_indicator_daily (16 MB) 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.s.StagingConsumerFactory(lambda$flushBufferFunction$3):158 - Flushing buffer for stream dp_indicator_daily (16 MB) to staging 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.r.BaseSerializedBuffer(flush):131 - Wrapping up compression and write GZIP trailer data. 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.r.BaseSerializedBuffer(flush):138 - Finished writing data to 68075c8c-18a6-4ad9-a52d-41027145441f5237892297201265500.csv.gz (16 MB) 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO a.m.s.StreamTransferManager(getMultiPartOutputStreams):329 - Initiated multipart upload to airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/26.csv.gz with full ID db47zNE8YMByA.MmrX4A5u2U60aBCIYn9A2wq1rFvO2hjFMKl6WR8wgzgPBx6ooI83kkVmB3FyY5HUrispDk_tUA0LERbYanHvaMKvMFU4zYDIXnq3xdbhlvLg0TYUZX 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO a.m.s.MultiPartOutputStream(close):158 - Called close() on [MultipartOutputStream for parts 1 - 10000] 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/26.csv.gz with id db47zNE8Y...vLg0TYUZX]: Finished uploading [Part number 1 containing 10.01 MB] 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO a.m.s.StreamTransferManager(uploadStreamPart):558 - [Manager uploading to airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/26.csv.gz with id db47zNE8Y...vLg0TYUZX]: Finished uploading [Part number 2 containing 6.19 MB] 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO a.m.s.StreamTransferManager(complete):395 - [Manager uploading to airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/26.csv.gz with id db47zNE8Y...vLg0TYUZX]: Completed 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.s.S3StorageOperations(loadDataIntoBucket):178 - Uploaded buffer file to storage: 68075c8c-18a6-4ad9-a52d-41027145441f5237892297201265500.csv.gz -> source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/26.csv.gz (filename: 26.csv.gz) 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.r.FileBuffer(deleteFile):81 - Deleting tempFile data 68075c8c-18a6-4ad9-a52d-41027145441f5237892297201265500.csv.gz 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.r.SerializedBufferingStrategy(close):127 - Closing buffer for stream dp_indicator_daily 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.s.StagingConsumerFactory(lambda$onCloseFunction$4):186 - Copying into tables in destination started for 1 streams 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.s.StagingConsumerFactory(lambda$onCloseFunction$4):195 - Copying stream dp_indicator_daily of schema source into tmp table _airbyte_tmp_ffp_dp_indicator_daily to final table _airbyte_raw_dp_indicator_daily from stage path source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/ with 106 file(s) [0.csv.gz,1.csv.gz,2.csv.gz,3.csv.gz,4.csv.gz,5.csv.gz,6.csv.gz,7.csv.gz,8.csv.gz,9.csv.gz,10.csv.gz,11.csv.gz,12.csv.gz,13.csv.gz,14.csv.gz,15.csv.gz,16.csv.gz,17.csv.gz,18.csv.gz,19.csv.gz,20.csv.gz,21.csv.gz,22.csv.gz,23.csv.gz,24.csv.gz,25.csv.gz,26.csv.gz,27.csv.gz,28.csv.gz,29.csv.gz,30.csv.gz,31.csv.gz,32.csv.gz,33.csv.gz,34.csv.gz,35.csv.gz,36.csv.gz,0.csv.gz,1.csv.gz,2.csv.gz,3.csv.gz,4.csv.gz,5.csv.gz,6.csv.gz,7.csv.gz,8.csv.gz,9.csv.gz,10.csv.gz,11.csv.gz,12.csv.gz,13.csv.gz,14.csv.gz,15.csv.gz,16.csv.gz,17.csv.gz,18.csv.gz,19.csv.gz,20.csv.gz,21.csv.gz,22.csv.gz,23.csv.gz,24.csv.gz,25.csv.gz,26.csv.gz,27.csv.gz,28.csv.gz,29.csv.gz,30.csv.gz,31.csv.gz,32.csv.gz,33.csv.gz,34.csv.gz,35.csv.gz,36.csv.gz,37.csv.gz,38.csv.gz,39.csv.gz,0.csv.gz,1.csv.gz,0.csv.gz,1.csv.gz,2.csv.gz,3.csv.gz,4.csv.gz,5.csv.gz,6.csv.gz,7.csv.gz,8.csv.gz,9.csv.gz,10.csv.gz,11.csv.gz,12.csv.gz,13.csv.gz,14.csv.gz,15.csv.gz,16.csv.gz,17.csv.gz,18.csv.gz,19.csv.gz,20.csv.gz,21.csv.gz,22.csv.gz,23.csv.gz,24.csv.gz,25.csv.gz,26.csv.gz] 2022-06-29 06:25:56 destination > 2022-06-29 06:25:56 INFO i.a.i.d.r.o.RedshiftS3StagingSqlOperations(copyIntoTmpTableFromStage):90 - Starting copy to tmp table from stage: _airbyte_tmp_ffp_dp_indicator_daily in destination from stage: source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/, schema: source, . 2022-06-29 06:25:58 destination > 2022-06-29 06:25:58 INFO i.a.i.d.s.S3StorageOperations(cleanUpBucketObject):270 - Storage bucket source_dp_indicator_daily has been cleaned-up (0 objects were deleted)... 2022-06-29 06:25:58 destination > 2022-06-29 06:25:58 INFO i.a.i.d.s.StagingConsumerFactory(lambda$onCloseFunction$4):203 - Cleaning stage path source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/ 2022-06-29 06:25:58 destination > 2022-06-29 06:25:58 ERROR i.a.i.d.b.BufferedStreamConsumer(close):199 - Close failed. 2022-06-29 06:25:58 destination > java.lang.RuntimeException: Failed to upload data from stage source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/ 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.staging.StagingConsumerFactory.lambda$onCloseFunction$4(StagingConsumerFactory.java:204) ~[io.airbyte.airbyte-integrations.connectors-destination-jdbc-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.buffered_stream_consumer.OnCloseFunction.accept(OnCloseFunction.java:9) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer.close(BufferedStreamConsumer.java:191) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.lambda$close$0(FailureTrackingAirbyteMessageConsumer.java:67) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) [io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.close(FailureTrackingAirbyteMessageConsumer.java:67) [io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:166) [io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:107) [io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.RedshiftDestination.main(RedshiftDestination.java:64) [io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > Caused by: java.lang.RuntimeException: java.sql.SQLException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > Details: ----------------------------------------------- 2022-06-29 06:25:58 destination > error: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > code: 8001 2022-06-29 06:25:58 destination > context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 2022-06-29 06:25:58 destination > query: 30062 2022-06-29 06:25:58 destination > location: s3_utility.cpp:400 2022-06-29 06:25:58 destination > process: padbmaster [pid=3189] 2022-06-29 06:25:58 destination > -----------------------------------------------; 2022-06-29 06:25:58 destination > at io.airbyte.commons.lang.Exceptions.castCheckedToRuntime(Exceptions.java:58) ~[io.airbyte-airbyte-commons-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.commons.lang.Exceptions.toRuntime(Exceptions.java:41) ~[io.airbyte-airbyte-commons-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.executeCopy(RedshiftS3StagingSqlOperations.java:117) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$3(RedshiftS3StagingSqlOperations.java:95) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$4(RedshiftS3StagingSqlOperations.java:95) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.commons.lang.Exceptions.castCheckedToRuntime(Exceptions.java:54) ~[io.airbyte-airbyte-commons-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.commons.lang.Exceptions.toRuntime(Exceptions.java:41) ~[io.airbyte-airbyte-commons-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$5(RedshiftS3StagingSqlOperations.java:93) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.copyIntoTmpTableFromStage(RedshiftS3StagingSqlOperations.java:92) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.staging.StagingConsumerFactory.lambda$onCloseFunction$4(StagingConsumerFactory.java:200) ~[io.airbyte.airbyte-integrations.connectors-destination-jdbc-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > ... 8 more 2022-06-29 06:25:58 destination > Caused by: java.sql.SQLException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > Details: ----------------------------------------------- 2022-06-29 06:25:58 destination > error: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > code: 8001 2022-06-29 06:25:58 destination > context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 2022-06-29 06:25:58 destination > query: 30062 2022-06-29 06:25:58 destination > location: s3_utility.cpp:400 2022-06-29 06:25:58 destination > process: padbmaster [pid=3189] 2022-06-29 06:25:58 destination > -----------------------------------------------; 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.messages.inbound.ErrorResponse.toErrorException(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.handleErrorResponse(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.handleMessage(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.jdbc.communications.InboundMessagesPipeline.getNextMessageOfClass(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.doMoveToNextClass(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.getErrorResponse(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGClient.handleErrorsScenario3(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGClient.handleErrors(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] 2022-06-29 06:25:58 destination > Caused by: com.amazon.support.exceptions.ErrorException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > Details: ----------------------------------------------- 2022-06-29 06:25:58 destination > error: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > code: 8001 2022-06-29 06:25:58 destination > context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 2022-06-29 06:25:58 destination > query: 30062 2022-06-29 06:25:58 destination > location: s3_utility.cpp:400 2022-06-29 06:25:58 destination > process: padbmaster [pid=3189] 2022-06-29 06:25:58 destination > -----------------------------------------------; 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.messages.inbound.ErrorResponse.toErrorException(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.handleErrorResponse(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.handleMessage(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.jdbc.communications.InboundMessagesPipeline.getNextMessageOfClass(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.doMoveToNextClass(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.getErrorResponse(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGClient.handleErrorsScenario3(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGClient.handleErrors(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] 2022-06-29 06:25:58 destination > 2022-06-29 06:25:58 ERROR i.a.i.b.AirbyteExceptionHandler(uncaughtException):26 - Something went wrong in the connector. See the logs for more details. 2022-06-29 06:25:58 destination > java.lang.RuntimeException: Failed to upload data from stage source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/ 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.staging.StagingConsumerFactory.lambda$onCloseFunction$4(StagingConsumerFactory.java:204) ~[io.airbyte.airbyte-integrations.connectors-destination-jdbc-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.buffered_stream_consumer.OnCloseFunction.accept(OnCloseFunction.java:9) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer.close(BufferedStreamConsumer.java:191) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.lambda$close$0(FailureTrackingAirbyteMessageConsumer.java:67) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.close(FailureTrackingAirbyteMessageConsumer.java:67) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:166) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:107) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.RedshiftDestination.main(RedshiftDestination.java:64) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > Caused by: java.lang.RuntimeException: java.sql.SQLException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > Details: ----------------------------------------------- 2022-06-29 06:25:58 destination > error: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > code: 8001 2022-06-29 06:25:58 destination > context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 2022-06-29 06:25:58 destination > query: 30062 2022-06-29 06:25:58 destination > location: s3_utility.cpp:400 2022-06-29 06:25:58 destination > process: padbmaster [pid=3189] 2022-06-29 06:25:58 destination > -----------------------------------------------; 2022-06-29 06:25:58 destination > at io.airbyte.commons.lang.Exceptions.castCheckedToRuntime(Exceptions.java:58) ~[io.airbyte-airbyte-commons-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.commons.lang.Exceptions.toRuntime(Exceptions.java:41) ~[io.airbyte-airbyte-commons-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.executeCopy(RedshiftS3StagingSqlOperations.java:117) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$3(RedshiftS3StagingSqlOperations.java:95) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[?:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$4(RedshiftS3StagingSqlOperations.java:95) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.commons.lang.Exceptions.castCheckedToRuntime(Exceptions.java:54) ~[io.airbyte-airbyte-commons-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.commons.lang.Exceptions.toRuntime(Exceptions.java:41) ~[io.airbyte-airbyte-commons-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$5(RedshiftS3StagingSqlOperations.java:93) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) ~[io.airbyte.airbyte-integrations.bases-base-java-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.copyIntoTmpTableFromStage(RedshiftS3StagingSqlOperations.java:92) ~[io.airbyte.airbyte-integrations.connectors-destination-redshift-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > at io.airbyte.integrations.destination.staging.StagingConsumerFactory.lambda$onCloseFunction$4(StagingConsumerFactory.java:200) ~[io.airbyte.airbyte-integrations.connectors-destination-jdbc-0.39.20-alpha.jar:?] 2022-06-29 06:25:58 destination > ... 8 more 2022-06-29 06:25:58 destination > Caused by: java.sql.SQLException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > Details: ----------------------------------------------- 2022-06-29 06:25:58 destination > error: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > code: 8001 2022-06-29 06:25:58 destination > context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 2022-06-29 06:25:58 destination > query: 30062 2022-06-29 06:25:58 destination > location: s3_utility.cpp:400 2022-06-29 06:25:58 destination > process: padbmaster [pid=3189] 2022-06-29 06:25:58 destination > -----------------------------------------------; 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.messages.inbound.ErrorResponse.toErrorException(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.handleErrorResponse(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.handleMessage(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.jdbc.communications.InboundMessagesPipeline.getNextMessageOfClass(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.doMoveToNextClass(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.getErrorResponse(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGClient.handleErrorsScenario3(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGClient.handleErrors(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] 2022-06-29 06:25:58 destination > Caused by: com.amazon.support.exceptions.ErrorException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > Details: ----------------------------------------------- 2022-06-29 06:25:58 destination > error: Mandatory url is not present in manifest file. 2022-06-29 06:25:58 destination > code: 8001 2022-06-29 06:25:58 destination > context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 2022-06-29 06:25:58 destination > query: 30062 2022-06-29 06:25:58 destination > location: s3_utility.cpp:400 2022-06-29 06:25:58 destination > process: padbmaster [pid=3189] 2022-06-29 06:25:58 destination > -----------------------------------------------; 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.messages.inbound.ErrorResponse.toErrorException(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.handleErrorResponse(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.handleMessage(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.jdbc.communications.InboundMessagesPipeline.getNextMessageOfClass(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.doMoveToNextClass(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGMessagingContext.getErrorResponse(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGClient.handleErrorsScenario3(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.client.PGClient.handleErrors(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) ~[redshift-jdbc42-no-awssdk-1.2.51.1078.jar:RedshiftJDBC_1.2.51.1078] 2022-06-29 06:25:58 destination > at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] 2022-06-29 06:25:58 destination > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] 2022-06-29 06:25:58 INFO i.a.w.g.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):405 - State in DefaultReplicationWorker from destination: io.airbyte.protocol.models.AirbyteMessage@4d7ea056[type=TRACE,log=,spec=,connectionStatus=,catalog=,record=,state=,trace=io.airbyte.protocol.models.AirbyteTraceMessage@27cb6455[type=ERROR,emittedAt=1.656483958788E12,error=io.airbyte.protocol.models.AirbyteErrorTraceMessage@1c6704ac[message=Something went wrong in the connector. See the logs for more details.,internalMessage=java.lang.RuntimeException: Failed to upload data from stage source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/,stackTrace=java.lang.RuntimeException: Failed to upload data from stage source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/ at io.airbyte.integrations.destination.staging.StagingConsumerFactory.lambda$onCloseFunction$4(StagingConsumerFactory.java:204) at io.airbyte.integrations.destination.buffered_stream_consumer.OnCloseFunction.accept(OnCloseFunction.java:9) at io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer.close(BufferedStreamConsumer.java:191) at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.lambda$close$0(FailureTrackingAirbyteMessageConsumer.java:67) at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.close(FailureTrackingAirbyteMessageConsumer.java:67) at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:166) at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:107) at io.airbyte.integrations.destination.redshift.RedshiftDestination.main(RedshiftDestination.java:64) Caused by: java.lang.RuntimeException: java.sql.SQLException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. Details: ----------------------------------------------- error: Mandatory url is not present in manifest file. code: 8001 context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 query: 30062 location: s3_utility.cpp:400 process: padbmaster [pid=3189] -----------------------------------------------; at io.airbyte.commons.lang.Exceptions.castCheckedToRuntime(Exceptions.java:58) at io.airbyte.commons.lang.Exceptions.toRuntime(Exceptions.java:41) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.executeCopy(RedshiftS3StagingSqlOperations.java:117) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$3(RedshiftS3StagingSqlOperations.java:95) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$4(RedshiftS3StagingSqlOperations.java:95) at io.airbyte.commons.lang.Exceptions.castCheckedToRuntime(Exceptions.java:54) at io.airbyte.commons.lang.Exceptions.toRuntime(Exceptions.java:41) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$5(RedshiftS3StagingSqlOperations.java:93) at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.copyIntoTmpTableFromStage(RedshiftS3StagingSqlOperations.java:92) at io.airbyte.integrations.destination.staging.StagingConsumerFactory.lambda$onCloseFunction$4(StagingConsumerFactory.java:200) ... 8 more Caused by: java.sql.SQLException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. Details: ----------------------------------------------- error: Mandatory url is not present in manifest file. code: 8001 context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 query: 30062 location: s3_utility.cpp:400 process: padbmaster [pid=3189] -----------------------------------------------; at com.amazon.redshift.client.messages.inbound.ErrorResponse.toErrorException(Unknown Source) at com.amazon.redshift.client.PGMessagingContext.handleErrorResponse(Unknown Source) at com.amazon.redshift.client.PGMessagingContext.handleMessage(Unknown Source) at com.amazon.jdbc.communications.InboundMessagesPipeline.getNextMessageOfClass(Unknown Source) at com.amazon.redshift.client.PGMessagingContext.doMoveToNextClass(Unknown Source) at com.amazon.redshift.client.PGMessagingContext.getErrorResponse(Unknown Source) at com.amazon.redshift.client.PGClient.handleErrorsScenario3(Unknown Source) at com.amazon.redshift.client.PGClient.handleErrors(Unknown Source) at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) Caused by: com.amazon.support.exceptions.ErrorException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. Details: ----------------------------------------------- error: Mandatory url is not present in manifest file. code: 8001 context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 query: 30062 location: s3_utility.cpp:400 process: padbmaster [pid=3189] -----------------------------------------------; ... 13 more ,failureType=system_error,additionalProperties={}],additionalProperties={}],additionalProperties={}] 2022-06-29 06:25:59 ERROR i.a.w.g.DefaultReplicationWorker(run):180 - Sync worker failed. java.util.concurrent.ExecutionException: io.airbyte.workers.general.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?] at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:175) ~[io.airbyte-airbyte-workers-0.39.20-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:65) ~[io.airbyte-airbyte-workers-0.39.20-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.20-alpha.jar:?] at java.lang.Thread.run(Thread.java:833) [?:?] Suppressed: io.airbyte.workers.exception.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled. at io.airbyte.workers.internal.DefaultAirbyteDestination.close(DefaultAirbyteDestination.java:119) ~[io.airbyte-airbyte-workers-0.39.20-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:137) ~[io.airbyte-airbyte-workers-0.39.20-alpha.jar:?] at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:65) ~[io.airbyte-airbyte-workers-0.39.20-alpha.jar:?] at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.20-alpha.jar:?] at java.lang.Thread.run(Thread.java:833) [?:?] Caused by: io.airbyte.workers.general.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$7(DefaultReplicationWorker.java:410) ~[io.airbyte-airbyte-workers-0.39.20-alpha.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?] ... 1 more 2022-06-29 06:25:59 INFO i.a.w.g.DefaultReplicationWorker(run):239 - sync summary: io.airbyte.config.ReplicationAttemptSummary@e70c166[status=failed,recordsSynced=531877647,bytesSynced=302944484330,startTime=1656408836048,endTime=1656483959286,totalStats=io.airbyte.config.SyncStats@24513ff0[recordsEmitted=531877647,bytesEmitted=302944484330,stateMessagesEmitted=1,recordsCommitted=0],streamStats=[io.airbyte.config.StreamSyncStats@254dafb2[streamName=dp_indicator_daily,stats=io.airbyte.config.SyncStats@2f84cc2[recordsEmitted=531877647,bytesEmitted=302944484330,stateMessagesEmitted=,recordsCommitted=]]]] 2022-06-29 06:25:59 INFO i.a.w.g.DefaultReplicationWorker(run):266 - Source output at least one state message 2022-06-29 06:25:59 WARN i.a.w.g.DefaultReplicationWorker(run):279 - State capture: No state retained. 2022-06-29 06:25:59 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-06-29 06:25:59 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):157 - sync summary: io.airbyte.config.StandardSyncOutput@485e245e[standardSyncSummary=io.airbyte.config.StandardSyncSummary@695cc66c[status=failed,recordsSynced=531877647,bytesSynced=302944484330,startTime=1656408836048,endTime=1656483959286,totalStats=io.airbyte.config.SyncStats@24513ff0[recordsEmitted=531877647,bytesEmitted=302944484330,stateMessagesEmitted=1,recordsCommitted=0],streamStats=[io.airbyte.config.StreamSyncStats@254dafb2[streamName=dp_indicator_daily,stats=io.airbyte.config.SyncStats@2f84cc2[recordsEmitted=531877647,bytesEmitted=302944484330,stateMessagesEmitted=,recordsCommitted=]]]],normalizationSummary=,state=,outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@79dccd24[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@113471bd[stream=io.airbyte.protocol.models.AirbyteStream@2871db11[name=dp_indicator_daily,jsonSchema={"type":"object","properties":{"day":{"type":"string","format":"date"},"invoice":{"type":"number"},"measured":{"type":"number"},"estimated":{"type":"number"},"predicted":{"type":"number"},"_ab_cdc_lsn":{"type":"number"},"gtb_reading":{"type":"number"},"iot_reading":{"type":"number"},"manual_reading":{"type":"number"},"dp_indicator_id":{"type":"number"},"measured_source":{"type":"string"},"estimated_source":{"type":"string"},"ipmvp_prediction":{"type":"number"},"predicted_source":{"type":"string"},"_ab_cdc_deleted_at":{"type":"string"},"_ab_cdc_updated_at":{"type":"string"},"distributor_billing":{"type":"number"},"distributor_reading":{"type":"number"},"measured_source_year":{"type":"number"},"estimated_source_year":{"type":"number"},"predicted_source_year":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=true,defaultCursorField=[],sourceDefinedPrimaryKey=[[dp_indicator_id], [day]],namespace=,additionalProperties={}],syncMode=incremental,cursorField=[],destinationSyncMode=append_dedup,primaryKey=[[dp_indicator_id], [day]],additionalProperties={}]],additionalProperties={}],failures=[io.airbyte.config.FailureReason@271117d5[failureOrigin=destination,failureType=system_error,internalMessage=java.lang.RuntimeException: Failed to upload data from stage source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/,externalMessage=Something went wrong in the connector. See the logs for more details.,metadata=io.airbyte.config.Metadata@383b4bfe[additionalProperties={attemptNumber=0, jobId=44, from_trace_message=true}],stacktrace=java.lang.RuntimeException: Failed to upload data from stage source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/ at io.airbyte.integrations.destination.staging.StagingConsumerFactory.lambda$onCloseFunction$4(StagingConsumerFactory.java:204) at io.airbyte.integrations.destination.buffered_stream_consumer.OnCloseFunction.accept(OnCloseFunction.java:9) at io.airbyte.integrations.destination.buffered_stream_consumer.BufferedStreamConsumer.close(BufferedStreamConsumer.java:191) at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.lambda$close$0(FailureTrackingAirbyteMessageConsumer.java:67) at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) at io.airbyte.integrations.base.FailureTrackingAirbyteMessageConsumer.close(FailureTrackingAirbyteMessageConsumer.java:67) at io.airbyte.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.java:166) at io.airbyte.integrations.base.IntegrationRunner.run(IntegrationRunner.java:107) at io.airbyte.integrations.destination.redshift.RedshiftDestination.main(RedshiftDestination.java:64) Caused by: java.lang.RuntimeException: java.sql.SQLException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. Details: ----------------------------------------------- error: Mandatory url is not present in manifest file. code: 8001 context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 query: 30062 location: s3_utility.cpp:400 process: padbmaster [pid=3189] -----------------------------------------------; at io.airbyte.commons.lang.Exceptions.castCheckedToRuntime(Exceptions.java:58) at io.airbyte.commons.lang.Exceptions.toRuntime(Exceptions.java:41) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.executeCopy(RedshiftS3StagingSqlOperations.java:117) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$3(RedshiftS3StagingSqlOperations.java:95) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) at java.base/java.util.stream.Streams$StreamBuilderImpl.forEachRemaining(Streams.java:411) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$4(RedshiftS3StagingSqlOperations.java:95) at io.airbyte.commons.lang.Exceptions.castCheckedToRuntime(Exceptions.java:54) at io.airbyte.commons.lang.Exceptions.toRuntime(Exceptions.java:41) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.lambda$copyIntoTmpTableFromStage$5(RedshiftS3StagingSqlOperations.java:93) at io.airbyte.integrations.base.sentry.AirbyteSentry.executeWithTracing(AirbyteSentry.java:54) at io.airbyte.integrations.destination.redshift.operations.RedshiftS3StagingSqlOperations.copyIntoTmpTableFromStage(RedshiftS3StagingSqlOperations.java:92) at io.airbyte.integrations.destination.staging.StagingConsumerFactory.lambda$onCloseFunction$4(StagingConsumerFactory.java:200) ... 8 more Caused by: java.sql.SQLException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. Details: ----------------------------------------------- error: Mandatory url is not present in manifest file. code: 8001 context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 query: 30062 location: s3_utility.cpp:400 process: padbmaster [pid=3189] -----------------------------------------------; at com.amazon.redshift.client.messages.inbound.ErrorResponse.toErrorException(Unknown Source) at com.amazon.redshift.client.PGMessagingContext.handleErrorResponse(Unknown Source) at com.amazon.redshift.client.PGMessagingContext.handleMessage(Unknown Source) at com.amazon.jdbc.communications.InboundMessagesPipeline.getNextMessageOfClass(Unknown Source) at com.amazon.redshift.client.PGMessagingContext.doMoveToNextClass(Unknown Source) at com.amazon.redshift.client.PGMessagingContext.getErrorResponse(Unknown Source) at com.amazon.redshift.client.PGClient.handleErrorsScenario3(Unknown Source) at com.amazon.redshift.client.PGClient.handleErrors(Unknown Source) at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) at com.amazon.redshift.dataengine.PGQueryExecutor$CallableExecuteTask.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) Caused by: com.amazon.support.exceptions.ErrorException: [Amazon](500310) Invalid operation: Mandatory url is not present in manifest file. Details: ----------------------------------------------- error: Mandatory url is not present in manifest file. code: 8001 context: Manifest file location=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-8847-ea88bf892c1d/8cfeb573-8f24-4933-8ace-1a28f14489cb.manifest url=s3://airbyte-dev-data/source_dp_indicator_daily/2022_06_28_09_0c089d8b-14d9-407b-884 query: 30062 location: s3_utility.cpp:400 process: padbmaster [pid=3189] -----------------------------------------------; ... 13 more ,retryable=,timestamp=1656483958788], io.airbyte.config.FailureReason@a307cdd[failureOrigin=destination,failureType=,internalMessage=io.airbyte.workers.general.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1,externalMessage=Something went wrong within the destination connector,metadata=io.airbyte.config.Metadata@2c3f87d9[additionalProperties={attemptNumber=0, jobId=44}],stacktrace=java.util.concurrent.CompletionException: io.airbyte.workers.general.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: io.airbyte.workers.general.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1 at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$7(DefaultReplicationWorker.java:410) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ... 3 more ,retryable=,timestamp=1656483959284]]] 2022-06-29 06:25:59 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):237 - Stopping temporal heartbeating... 2022-06-29 06:25:59 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/44/0/logs.log 2022-06-29 06:25:59 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.39.20-alpha 2022-06-29 06:25:59 INFO i.a.w.g.DefaultNormalizationWorker(run):49 - Running normalization. 2022-06-29 06:25:59 INFO i.a.w.n.DefaultNormalizationRunner(runProcess):122 - Running with normalization version: airbyte/normalization-redshift:0.2.5 2022-06-29 06:25:59 INFO i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/normalization-redshift:0.2.5 exists... 2022-06-29 06:25:59 INFO i.a.c.i.LineGobbler(voidCall):82 - airbyte/normalization-redshift:0.2.5 was found locally. 2022-06-29 06:25:59 INFO i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 44 2022-06-29 06:25:59 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/44/0/normalize --log-driver none --name normalization-redshift-normalize-44-0-cekyr --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e AIRBYTE_VERSION=0.39.20-alpha airbyte/normalization-redshift:0.2.5 run --integration-type redshift --config destination_config.json --catalog destination_catalog.json 2022-06-29 06:25:59 normalization > Running: transform-config --config destination_config.json --integration-type redshift --out /data/44/0/normalize 2022-06-29 06:25:59 normalization > Namespace(config='destination_config.json', integration_type=, out='/data/44/0/normalize') 2022-06-29 06:25:59 normalization > transform_redshift 2022-06-29 06:26:00 normalization > Running: transform-catalog --integration-type redshift --profile-config-dir /data/44/0/normalize --catalog destination_catalog.json --out /data/44/0/normalize/models/generated/ --json-column _airbyte_data 2022-06-29 06:26:00 normalization > Processing destination_catalog.json... 2022-06-29 06:26:00 normalization > Generating airbyte_ctes/source/dp_indicator_daily_ab1.sql from dp_indicator_daily 2022-06-29 06:26:00 normalization > Generating airbyte_ctes/source/dp_indicator_daily_ab2.sql from dp_indicator_daily 2022-06-29 06:26:00 normalization > Generating airbyte_views/source/dp_indicator_daily_stg.sql from dp_indicator_daily 2022-06-29 06:26:00 normalization > Generating airbyte_incremental/scd/source/dp_indicator_daily_scd.sql from dp_indicator_daily 2022-06-29 06:26:00 normalization > Generating airbyte_incremental/source/dp_indicator_daily.sql from dp_indicator_daily 2022-06-29 06:26:00 normalization > detected no config file for ssh, assuming ssh is off. 2022-06-29 06:26:02 normalization > [--event-buffer-size EVENT_BUFFER_SIZE] 2022-06-29 06:26:02 normalization > --event-buffer-size EVENT_BUFFER_SIZE 2022-06-29 06:26:02 normalization > 2022-06-29 06:26:02 normalization > DBT >=1.0.0 detected; using 10K event buffer size 2022-06-29 06:26:02 normalization > 2022-06-29 06:26:05 normalization > 06:26:05 Running with dbt=1.0.0 2022-06-29 06:26:05 normalization > 06:26:05 Partial parse save file not found. Starting full parse. 2022-06-29 06:26:07 normalization > 06:26:07 [WARNING]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources. 2022-06-29 06:26:07 normalization > There are 1 unused configuration paths: 2022-06-29 06:26:07 normalization > - models.airbyte_utils.generated.airbyte_tables 2022-06-29 06:26:07 normalization > 2022-06-29 06:26:07 normalization > 06:26:07 Found 5 models, 0 tests, 0 snapshots, 0 analyses, 552 macros, 0 operations, 0 seed files, 1 source, 0 exposures, 0 metrics 2022-06-29 06:26:07 normalization > 06:26:07 2022-06-29 06:26:07 normalization > 06:26:07 Concurrency: 4 threads (target='prod') 2022-06-29 06:26:07 normalization > 06:26:07 2022-06-29 06:26:11 normalization > 06:26:11 1 of 3 START view model _airbyte_source.dp_indicator_daily_stg.......................................................... [RUN] 2022-06-29 06:26:11 normalization > 06:26:11 1 of 3 OK created view model _airbyte_source.dp_indicator_daily_stg..................................................... [CREATE VIEW in 0.49s] 2022-06-29 06:26:11 normalization > 06:26:11 2 of 3 START incremental model source.dp_indicator_daily_scd............................................................ [RUN] 2022-06-29 06:26:23 normalization > 06:26:23 2 of 3 OK created incremental model source.dp_indicator_daily_scd....................................................... [INSERT 0 0 in 11.22s] 2022-06-29 06:26:23 normalization > 06:26:23 3 of 3 START incremental model source.dp_indicator_daily................................................................ [RUN] 2022-06-29 06:26:33 normalization > 06:26:33 3 of 3 OK created incremental model source.dp_indicator_daily........................................................... [INSERT 0 0 in 10.00s] 2022-06-29 06:26:33 normalization > 06:26:33 2022-06-29 06:26:33 normalization > 06:26:33 Finished running 1 view model, 2 incremental models in 26.02s. 2022-06-29 06:26:33 normalization > 06:26:33 2022-06-29 06:26:33 normalization > 06:26:33 Completed successfully 2022-06-29 06:26:33 normalization > 06:26:33 2022-06-29 06:26:33 normalization > 06:26:33 Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3 2022-06-29 06:26:33 INFO i.a.w.g.DefaultNormalizationWorker(run):73 - Normalization executed in 34 seconds. 2022-06-29 06:26:33 INFO i.a.w.g.DefaultNormalizationWorker(run):79 - Normalization summary: io.airbyte.config.NormalizationSummary@daeb303[startTime=1656483959347,endTime=1656483993391] 2022-06-29 06:26:33 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling... 2022-06-29 06:26:33 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):237 - Stopping temporal heartbeating...