Source Hubspot 0.1.53: ticket_pipelines stream with wrong data type

RemyTinco · April 13, 2022, 9:32am

Is this your first time deploying Airbyte?: No
OS Version / Instance: Debian
Memory / Disk: 12 GB
Deployment: Kubernetes deployment
Airbyte Version: 0.35.64-alpha
Source name/version: Hubspot
Destination name/version: Bigquery
Step: Sync
Description:

Seems I got 2 errors in models that I’m currently investigating on my side.
But there is also a strange error that fails the JSON Schema validation?

Here are the end logs =>
logs-90.txt (4.4 MB)

2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.102712 [debug] [MainThread]: Connection 'model.airbyte_utils.engagements_metadata_bcc' was properly closed.
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.188253 [info ] [MainThread]: 
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.188673 [info ] [MainThread]: e[31mCompleted with 2 errors and 0 warnings:e[0m
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.189019 [info ] [MainThread]: 
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.189306 [error] [MainThread]: e[33mDatabase Error in model ticket_pipelines (models/generated/airbyte_tables/hubspot_/ticket_pipelines.sql)e[0m
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.189600 [error] [MainThread]:   Bad int64 value: 1970-01-01T00:00:00Z
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.189853 [error] [MainThread]:   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/hubspot_/ticket_pipelines.sql
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.190107 [info ] [MainThread]: 
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.190381 [error] [MainThread]: e[33mDatabase Error in model contacts_properties (models/generated/airbyte_incremental/hubspot_/contacts_properties.sql)e[0m
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.190657 [error] [MainThread]:   Invalid date: '1 an et demi'
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.191009 [info ] [MainThread]: 
2022-04-13 05:22:22 e[42mnormalizatione[0m > 05:22:18.191296 [info ] [MainThread]: Done. PASS=73 WARN=0 ERROR=2 SKIP=2 TOTAL=77
2022-04-13 05:22:22 e[32mINFOe[m i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
errors: $.credential: is not defined in the schema and the schema does not allow additional properties, $.part_size_mb: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_name: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_path: is not defined in the schema and the schema does not allow additional properties, $.keep_files_in_gcs-bucket: is not defined in the schema and the schema does not allow additional properties, $.method: must be a constant value Standard
2022-04-13 05:22:22 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(lambda$getWorkerThread$2):158 - Completing future exceptionally...
io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	... 3 more
	Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
		at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:160) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:45) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:833) [?:?]
2022-04-13 05:22:22 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling...
2022-04-13 05:22:22 e[32mINFOe[m i.a.w.t.TemporalUtils(withBackgroundHeartbeat):235 - Stopping temporal heartbeating...
2022-04-13 05:22:22 e[33mWARNe[m i.t.i.a.POJOActivityTaskHandler(activityFailureToResult):307 - Activity failure. ActivityId=b3bf8196-7d42-34a5-9555-0704916657f7, activityType=Normalize, attempt=1
java.lang.RuntimeException: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:233) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.normalize(NormalizationActivityImpl.java:71) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at jdk.internal.reflect.GeneratedMethodAccessor510.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:214) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:180) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.activity.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:120) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:204) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:164) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:93) ~[temporal-sdk-1.8.1.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.temporal.serviceclient.CheckedExceptionWrapper: java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.temporal.serviceclient.CheckedExceptionWrapper.wrap(CheckedExceptionWrapper.java:56) ~[temporal-serviceclient-1.8.1.jar:?]
	at io.temporal.internal.sync.WorkflowInternal.wrap(WorkflowInternal.java:448) ~[temporal-sdk-1.8.1.jar:?]
	at io.temporal.activity.Activity.wrap(Activity.java:51) ~[temporal-sdk-1.8.1.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:135) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$1(NormalizationActivityImpl.java:97) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:228) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	... 13 more
Caused by: java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed.
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:129) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.sync.NormalizationActivityImpl.lambda$normalize$1(NormalizationActivityImpl.java:97) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalUtils.withBackgroundHeartbeat(TemporalUtils.java:228) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	... 13 more
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	... 1 more
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	... 1 more
	Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
		at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:160) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:45) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:18) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:833) [?:?]

And btw, thanks for this discourse!

marcosmarxm · April 13, 2022, 7:10pm

The JSON validation is only an info don’t need to be worry about it.
What version of the Hubspot connector version are you using?

RemyTinco · April 27, 2022, 9:26am

Ok, we’re using version 0.1.52

marcosmarxm · April 27, 2022, 10:58pm

See issue: Hubspot Connector 0.1.53 uses incorrect ticket_pipeline date type · Issue #12318 · airbytehq/airbyte · GitHub

marcosmarxm · April 28, 2022, 4:01pm

@RemyTinco can you update to version 0.1.53?

marcosmarxm · July 13, 2022, 12:00am

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.

Topic		Replies	Views
Hubspot Normalization Failure Connector Questions & Issues normalization	5	661	July 11, 2022
Hubspot Connector 0.1.59 failing on date field Connector Questions & Issues data-loading , connectors	4	261	July 14, 2022
HubSpot deal_pipelines endpoint missing rows on sync Connector Questions & Issues source-hubspot , connectors	4	400	July 14, 2022
Hubspot to Postgres Not Supporting Normalization Connector Questions & Issues source-hubspot , destination-postgres , normalization	1	895	February 12, 2023
Hubspot -- >> Snowflake connection issue Connector Questions & Issues connectors	3	232	July 14, 2022

Source Hubspot 0.1.53: ticket_pipelines stream with wrong data type

Related topics