Hubspot Connector 0.1.59 failing on date field

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: GKE via HELM
  • Memory / Disk: n/a
  • Deployment: Kubernetes
  • Airbyte Version: 0.39.1-alpha
  • Source name/version: Hubspot/0.1.59
  • Destination name/version: BigQuery/1.1.16
  • Step: sync
  • Description:

2022-05-26 04:04:17 ERROR c.n.s.DateTimeValidator(isLegalDateTime):70 - Failed to apply RFC3339 pattern on 2021-01-05 05:00:00

I basically have like 800,000+ of these errors in the log

Additional Errors (the log file is too big for me to attach and it won’t allow me to upload .zip files) Please let me know if sending the full log file would be helpful

My guess for this one is that api rate limit has been exceeded? Not sure though…

2022-05-26 04:38:32 e[44msourcee[0m > Syncing stream: feedback_submissions 
2022-05-26 04:38:32 e[44msourcee[0m > Encountered an exception while reading stream SourceHubspot
Traceback (most recent call last):
  File "/airbyte/integration_code/source_hubspot/streams.py", line 340, in read_records
    stream_records, response = self._read_stream_records(
  File "/airbyte/integration_code/source_hubspot/streams.py", line 311, in _read_stream_records
    response = self.handle_request(
  File "/airbyte/integration_code/source_hubspot/streams.py", line 288, in handle_request
    response = self._send_request(request, request_kwargs)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 337, in _send_request
    return backoff_handler(user_backoff_handler)(request, request_kwargs)
  File "/usr/local/lib/python3.9/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/streams/http/http.py", line 299, in _send
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.hubapi.com/crm/v3/objects/feedback_submissions?hapikey=*****&limit=100&startTimestamp=1485302400000&endTimestamp=1653539911984&archived=false&associations=contacts&properties=hs_agent_email%2Chs_agent_id%2Chs_agent_name%2Chs_all_accessible_team_ids%2Chs_all_assigned_business_unit_ids%2Chs_all_owner_ids%2Chs_all_team_ids%2Chs_chatflow_name%2Chs_chatflow_object_id%2Chs_contact_id%2Chs_conversation_thread_id%2Chs_created_by_user_id%2Chs_createdate%2Chs_lastmodifieddate%2Chs_merged_object_ids%2Chs_object_id%2Chs_unique_creation_key%2Chs_updated_by_user_id%2Chs_user_ids_of_all_notification_followers%2Chs_user_ids_of_all_notification_unfollowers%2Chs_user_ids_of_all_owners%2Chubspot_owner_assigneddate%2Chubspot_owner_id%2Chubspot_team_id%2Chs_industry_standard_question_type%2Chs_sentiment%2Chs_survey_id%2Chs_survey_type%2Chs_survey_channel%2Chs_submission_timestamp%2Chs_value%2Chs_response_group%2Chs_content%2Chs_ingestion_id%2Chs_knowledge_article_id%2Chs_visitor_id%2Chs_engagement_id%2Chs_submission_url%2Chs_survey_name%2Chs_form_guid%2Chs_contact_email_rollup%2Chs_submission_name

For this one - I’m assuming it’s bleed-over from the date field error I saw earler…

2022-05-26 04:39:24 e[1;31mERRORe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null
2022-05-26 04:39:27 e[32mINFOe[m i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
errors: $: null found, object expected
2022-05-26 04:39:27 e[1;31mERRORe[m i.a.w.i.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null
2022-05-26 04:39:33 e[32mINFOe[m i.a.w.p.KubePodProcess(destroy):658 - (pod: airbyte / destination-bigquery-write-74-2-tcrfc) - Destroying Kube process.
2022-05-26 04:39:33 e[32mINFOe[m i.a.w.p.KubePodProcess(close):713 - (pod: airbyte / destination-bigquery-write-74-2-tcrfc) - Closed all resources for pod
2022-05-26 04:39:33 e[32mINFOe[m i.a.w.p.KubePodProcess(destroy):664 - (pod: airbyte / destination-bigquery-write-74-2-tcrfc) - Destroyed Kube process.
2022-05-26 04:39:33 e[33mWARNe[m i.a.c.i.LineGobbler(voidCall):86 - airbyte-destination gobbler IOException: Socket closed. Typically happens when cancelling a job.
2022-05-26 04:39:33 e[1;31mERRORe[m i.a.w.g.DefaultReplicationWorker(run):177 - Sync worker failed.
java.util.concurrent.ExecutionException: io.airbyte.workers.general.DefaultReplicationWorker$SourceException: Source cannot be stopped!
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:170) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:62) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
	Suppressed: io.airbyte.workers.exception.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled.
		at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
		at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:134) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
		at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:62) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:833) [?:?]
	Suppressed: io.airbyte.workers.exception.WorkerException: Destination process exit with code 143. This warning is normal if the job was cancelled.
		at io.airbyte.workers.internal.DefaultAirbyteDestination.close(DefaultAirbyteDestination.java:119) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
		at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:134) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
		at io.airbyte.workers.general.DefaultReplicationWorker.run(DefaultReplicationWorker.java:62) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:158) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.general.DefaultReplicationWorker$SourceException: Source cannot be stopped!
	at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getReplicationRunnable$6(DefaultReplicationWorker.java:341) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	... 1 more
Caused by: io.airbyte.workers.exception.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled.
	at io.airbyte.workers.internal.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
	at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getReplicationRunnable$6(DefaultReplicationWorker.java:339) ~[io.airbyte-airbyte-workers-0.39.1-alpha.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	... 1 more
2022-05-26 04:39:33 e[32mINFOe[m i.a.w.g.DefaultReplicationWorker(run):236 - sync summary: io.airbyte.config.ReplicationAttemptSummary@45a740c0[status=failed,recordsSynced=346752,bytesSynced=3946005833,startTime=1653537848138,endTime=1653539973062,totalStats=io.airbyte.config.SyncStats@66acae24[recordsEmitted=346752,bytesEmitted=3946005833,stateMessagesEmitted=0,recordsCommitted=0],streamStats=[io.airbyte.config.StreamSyncStats@3cd597e2[streamName=contact_lists,stats=io.airbyte.config.SyncStats@51d57bb4[recordsEmitted=36,bytesEmitted=54282,stateMessagesEmitted=<null>,recordsCommitted=<null>]], io.airbyte.config.StreamSyncStats@2e0b900e[streamName=email_events,stats=io.airbyte.config.SyncStats@2dd5c665[recordsEmitted=2373,bytesEmitted=1180873,stateMessagesEmitted=<null>,recordsCommitted=<null>]], 

…etc

2022-05-26 04:04:17 ERROR c.n.s.DateTimeValidator(isLegalDateTime):70 - Failed to apply RFC3339 pattern on 2021-01-05 05:00:00

There is an issue to solve this problem: https://github.com/airbytehq/airbyte/issues/12972

Are you able to cURL the 403 endpoint manually to see the response from Hubspot about the issue?