Worday-BigQuery Sync succeeded with errors

Connector Questions & Issues
1
  • Is this your first time deploying Airbyte?: Yes
  • OS Version / Instance: MacOS
  • Memory / Disk: 4Gb
  • Deployment: Docker
  • Airbyte Version: 0.38.4-alpha
  • Source name/version: Workday that I built
  • Destination name/version: BigQuery
  • Step: Sync completed but with error
2022-07-11 20:28:11 destination > 2022-07-11 20:28:11 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword

2022-07-11 20:28:11 destination > 2022-07-11 20:28:11 WARN c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword

2022-07-11 20:28:11 destination > 2022-07-11 20:28:11 INFO i.a.i.d.b.BigQueryUtils(getLoadingMethod):303 - Selected loading method is set to: GCS

2022-07-11 20:30:51 normalization > 20:30:51  1 of 1 START incremental model airbyte_workday_test.report.............................................................. [RUN]
2022-07-11 20:30:52 normalization > 20:30:52  20:30:52 + `unity-data-idh-test`.airbyte_workday_test.`report`._airbyte_ab_id does not exist yet. The table will be created or rebuilt with dbt.full_refresh
2022-07-11 20:30:56 normalization > 20:30:56  1 of 1 OK created incremental model airbyte_workday_test.report......................................................... [CREATE TABLE (3.5k rows, 2.9 MB processed) in 5.26s]
2022-07-11 20:30:56 normalization > 20:30:56  
2022-07-11 20:30:56 normalization > 20:30:56  Finished running 1 incremental model in 9.51s.
2022-07-11 20:30:56 normalization > 20:30:56  
2022-07-11 20:30:57 normalization > 20:30:56  Completed successfully
2022-07-11 20:30:57 normalization > 20:30:57  
2022-07-11 20:30:57 normalization > 20:30:57  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
2022-07-11 20:30:57 INFO i.a.w.DefaultNormalizationWorker(run):71 - Normalization executed in 1 minute 32 seconds.
2022-07-11 20:30:57 INFO i.a.w.DefaultNormalizationWorker(run):77 - Normalization summary: io.airbyte.config.NormalizationSummary@45049e30[startTime=1657571364713,endTime=1657571457410]
2022-07-11 20:30:57 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...
2022-07-11 20:30:57 INFO i.a.w.t.TemporalUtils(withBackgroundHeartbeat):236 - Stopping temporal heartbeating...
**2022-07-11 20:30:57 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed.** 
errors: $.credential: is not defined in the schema and the schema does not allow additional properties, $.part_size_mb: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_name: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_path: is not defined in the schema and the schema does not allow additional properties, $.keep_files_in_gcs-bucket: is not defined in the schema and the schema does not allow additional properties, $.method: must be a constant value Standard
2022-07-11 20:30:57 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
**errors: $.credential.hmac_key_access_id: object found, string expected, $.credential.hmac_key_secret: object found, string expected**
2

Hey do you see this sync as error in the UI? The error we see is just warning errors and ideally do not affect the sync

3

I got this error on [http://localhost:8000], that the UI right?
Do these warnings affect future sync result?

4

I didn’t get what is that error you have got. Is it possible to share a screenshot of the UI? Ideally they don’t effect later syncs

5

After I added a new source and rerun the connector, I got thousands of these DateTimeValidator ERRORs, but I still got the synced report in destination and the field with date-time schema looks fine.
It also said the record schema validation failed for report, but when I test it locally, it works fine without any error

image
image1449×414 107 KB

6

Gotcha so it is resolved?

7

Nope, it still have the DateTimeValidator error

8

I would suggest to change it to string and convert into date-time using custom DBT