Cannot invoke “com.fasterxml.jackson.databind.node.ObjectNode.put(String, String)” because “matchingSchema”

Hi,

I just installed airbyte on my google cloud vm. I have added 1 source ( app store ) and 1 destination ( bigquery ) but the sync always fails with this error:

2022-08-08 10:43:36 INFO i.a.w.g.DefaultReplicationWorker(run):269 - Source did not output any state messages

2022-08-08 10:43:36 WARN i.a.w.g.DefaultReplicationWorker(run):277 - State capture: No new state, falling back on input state: io.airbyte.config.State@3f32a2b3[state={}]

2022-08-08 10:43:36 INFO i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling…

2022-08-08 10:43:36 INFO i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$3):161 - sync summary: io.airbyte.config.StandardSyncOutput@1dbf2aa4[standardSyncSummary=io.airbyte.config.StandardSyncSummary@1688c8fb[status=failed,recordsSynced=0,bytesSynced=0,startTime=1659955344881,endTime=1659955416774,totalStats=io.airbyte.config.SyncStats@2c1bd589[recordsEmitted=0,bytesEmitted=0,stateMessagesEmitted=0,recordsCommitted=0],streamStats=],normalizationSummary=,state=io.airbyte.config.State@3f32a2b3[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@d656e16[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@75afa72a[stream=io.airbyte.protocol.models.AirbyteStream@11a14931[name=basmosales_report,jsonSchema={“type”:[“null”,“object”],“properties”:{“cmb”:{“type”:[“null”,“string”]},“sku”:{“type”:[“null”,“string”]},“title”:{“type”:[“null”,“string”]},“units”:{“type”:[“null”,“integer”]},“client”:{“type”:[“null”,“string”]},“device”:{“type”:[“null”,“string”]},“period”:{“type”:[“null”,“string”]},“version”:{“type”:[“null”,“string”]},“_line_id”:{“type”:[“null”,“integer”]},“category”:{“type”:[“null”,“string”]},“end_date”:{“type”:[“null”,“string”],“format”:“date”},“provider”:{“type”:[“null”,“string”]},“developer”:{“type”:[“null”,“string”]},“begin_date”:{“type”:[“null”,“string”],“format”:“date”},“order_type”:{“type”:[“null”,“string”]},“promo_code”:{“type”:[“null”,“string”]},“country_code”:{“type”:[“null”,“string”]},“subscription”:{“type”:[“null”,“string”]},“customer_price”:{“type”:[“null”,“string”]},“_time_extracted”:{“type”:[“null”,“string”],“format”:“date-time”},“proceeds_reason”:{“type”:[“null”,“string”]},“_api_report_date”:{“type”:[“null”,“string”],“format”:“date”},“apple_identifier”:{“type”:[“null”,“string”]},“provider_country”:{“type”:[“null”,“string”]},“customer_currency”:{“type”:[“null”,“string”]},“parent_identifier”:{“type”:[“null”,“string”]},“preserved_pricing”:{“type”:[“null”,“string”]},“developer_proceeds”:{“type”:[“null”,“string”]},“supported_platforms”:{“type”:[“null”,“string”]},“currency_of_proceeds”:{“type”:[“null”,“string”]},“product_type_identifier”:{“type”:[“null”,“string”]}}},supportedSyncModes=[incremental],sourceDefinedCursor=true,defaultCursorField=,sourceDefinedPrimaryKey=,namespace=,additionalProperties={}],syncMode=incremental,cursorField=,destinationSyncMode=append_dedup,primaryKey=[[_line_id]],additionalProperties={}]],additionalProperties={}],failures=[io.airbyte.config.FailureReason@413e84ee[failureOrigin=replication,failureType=,internalMessage=java.lang.RuntimeException: java.lang.NullPointerException: Cannot invoke “com.fasterxml.jackson.databind.node.ObjectNode.put(String, String)” because “matchingSchema” is null,externalMessage=Something went wrong during replication,metadata=io.airbyte.config.Metadata@678d5701[additionalProperties={attemptNumber=1, jobId=14}],stacktrace=java.util.concurrent.CompletionException: java.lang.RuntimeException: java.lang.NullPointerException: Cannot invoke “com.fasterxml.jackson.databind.node.ObjectNode.put(String, String)” because “matchingSchema” is null

at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)

at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)

at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)

at java.base/java.lang.Thread.run(Thread.java:1589)

Caused by: java.lang.RuntimeException: java.lang.NullPointerException: Cannot invoke “com.fasterxml.jackson.databind.node.ObjectNode.put(String, String)” because “matchingSchema” is null

at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getReplicationRunnable$6(DefaultReplicationWorker.java:363)

at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)

… 3 more

Caused by: java.lang.NullPointerException: Cannot invoke “com.fasterxml.jackson.databind.node.ObjectNode.put(String, String)” because “matchingSchema” is null

at io.airbyte.workers.RecordSchemaValidator.validateSchema(RecordSchemaValidator.java:48)

at io.airbyte.workers.general.DefaultReplicationWorker.validateSchema(DefaultReplicationWorker.java:384)

at io.airbyte.workers.general.DefaultReplicationWorker.lambda$getReplicationRunnable$6(DefaultReplicationWorker.java:313)

… 4 more

,retryable=,timestamp=1659955353994]]]

Hey could you share the complete sync log here as a file?

Hello I have the same problem while testing my custom connector.
I uploaded my logs.
logs-35.txt (40.3 KB)
On my API connector I’m building the schema dynamically.
Airbyte version 40.0
Docker on windows 11.
Custom connector on python.

Replied on this issue

logs-31.txt (73.4 KB)

Tables are created in bigquery, but they miss the data in them.

The opened issue is this

Hey could you help me with information.

  1. What is the configured schema in the connection? may be a screenshot?
  2. What is the output of one of the record? If you can share the json here