Summary
The error ‘No properties node in stream schema’ is occurring when trying to run a connection with a YAML connector. It seems to be related to field selection in the replication worker.
Question
Successfully deployed YAML connector, but after successfully setting up and trying to run the connection I get this error on the worker: java.lang.RuntimeException: No properties node in stream schema\n\tat io.airbyte.workers.internal.FieldSelector.populatedStreamToSelectedFields(FieldSelector.java:150)\n\tat io.airbyte.workers.internal.FieldSelector.populateFields(FieldSelector.java:64)\n\tat io.airbyte.workers.general.ReplicationWorkerHelper.startSource(ReplicationWorkerHelper.kt:199)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$run$1(BufferedReplicationWorker.java:153)\n\tat io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsync$2(BufferedReplicationWorker.java:228)\n\tat java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\n
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["error", "java.lang.RuntimeException", "no-properties-node", "stream-schema", "yaml-connector", "replication-worker"]
The error message you’re seeing, java.lang.RuntimeException: No properties node in stream schema, indicates that there’s an issue with the schema of your data stream. This error typically occurs when the schema of the data being processed does not match the expected schema. From the information provided in the extracts, it seems that this issue has been encountered in various contexts, such as when creating a new custom low code connector (source), setting up an Apify Source Connector (source), and using the Connector Builder (source). In the case of the Connector Builder, the issue was due to the Inline schema being empty. The expected behavior is that the user should not be able to publish a connector that will fail due to an empty schema. In the case of the Apify Source Connector, the error was due to the connector not being able to recognize the dataset schema. Unfortunately, without more specific information about your YAML connector and the data you’re trying to process, it’s difficult to provide a more precise solution. However, you should check the schema of your data stream and ensure it matches the expected schema for your connector. If you’re using a custom connector, make sure you’ve defined the schema correctly. If you’re using a pre-built connector, you may need to adjust the data you’re sending to match the expected schema. If you continue to encounter issues, I would recommend reaching out to the Airbyte community on their forum or Slack for more specific assistance.
I was missing a schema file for one of the streams; problem solved… thanks