How to debug connections

Hi there :slightly_smiling_face:
Im having a problem and I will like to debug it to understand whats going on.
I builded a custom connector (source)
Then I created my connection from that source to Local CSV destination and it works!
On my second custom connector(source)
I did the same steps but now I get an error on writing to the destination:

at java.util.concurrent.CompletableFuture.reportGet( ~[?:?]
	at java.util.concurrent.CompletableFuture.get( ~[?:?]
	at ~[io.airbyte-airbyte-workers-0.36.3-alpha.jar:?]
	at ~[io.airbyte-airbyte-workers-0.36.3-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2( ~[io.airbyte-airbyte-workers-0.36.3-alpha.jar:?]
	at [?:?]
	Suppressed: io.airbyte.workers.WorkerException: Destination process exit with code 1. This warning is normal if the job was cancelled.
		at io.airbyte.workers.protocols.airbyte.DefaultAirbyteDestination.close( ~[io.airbyte-airbyte-workers-0.36.3-alpha.jar:?]
		at ~[io.airbyte-airbyte-workers-0.36.3-alpha.jar:?]
		at ~[io.airbyte-airbyte-workers-0.36.3-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2( ~[io.airbyte-airbyte-workers-0.36.3-alpha.jar:?]
		at [?:?]
Caused by: io.airbyte.workers.DefaultReplicationWorker$DestinationException: Destination process exited with non-zero exit code 1
	at io.airbyte.workers.DefaultReplicationWorker.lambda$getDestinationOutputRunnable$6( ~[io.airbyte-airbyte-workers-0.36.3-alpha.jar:?]
	at java.util.concurrent.CompletableFuture$ ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker( ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$ ~[?:?]
	... 1 more

So is there a way to debug locally a connection? Thanks!

Hey @theArg,
To make sure your source connectors read work you can run it locally on your connector container:
docker run --rm -i airbyte/source-<your-source>:dev read --config airbyte-integrations/connectors/source-<your-source>/secrets/config.json --catalog airbyte-integrations/connectors/source-<your-source>/integration_tests/catalog.json

Let me know what this command outputs :pray:

Hi @alafanechere,
The source works as expected but when I create a connection from my second custom source to LocalCSV, there is when it fails.

By running the read command I already see the records being emitted, Im interested on debugging a connection (source → destination) to see why he is not able to write.

Could you please share your full sync logs and your catalog + configured catalog?


  "streams": [
      "stream": {
        "name": "jrct",
        "json_schema": {
          "$schema": "",
          "title": "JRCT clinical trials",
          "type": "object",
          "supported_sync_modes": [
          "properties": {
            "raw": {
              "type": "string"
      "sync_mode": "full_refresh",
      "destination_sync_mode": "overwrite"

Sync logs

logs-50.txt (65.2 KB)

Your connector outputs records on a stream called raw which is not declared in the catalog.
It correspond to the following error in your logs:
Exception in thread "main" java.lang.IllegalArgumentException: Message contained record from a stream that was not in the catalog.

You need to declare all your stream’s schema in your catalog.json

I had the wrong name of the stream in the discover. Now its working, many thanks!

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.