Teradata destination connector - Run write command using docker

I’m currently building java destination connector for teradata. Implementing destination with AbstractJdbcDestination as teradata supports JDBC connections. I am able to run spec, check and write directly from docker. When I ran write command with file message.jsonl file, content was not loading to teradata table. The following command I am using to load data to teradata.

cat messages.jsonl | docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/sample_files:/sample_files airbyte/destination-teradata:dev write --config /secrets/config.json --catalog /sample_files/configured_catalog.json

Please find the messages.jsonl and configured_catalog.json files.

airbyte/airbyte-integrations/connectors/destination-teradata/sample_files/configured_catalog.json

airbyte/airbyte-integrations/connectors/destination-teradata/messages.jsonl

Hey @SatishChGit, thanks for your patience while I look into this! Were you following a specific tutorial/piece of documentation while working on this?

1 Like

Yes @natalyjazzviolin I followed documentation Building a Java Destination | Airbyte Documentation

Are you handling exceptions with the AirbyteTraceMessage class?
https://docs.airbyte.com/connector-development/tutorials/building-a-java-destination/#handling-exceptions

This seems like it should raise an exception, since the connector is not running as expected.

Yes I handled exceptions as per documentation. But I don’t see any exceptions. attached output logs for write command execution.

IssueLog.txt (6.6 KB)

Here the implemented class of teradata for your reference.
airbyte/TeradataSqlOperations.java at teradata_destination · SatishChGit/airbyte (github.com)

What I observed from logs is Implementation of AirbyteMessageConsumer - accept method is not calling. Only start and close methods are executing.

I found this doc:
https://docs.airbyte.com/understanding-airbyte/airbyte-protocol/?__hstc=27854691.41ad55c5e69bffd5cbc92e8078bb174c.1662731983053.1665592279594.1665669814594.38&__hssc=27854691.7.1665669814594&__hsfp=2875269484#write

It states that “The destination should ignore fields or streams that are out of sync with the catalog.”

Could you double check that all streams/fields are in the catalog?