Trouble building connection with SFTP source and Oracle DB destination

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

User is facing issues building a connection with SFTP source and Oracle DB destination in Airbyte. Files in SFTP folder are not being picked up by the connection builder.


Question

Hey guys, i am trying to get to know airbyte and i have a sftp source and an oracle db destination configured. Now i am trying to build a connection. But it needs streams from the source. I have a json and a csv file in the sftp folder that airbyte could access, but nothing is picked up in the connections builder. What am i missing?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["sftp-source", "oracle-db-destination", "connection-builder", "json-file", "csv-file"]
2

Have you tried to set Optional fields for SFTP source connector?

3

Its configured same as your screenshot

4

Ok, i forgot the files are in a sub-folder called /Source. I have added that to the Folder Path field and now i get this error message when trying to select stream

5

Hm, i think it has something to do with my csv file. If i remove that and only leave the json file there it works

6

yeah, I guess it cannot parse this file properly
https://github.com/airbytehq/airbyte/blob/59e981a72915dc57461250c62b60d3a8a04b5b0b/airbyte-integrations/connectors/source-sftp/src/main/java/io/airbyte/integrations/source/sftp/SftpCommand.java#L104-L113|SftpCommand.java#L104-L113

7

Ok, is there some information on how the csv file should be structured? Headers, delimiters, etc?

8

You need to check code https://github.com/airbytehq/airbyte/blob/59e981a72915dc57461250c62b60d3a8a04b5b0b/airbyte-integrations/connectors/source-sftp/src/main/java/io/airbyte/integrations/source/sftp/parsers/CsvFileParser.java#L36-L40|CsvFileParser.java#L36-L40
It looks like whatever is detected.
Because there are no other options like for some other connectors, I guess the safest option is to have header and comma separated values.