Airbyte is naming the fetched stream from the S3 source with the dataset which is wrong as dataset is the name of table at destination

Troubleshooting
#1

Hi,

I’ve built up connection from S3 to redshift.
My Amazon S3 bucket has a csv file that i am trying to replicate but when it fetches the schema from the source it names it as dataset name given in source configuration instead of the actual file name.
is this the expected behavior? Because my understanding for the field “Dataset” is that it is the name of the table at the destination but here it is also naming the source stream with the same name

My file in AMAZON S3 BUCKET

image
image793×54 3.9 KB

Source configs settings > Here you can observe that dataset is named as “airbyte_dataset”

image
image776×877 84.7 KB

Now fetched stream is named same instead of the actual file name that i have in my bucket

image
image1174×805 64.4 KB

#2

Hi @MahakUrooj,
The S3 connector is made to sync multiple files with the same schema, and gather these files into the logic of streams.

Now fetched stream is named same instead of the actual file name that i have in my bucket

You have set the dataset name manually and this connector can only handle a single stream, so this behavior is the expected one.

Let me know if you have additional questions. Is something blocking you with the current connector behavior?