Hello, I have a question after using the airbite. The data connector environment was tested in a 32 cpu and 128 gb memory environment. Data with a table size of 750mb and a record of 6.6 million was converted into csv in s3, and it took about 10 minutes. I tried modifying the number of threads and workers in the .env file, but there was no significant difference. I ask if it takes this long in terms of performance, and if there is a way to tune performance, I hope to hear from you soon. Thank you!
The worker configuration you changed in .env file only will increase/decrease the number of parallel syncs you’ll run. The performance of each connector depends of the connector configuration and the API you’re trying to replicate.