Hey Team,
I’m running into an issue with regards to source acceptance test.
When I do source acceptance test, it fails and throws me this message.
My understanding is that the two sequential reads should be the same, but the end point i’m hitting to retrieve the data it keeps updating every minute around 150k+ records are emitted and it keeps growing.
So how to do I proceed ahead with this test?
Hey Team,
any update on how to proceed ahead?
Hey @Yash, for an incremental sync: the output of two sequential reads could be different or may be the same (if there have been no updates since the first read.)
I’m not sure if you’ve taken a look at our documentation but you can find a description for what this test entails here: https://docs.airbyte.com/connector-development/testing-connectors/source-acceptance-tests-reference/#testtwosequentialreads
Let me know if it makes sense or if you have any questions!
Hey @sajarin,
The custom connector I’m using is a full refresh overwrite, I’m not gonna be doing incremental.
and the error I have it is in TestFullRefresh::test_sequential_reads[inputs0]?
can you let me know how to debug them.
Hey @Yash,
Sorry about that, I read your debug message incorrectly. You are right: the two sequential reads for a full refresh overwrite should be the same, but you can also check to see if the first read is a strict subset of the second read.
For the endpoint you’re hitting the data is always completely different between each read? Or is the data in the first read included in the second read?
Hey @sajarin,
I checked the number of records, and it is the same sometime and sometime as a new record is inserted the inputs do not match?
Can I submit a PR and the Airbyte team can help me debug this issue?
Hey @Yash,
By all means, please submit an PR with your connector work so far and hopefully we can help debug your issue and get a new connector merged.
Here’s the link to the repo for your convenience: https://github.com/airbytehq/airbyte/pulls
