Summary
Error occurs when extracting large dataset from Pendo integration in Airbyte, with message ‘The CDK command test_read completed properly but no records nor trace were found. Logs were: 0.’
Question
Hi!!
I have been working on Pendo integration via Airbyte,
I have been able to extract data for smaller dataset but when it comes to a large dataset the integration fails with error
error : The CDK command test_read completed properly but no records nor trace were found. Logs were: 0.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["pendo-integration", "airbyte", "large-dataset", "error"]
Yeah, that’s a bug. What happened is that your test_read received quite a bit of data, and our infra could not deserialize it. We’re working on a fix for this as we speak, so you’d be able to test larger reads.
In the meantime, I expect that later this week we ship controls for how many records and pages to fetch during a test_read, so you should be able to test the behavior you want on smaller subset of data and that should work successfully. Right now, Builder sets records limit to a 1000, and if those have nested records, things could get big enough to cause the above bug AFAIK.
reader was able to deserialize for smaller dataset and I released it, since then the data has not coming in through the integration.
If Test does not work but new release should work?
Does the love connection work? If not, ping us with logs, we can take a look at what’s broken. But yeah, generally once you get it to work in the builder and deploy it, it should be fine even if test reads in the builder fail with the message that you posted.
yikes autocorrect, live connection in your workspace actually running
this is the logs, if you notice the size if is in KB’s . Earlier sync was failing with 197MB data
Oh, so it worked for a while, and then syncs started being bigger, 5 records over 180 megs, and it’s failing?
5 records are basically 5 streams
another problem I faced was, the stream did not flat
it was just extracting data in json
tried multiple ways but did not work