Hi everyone.
I’m trying to ingest a custom model from Zoho CRM, but it is always raising the IncompleteMetaDataException error. The weird thing is that some custom models are being loaded, but the one we need only loads 2 old test records and the table name comes with the “x” prefix, which I believe has something to do with incremental sync. We are using a super admin account with all the permissions listed on the documentation. We’ve also tried to add the stream on the configured_catalog.json and the test_stream_factory.py (even if from what I understand it should build the schemas automatically) and rebuild the image, but it did not work.
Am I missing some config or, since the connector is still on alpha, do I have to write new code to pull the custom modules?
Log with the 204 error (I’ll be happy to share more logs if needed):
You can find some information about it here: https://github.com/airbytehq/airbyte/blob/bb469418679e1605689d994a57cf59b1095c677d/airbyte-integrations/connectors/source-zoho-crm/source_zoho_crm/streams.py#L49-L65
Probably because the connector is alpha and our integration account doesn’t have too many data records we’re missing something for this connector.
But your problem is that is not bringing all data and the table/stream is created right?
Yeah, there are 2 main problems:
1: It is unable to retrieve the metadata of some custom modules, so the stream is not created
2: Some custom modules that were created with the “x” prefix are not loading all the rows
About the first one, I do not think it is a permission issue, since other modules are loaded and I’m using the admin account. Maybe there is some configuration issue on zoho, in that case I was thinking about “hardcoding” the schema so it can build the stream.
About the second, I was not able to find in the code the exact meaning of the prefix, so I don’t have any good hypothesis.
(schedule a call with Daniel next week to test the changes in Debezium parameters)
