Is this your first time deploying Airbyte : No Deployment : Docker Airbyte Version : 0.35.3-alpha Source name/version : Redshift Destination name/version : Snowflake Step : Schema validation Description :
Hi all, I have successfully loaded data from Redshift to Snowflake already, but I am experiencing a JSON schema validation error when trying to load from another cluster. This does not seem to be a normalization issue, as it shows as completing successfully in the logs, which are attached. Thanks very much.
Hi @harshith I am trying to download through the UI but nothing is happening when I click on the buttons. Is there a way that I can do so through the command line? Sorry if this is a silly question, but not seeing a way that other people have done so in the Slack chat, although a few people have reported the same issue with the logs files not downloading locally when the buttons are clicked.
Hi @yli can you check whether you have this line in your logs Invalid operation: permission denied for schema?
Our problem ended up being that the person who set up the source did not specify the schema / namespace, and we only had permissions to access one schema / namespace across the cluster, so we hit permissions issues that were resolved once we specified the schema. Even if you only have objects from the schema(s) that you have permissions for specified in your connection, it will still throw this error.
UPDATE: Just checked your logs and I don’t see that line but not specifying your source schema could possibly still be the issue here. Hopefully this helps.
@harshith please see my response to @yli above. In digging deeper into the logs, we realized that the problem was not actually the JSON schema validation failure, but that it was returning as such because there was an underlying permissions error that prevented the connection from accessing the object. Once we resolved by specifying the schema in the source, the connection successfully ran. This issue can be closed.
Perhaps a permission denied for schema issue could be flagged more explicitly as an error for these logs?
@kev1021 there’s no such log in the job. The connection reads the sample data from snowflake and loads to postgres (tpch_sf1.nation). I didn’t run dbt or other transformation. I checkecd the query history in snowflake, all queries are run successfully.
The dataset is very small, only 25 rows, 4 columns. And the error doesn’t block me to complete the sync. I just got the error of JSON schema validation failed, and the validation says the configuration on source-snowflake is wrong at credentials, though data is read correctly.