Summary
Error message ‘Internal message: java.lang.NullPointerException: getSQLState(…) must not be null’ when using Redshift destination. Data is being populated in airbyte_internal schema and written to S3.
Question
I’m having trouble getting the Redshift destination to work. I keep getting an error Internal message: java.lang.NullPointerException: getSQLState(...) must not be null I’m not seeing anything in cloudtrail, the airbyte_internal schema is being populated with information and I’m seeing objects written to S3. Anyone have any suggestion as to what’s going on here?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["redshift-destination", "error", "null-pointer-exception", "airbyte-internal", "s3"]
Here’s the log output
2024-07-19 22:42:26 > Executing worker wrapper. Airbyte version: 0.63.8
2024-07-19 22:42:26 >
2024-07-19 22:42:26 > ----- START CHECK -----
2024-07-19 22:42:26 > Using default value for environment variable SIDECAR_KUBE_CPU_LIMIT: '2.0'
2024-07-19 22:42:26 >
2024-07-19 22:42:26 > Using default value for environment variable SOCAT_KUBE_CPU_LIMIT: '2.0'
2024-07-19 22:42:26 > Using default value for environment variable SIDECAR_KUBE_CPU_REQUEST: '0.1'
2024-07-19 22:42:26 > Using default value for environment variable SOCAT_KUBE_CPU_REQUEST: '0.1'
2024-07-19 22:42:26 > Attempting to start pod = ion-redshift-check-e4d36442-c12f-4508-9f90-5cabe5dd428d-0-pohgl for airbyte/destination-redshift:3.3.0 with resources ConnectorResourceRequirements[main=io.airbyte.config.ResourceRequirements@614ec5a8[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=,additionalProperties={}], heartbeat=io.airbyte.config.ResourceRequirements@1629527e[cpuRequest=0.1,cpuLimit=2.0,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}], stdErr=io.airbyte.config.ResourceRequirements@2b613a1c[cpuRequest=0.25,cpuLimit=2,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}], stdIn=io.airbyte.config.ResourceRequirements@10d83c31[cpuRequest=0.1,cpuLimit=2.0,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}], stdOut=io.airbyte.config.ResourceRequirements@10d83c31[cpuRequest=0.1,cpuLimit=2.0,memoryRequest=25Mi,memoryLimit=50Mi,additionalProperties={}]] and allowedHosts null
2024-07-19 22:42:26 > ion-redshift-check-e4d36442-c12f-4508-9f90-5cabe5dd428d-0-pohgl stdoutLocalPort = 9032
2024-07-19 22:42:26 > ion-redshift-check-e4d36442-c12f-4508-9f90-5cabe5dd428d-0-pohgl stderrLocalPort = 9033
2024-07-19 22:42:26 > Creating stdout socket server...
2024-07-19 22:42:26 > Creating stderr socket server...
2024-07-19 22:42:26 > Creating pod ion-redshift-check-e4d36442-c12f-4508-9f90-5cabe5dd428d-0-pohgl...
2024-07-19 22:42:26 > Waiting for init container to be ready before copying files...
2024-07-19 22:42:27 > Init container ready..
2024-07-19 22:42:27 > Copying files...
2024-07-19 22:42:27 > Uploading file: source_config.json
2024-07-19 22:42:27 > kubectl cp /tmp/89ea3e80-91d9-450d-94b0-0a22bd07e238/source_config.json airbyte/ion-redshift-check-e4d36442-c12f-4508-9f90-5cabe5dd428d-0-pohgl:/config/source_config.json -c init --retries=3
2024-07-19 22:42:27 > Waiting for kubectl cp to complete
2024-07-19 22:42:27 > kubectl cp complete, closing process
2024-07-19 22:42:27 > Uploading file: FINISHED_UPLOADING
2024-07-19 22:42:27 > kubectl cp /tmp/f30854f6-0326-4de7-8a07-422ce4b229d2/FINISHED_UPLOADING airbyte/ion-redshift-check-e4d36442-c12f-4508-9f90-5cabe5dd428d-0-pohgl:/config/FINISHED_UPLOADING -c init --retries=3
2024-07-19 22:42:27 > Waiting for kubectl cp to complete
2024-07-19 22:42:27 > kubectl cp complete, closing process
2024-07-19 22:42:27 > Waiting until pod is ready...
2024-07-19 22:42:28 > Setting stdout...
2024-07-19 22:42:28 > Setting stderr...
2024-07-19 22:42:29 > Reading pod IP...
2024-07-19 22:42:29 > Pod IP: 10.2.45.110
2024-07-19 22:42:29 > Using null stdin output stream...
2024-07-19 22:42:29 > Reading messages from protocol version 0.2.0
2024-07-19 22:42:29 > INFO main i.a.i.d.r.RedshiftDestination$Companion(main):514 starting destination: class io.airbyte.integrations.destination.redshift.RedshiftDestination
2024-07-19 22:42:29 > INFO main i.a.c.i.b.IntegrationCliParser$Companion(parseOptions):144 integration args: {check=null, config=source_config.json}
2024-07-19 22:42:29 > INFO main i.a.c.i.b.IntegrationRunner(runInternal):124 Running integration: io.airbyte.cdk.integrations.base.ssh.SshWrappedDestination
2024-07-19 22:42:29 > INFO main i.a.c.i.b.IntegrationRunner(runInternal):125 Command: CHECK
2024-07-19 22:42:29 > INFO main i.a.c.i.b.IntegrationRunner(runInternal):126 Integration config: IntegrationConfig{command=CHECK, configPath='source_config.json', catalogPath='null', statePath='null'}
2024-07-19 22:42:30 > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword groups - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-07-19 22:42:30 > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword group - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-07-19 22:42:30 > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword order - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-07-19 22:42:30 > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-07-19 22:42:30 > WARN main c.n.s.JsonMetaSchema(newValidator):278 Unknown keyword display_type - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2024-07-19 22:42:30 > INFO main i.a.c.i.b.s.SshTunnel$Companion(getInstance):423 Starting connection with method: NO_TUNNEL
2024-07-19 22:42:30 > INFO main i.a.c.i.d.s.S3DestinationConfig(createS3Client):112 Creating S3 client...
2024-07-19 22:42:30 > INFO main i.a.c.i.d.s.S3BaseChecks(testIAMUserHasListObjectPermission):151 Started testing if IAM user can call listObjects on the destination bucket
2024-07-19 22:42:30 > INFO main i.a.c.i.d.s.S3BaseChecks(testIAMUserHasListObjectPermission):154 Finished checking for listObjects permission
2024-07-19 22:42:30 > INFO main c.z.h.HikariDataSource(<init>):79 HikariPool-1 - Starting...
2024-07-19 22:42:30 > WARN main c.z.h.u.DriverDataSource(<init>):68 Registered driver with driverClassName=com.amazon.redshift.jdbc.Driver was not found, trying direct instantiation.
2024-07-19 22:42:30 > INFO main c.z.h.HikariDataSource(<init>):81 HikariPool-1 - Start completed.```