Airbyte Netsuite Connector 400 Client Error

AGanier · October 19, 2022, 3:18pm

Is this your first time deploying Airbyte?: No
OS Version / Instance: Amazon Linux version 2
Memory / Disk:
Deployment: Docker to AWS EC2
Airbyte Version: 40.10
Source name/version: Netsuite 0.1.1
Destination name/version: S3
Step: We are following the provided documentation for setting up the Netsuite Connector. When we attempt step 3: Set up Source Connection in Airbyte OSS
Description: Netsuite logs show successful authentication with access token. But Airbyte connection test fails showing the error below. Can anyone provide clarity on how to resolve this error?

ERROR:

The connection tests failed.HTTPError('400 Client Error: Bad Request for url: https://{accountId}.suitetalk.api.netsuite.com/services/rest/record/v1/contact?limit=1')

PERMISSIONS IN NETSUITE INCLUDE (BUT ARE NOT LIMITED TO):

Login Using Access Tokens (FULL)
REST Web Services (FULL)
SOAP Web Services (FULL)

natalyjazzviolin · October 19, 2022, 4:36pm

Hi @AGanier! I see you’ve posted another topic to the forum as well. Let me read over everything and I’ll get back to you in a bit!

natalyjazzviolin · October 19, 2022, 4:42pm

Could you please give me the full Airbyte logs? This indeed sounds odd, but I’d need more info to help you figure this out!

AGanier · October 19, 2022, 6:30pm

ALL AVAILABLE LOGS

2022-10-19 18:21:05 INFO i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/d8b3b0b3-7b6b-4009-88cf-4af1f0699202/0/logs.log

2022-10-19 18:21:05 INFO i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.40.10

2022-10-19 18:21:05 INFO i.a.c.i.LineGobbler(voidCall):114 -

2022-10-19 18:21:05 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK -----

2022-10-19 18:21:05 INFO i.a.c.i.LineGobbler(voidCall):114 -

2022-10-19 18:21:05 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/source-netsuite:0.1.1 exists...

2022-10-19 18:21:05 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/source-netsuite:0.1.1 was found locally.

2022-10-19 18:21:05 INFO i.a.w.p.DockerProcessFactory(create):119 - Creating docker container = source-netsuite-check-d8b3b0b3-7b6b-4009-88cf-4af1f0699202-0-jlnhf with resources io.airbyte.config.ResourceRequirements@746582d2[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=]

2022-10-19 18:21:05 INFO i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/d8b3b0b3-7b6b-4009-88cf-4af1f0699202/0 --log-driver none --name source-netsuite-check-d8b3b0b3-7b6b-4009-88cf-4af1f0699202-0-jlnhf --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e WORKER_JOB_ATTEMPT=0 -e WORKER_CONNECTOR_IMAGE=airbyte/source-netsuite:0.1.1 -e AIRBYTE_VERSION=0.40.10 -e WORKER_JOB_ID=d8b3b0b3-7b6b-4009-88cf-4af1f0699202 airbyte/source-netsuite:0.1.1 check --config source_config.json

2022-10-19 18:21:08 ERROR i.a.w.i.DefaultAirbyteStreamFactory(internalLog):99 - Check failed

2022-10-19 18:21:08 INFO i.a.w.t.TemporalAttemptExecution(get):134 - Stopping cancellation check scheduling...

2022-10-19 18:21:08 INFO i.a.c.i.LineGobbler(voidCall):114 -

2022-10-19 18:21:08 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK -----

2022-10-19 18:21:08 INFO i.a.c.i.LineGobbler(voidCall):114 -
2022-10-19 18:19:52 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/web_backend/connections/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-19 18:19:56 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/sources/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-19 18:19:57 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 GET 200 /api/v1/health
2022-10-19 18:19:58 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/destinations/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-19 18:19:58 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/source_definitions/list_latest
2022-10-19 18:19:58 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/source_definitions/list
2022-10-19 18:20:02 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/source_definition_specifications/get - {"sourceDefinitionId":"4f2f093d-ce44-4121-8118-9d13b7bfccd0","workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-19 18:20:17 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 GET 200 /api/v1/health
2022-10-19 18:20:38 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 GET 200 /api/v1/health
2022-10-19 18:20:57 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 GET 200 /api/v1/health
2022-10-19 18:21:08 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/scheduler/sources/check_connection - {"connectionConfiguration":"REDACTED","sourceDefinitionId":"4f2f093d-ce44-4121-8118-9d13b7bfccd0"}
2022-10-19 18:21:17 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 GET 200 /api/v1/health
2022-10-19 18:21:38 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 GET 200 /api/v1/health
2022-10-19 18:21:45 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/source_definitions/list_latest
2022-10-19 18:21:45 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/destination_definitions/list
2022-10-19 18:21:45 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/destination_definitions/list_latest
2022-10-19 18:21:45 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/source_definitions/list
2022-10-19 18:21:49 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.18.0.7 POST 200 /api/v1/logs/get - {"logType":"server"}

natalyjazzviolin · October 19, 2022, 7:05pm

Could you please update your Airbyte version to latest? I’m not able to reproduce this issue, so I think there is something wrong either with your Airbyte instance or with how things are setup. You’re hosting Airbyte locally on Docker?

AGanier · October 25, 2022, 7:37pm

In addition to following advice on upgrading to the most recent version, I’ve been reviewing the configs in Netsuite to determine if I made an error there.

Based on this thread in GitHub I’m thinking that the Integration config in Netsuite may need SuiteAnalytics Connect selected under Auth 2.0. However, in order to check that box I also need to provide a redirect URI but I don’t know what that value should be.

natalyjazzviolin · October 26, 2022, 5:01pm

A user recently had problems with the Netsuite connector as well and found that a piece of information was missing in the docs, no.5 form this list:

Step 2.2: Enable features

Go to Setup » Company » Enable Features
Click on SuiteCloud tab
Scroll down to Manage Authentication section
Enable checkbox TOKEN-BASED AUTHENTICATION
Enable checkbox REST WEB SERVICES ← missing in docs!
Save changes

Could you try this and see if it fixes the 400 client error?

AGanier · October 26, 2022, 6:24pm

@natalyjazzviolin

Thanks for the follow up! We’ve got both REST and SOAP enabled already, so that is likely not the Netsuite configuration prompting the error.

AGanier · October 27, 2022, 6:33pm

@natalyjazzviolin

Update: We upgraded to the latest version of Airbyte 0.40.17 and the Connector 0.1.1 but are still getting the same 400 error.

Server logs are below. Most show 200s but you may want to call someone’s attention to: BeanExecutable(getParameters):216 - HV000254: Missing parameter metadata for LogType(String, int, String), which declares implicit or synthetic parameters. Automatic resolution of generic type information for method parameters may yield incorrect results if multiple parameters have the same erasure. To solve this, compile your code with the '-parameters' flag.

Thanks!
Andréa

2022-10-27 18:26:42 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/logs/get - {"logType":"server"}
2022-10-27 18:26:43 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/logs/get - {"logType":"scheduler"}
2022-10-27 18:26:44 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:27:04 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:27:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/sources/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-27 18:27:13 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/web_backend/connections/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-27 18:27:13 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/sources/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-27 18:27:15 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list_latest
2022-10-27 18:27:15 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list
2022-10-27 18:27:20 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definition_specifications/get - {"sourceDefinitionId":"4f2f093d-ce44-4121-8118-9d13b7bfccd0","workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-27 18:27:24 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:27:44 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:28:04 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:28:11 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/scheduler/sources/check_connection - {"connectionConfiguration":"REDACTED","workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817","sourceDefinitionId":"4f2f093d-ce44-4121-8118-9d13b7bfccd0"}
2022-10-27 18:28:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list_latest
2022-10-27 18:28:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/destination_definitions/list
2022-10-27 18:28:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/destination_definitions/list_latest
2022-10-27 18:28:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list
2022-10-27 18:28:14 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/logs/get - {"logType":"server"}

2022-10-27 18:26:37 e[32mINFOe[m i.a.s.Requ... by Greg Smith
Greg Smith1:30 PM
2022-10-27 18:26:37 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list_latest
2022-10-27 18:26:37 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/destination_definitions/list
2022-10-27 18:26:37 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/destination_definitions/list_latest
2022-10-27 18:26:37 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list
2022-10-27 18:26:42 e[33mWARNe[m o.h.v.i.p.j.JavaBeanExecutable(getParameters):216 - HV000254: Missing parameter metadata for LogType(String, int, String), which declares implicit or synthetic parameters. Automatic resolution of generic type information for method parameters may yield incorrect results if multiple parameters have the same erasure. To solve this, compile your code with the '-parameters' flag.
2022-10-27 18:26:42 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/logs/get - {"logType":"server"}
2022-10-27 18:26:43 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/logs/get - {"logType":"scheduler"}
2022-10-27 18:26:44 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:27:04 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:27:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/sources/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-27 18:27:13 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/web_backend/connections/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-27 18:27:13 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/sources/list - {"workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-27 18:27:15 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list_latest
2022-10-27 18:27:15 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list
2022-10-27 18:27:20 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definition_specifications/get - {"sourceDefinitionId":"4f2f093d-ce44-4121-8118-9d13b7bfccd0","workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817"}
2022-10-27 18:27:24 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:27:44 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:28:04 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 GET 200 /api/v1/health
2022-10-27 18:28:11 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/scheduler/sources/check_connection - {"connectionConfiguration":"REDACTED","workspaceId":"0c6b710d-78b3-4eeb-87f6-bc57b3368817","sourceDefinitionId":"4f2f093d-ce44-4121-8118-9d13b7bfccd0"}
2022-10-27 18:28:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list_latest
2022-10-27 18:28:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/destination_definitions/list
2022-10-27 18:28:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/destination_definitions/list_latest
2022-10-27 18:28:12 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/source_definitions/list
2022-10-27 18:28:14 e[32mINFOe[m i.a.s.RequestLogger(filter):112 - REQ 172.19.0.3 POST 200 /api/v1/logs/get - {"logType":"server"}

has context menu

i-zaykov · November 10, 2022, 2:21pm

@AGanier were you able to solve this issue? I’m also getting the 401 error

AGanier · November 10, 2022, 4:33pm

Update:

Because we tweaked some network configurations (NAT gateway and internet gateway on AWS) to resolve some security concerns, we retried the Airbyte Netsuite connector again today. We are still getting the same error. As the screenshot below shows, we also tried passing an object type.

Thanks!
Andréa

god830 · January 3, 2023, 1:22am

Hi,

Is there any update on this?

AGanier · January 5, 2023, 4:21pm

We’ve retried the connector after upgrading and the connection error persists

natalyjazzviolin · January 18, 2023, 3:30pm

Hi! So sorry to hear the issue persists! The error code is different though - 401 error indicates wrong authorization credentials. Could you double check the credentials you are using?

AGanier · January 24, 2023, 6:28pm

We’ve retried the connector again today after affirming our setup in Netsuite. Still getting a 401 client error.

2023-01-24 18:24:25 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- START CHECK -----2023-01-24 18:24:25 INFO i.a.c.i.LineGobbler(voidCall):114 -2023-01-24 18:24:25 INFO i.a.c.i.LineGobbler(voidCall):114 - Checking if airbyte/source-netsuite:0.1.3 exists...2023-01-24 18:24:25 INFO i.a.c.i.LineGobbler(voidCall):114 - airbyte/source-netsuite:0.1.3 was found locally.2023-01-24 18:24:25 INFO i.a.w.p.DockerProcessFactory(create):120 - Creating docker container = source-netsuite-check-aef3c0fe-a151-4521-b31e-ec4bbd48ca3a-0-tqoal with resources io.airbyte.config.ResourceRequirements@472beb2b[cpuRequest=,cpuLimit=,memoryRequest=,memoryLimit=]2023-01-24 18:24:25 INFO i.a.w.p.DockerProcessFactory(create):164 - Preparing command: docker run --rm --init -i -w /data/aef3c0fe-a151-4521-b31e-ec4bbd48ca3a/0 --log-driver none --name source-netsuite-check-aef3c0fe-a151-4521-b31e-ec4bbd48ca3a-0-tqoal --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e DEPLOYMENT_MODE=OSS -e FIELD_SELECTION_WORKSPACES= -e USE_STREAM_CAPABLE_STATE=true -e AIRBYTE_ROLE= -e WORKER_ENVIRONMENT=DOCKER -e APPLY_FIELD_SELECTION=false -e WORKER_CONNECTOR_IMAGE=airbyte/source-netsuite:0.1.3 -e WORKER_JOB_ATTEMPT=0 -e AUTO_DETECT_SCHEMA=false -e AIRBYTE_VERSION=0.40.28 -e WORKER_JOB_ID=aef3c0fe-a151-4521-b31e-ec4bbd48ca3a airbyte/source-netsuite:0.1.3 check --config source_config.json2023-01-24 18:24:25 INFO i.a.w.i.VersionedAirbyteStreamFactory(create):100 - Reading messages from protocol version 0.2.02023-01-24 18:24:28 ERROR i.a.w.i.DefaultAirbyteStreamFactory(internalLog):116 - Check failed2023-01-24 18:24:28 INFO i.a.w.g.DefaultCheckConnectionWorker(run):110 - Check connection job received output: io.airbyte.config.StandardCheckConnectionOutput@40726542[status=failed,message=HTTPError('401 Client Error: Unauthorized for url: https://4811579.suitetalk.api.netsuite.com/services/rest/record/v1/contact?limit=1')]2023-01-24 18:24:28 INFO i.a.c.i.LineGobbler(voidCall):114 -2023-01-24 18:24:28 INFO i.a.w.t.TemporalAttemptExecution(get):163 - Stopping cancellation check scheduling...2023-01-24 18:24:28 INFO i.a.c.i.LineGobbler(voidCall):114 - ----- END CHECK -----```

AGanier · January 26, 2023, 8:41pm

Great News! Today we were able to successfully connect to the Airbyte Netsuite Connector after re-generating the access token.

natalyjazzviolin · January 30, 2023, 8:01pm

That is amazing to hear!

Topic		Replies	Views
SP-API Sandbox 400 Client Error: Bad Request for url Connector Questions & Issues getting-started , connectors	1	302	March 22, 2023
Error setting up source connection to Netsuite Connector Development connector , error , question , source-connection , netsuite	1	45	May 16, 2024
Netsuite source connector Date Issue Connector Questions & Issues connectors	3	622	September 22, 2022
Netsuite Source: Error fetching schema non json response Connector Questions & Issues connectors , schema	1	414	May 22, 2023
Error Testing S3 Connector on Airbyte OSS with Docker on EC2 Connector Questions docker , connector , airbyte-oss , bug , ec2	0	46	May 14, 2024

Airbyte Netsuite Connector 400 Client Error

Related topics